Plastic shell defect detection model training method, plastic shell defect detection method and device

Abstract

The application provides a plastic shell defect detection model training method, a plastic shell defect detection method and device. The plastic shell defect detection model is used for detecting plastic shell defects in a plug-in assembly unit. The plug-in assembly unit includes a copper piece and a plastic shell arranged around the copper piece. The plastic shell defect detection model training method includes: obtaining a copper piece region training set. The copper piece region training set includes multiple copper piece region images and corresponding first classification labels. The first classification labels include copper piece region defect categories and copper piece region non-defect categories. A small sample reference set is obtained. The small sample reference set includes multiple plug-in assembly unit images and corresponding second classification labels. The second classification labels include plastic shell defect categories and plastic shell non-defect categories. A pre-training model is subjected to small sample learning according to the copper piece region training set and the small sample reference set, and a plastic shell defect detection model is obtained. The application can improve the accuracy of plastic shell defect detection.

Description

TECHNICAL FIELD

[0001] The present application mainly relates to the technical field of image processing, in particular to a training method of a plastic shell defect detection model, a plastic shell defect detection method and device. BACKGROUND

[0002] The 86 type socket is widely used in production and life, and in the intermediate link of the production of the 86 type socket, an LNE socket assembly unit is assembled, the LNE socket assembly unit includes a copper piece and a plastic shell, and the LNE socket assembly unit needs to be detected for defects, the types of defect detection include missed detection, misassembly of the copper piece, and damage of the plastic shell, etc., wherein the damage defect of the plastic shell is difficult to detect due to the low frequency in the production process, and the damaged picture is similar to the normal picture after being captured by a camera. At present, the defect detection system applied in the industrial field mainly uses a CCD camera to capture pictures, and detects and judges based on the pictures. There are two main directions of methods: one is a traditional method based on template matching, taking a normal picture as a reference template, and distinguishing and judging by using the characteristics that the local positions of a defect picture and a normal picture are different and similar, and for the damage defect of the plastic shell of the LNE socket assembly unit, due to the reasons that the assembly may rotate to a certain extent and the color features of the plastic shell of some models are similar before and after damage, the template matching method cannot be used for good detection of the damage defect of the plastic shell. The other direction is based on deep learning, taking normal samples and defect samples as a training set to train a deep learning classification model for prediction and judgment, but the deep learning classification training has certain requirements for the data volume of positive and negative samples, and due to the small number of plastic shell damage samples, the prediction accuracy of the deep learning classification model directly trained is low.

[0003] That is, the accuracy of the plastic shell defect detection in the prior art is low. SUMMARY

[0004] The present application provides a training method of a plastic shell defect detection model, a plastic shell defect detection method and device, aiming at solving the problem of low accuracy of the plastic shell defect detection in the prior art.

[0005] In a first aspect, the present application provides a training method of a plastic shell defect detection model, the plastic shell defect detection model being used for detecting a plastic shell defect in a socket assembly unit, the socket assembly unit including a copper piece and a plastic shell arranged around the copper piece, and the training method of the plastic shell defect detection model including:

[0006] obtaining a copper piece region training set, the copper piece region training set including a plurality of copper piece region pictures and corresponding first classification labels, the first classification labels including a copper piece region defect category and a copper piece region non-defect category;

[0007] Obtaining a small sample reference set, the small sample reference set comprising a plurality of plug-in assembly unit pictures and corresponding second classification labels, the second classification labels comprising a defective shell category and a non-defective shell category;

[0008] According to the copper piece region training set and the small sample reference set, a pre-trained model is small sample learned to obtain a shell defect detection model.

[0009] Optionally, the small sample learning of the pre-trained model according to the copper piece region training set and the small sample reference set to obtain the shell defect detection model comprises:

[0010] According to the copper piece region training set, an image classification training is performed on a pre-trained model to obtain a network weight model;

[0011] According to the small sample reference set, a small sample training is performed on the network weight model to obtain a shell defect detection model.

[0012] Optionally, the copper piece region training set comprises:

[0013] Obtaining an original picture set of the plug-in assembly unit;

[0014] Positioning a copper piece region in each original picture in the original picture set;

[0015] According to the copper piece region, first annotation information of each copper piece region is determined;

[0016] According to each first annotation information, a corresponding original picture is segmented to obtain a copper piece region training set.

[0017] Optionally, the segmentation of the corresponding original picture according to each first annotation information to obtain the copper piece region training set comprises:

[0018] Positioning a plug-in assembly unit overall region in each original picture in the original picture set;

[0019] According to the plug-in assembly unit overall region, second annotation information of each plug-in assembly unit overall region is determined;

[0020] Using the second annotation information, the plug-in assembly unit overall region in the original picture is segmented to obtain a plug-in assembly unit picture;

[0021] Using the first annotation information, the plug-in assembly unit picture is segmented to obtain a copper piece region picture.

[0022] Optionally, the positioning of the copper piece region in each original picture in the original picture set comprises:

[0023] A target detection weight model is used to locate a copper piece region in each original picture in the original picture set.

[0024] Optionally, the small sample training of the network weight model according to the small sample reference set comprises:

[0025] The network weight model is small sample trained according to the small sample reference set to obtain a classification small sample weight model.

[0026] The target detection weight model and the classification small sample weight model are tested on a preset verification set to obtain test question data.

[0027] The test question data is added to the small sample reference set, and the classification small sample weight model is trained to obtain a plastic shell defect detection model.

[0028] In a second aspect, the present application provides a plastic shell defect detection method, which comprises:

[0029] An image of a socket to be identified is obtained.

[0030] The image of the socket to be identified is input into a plastic shell defect detection model to obtain a plastic shell defect detection result, the plastic shell defect detection result being a plastic shell defect category or a plastic shell non-defect category, and the plastic shell defect detection model being the plastic shell defect detection model of any one of the first aspect.

[0031] In a third aspect, the present application provides a training device of a plastic shell defect detection model, the plastic shell defect detection model being used to detect a plastic shell defect in a socket assembly unit, the socket assembly unit comprising a copper piece and a plastic shell arranged around the copper piece, and the training device of the plastic shell defect detection model comprising:

[0032] A first obtaining unit is configured to obtain a copper piece region training set, the copper piece region training set comprising a plurality of copper piece region pictures and corresponding first classification labels, the first classification labels comprising a copper piece region defect category and a copper piece region non-defect category.

[0033] A second obtaining unit is configured to obtain a small sample reference set, the small sample reference set comprising a plurality of socket assembly unit pictures and corresponding second classification labels, the second classification labels comprising a plastic shell defect category and a plastic shell non-defect category.

[0034] A model training unit is configured to perform small sample learning on a pre-training model according to the copper piece region training set and the small sample reference set to obtain a plastic shell defect detection model.

[0035] Optionally, the model training unit is configured to:

[0036] The pre-trained model is trained for image classification based on the training set of the copper parts region to obtain the network weight model;

[0037] The network weight model is trained using the small sample reference set to obtain the plastic shell defect detection model.

[0038] Optionally, the first acquisition unit is configured to:

[0039] Obtain the original image set of the socket assembly unit;

[0040] The copper component area is located in each original image of the original image set.

[0041] Determine the first annotation information for each of the copper component regions based on the copper component regions;

[0042] The original image is segmented according to each of the first annotation information to obtain the training set of the copper part region.

[0043] Optionally, the first acquisition unit is configured to:

[0044] The overall area of ​​each original image positioning socket assembly unit in the original image set;

[0045] The second labeling information for each overall area of ​​the socket assembly unit is determined based on the overall area of ​​the socket assembly unit;

[0046] The entire area of ​​the plug-in assembly unit in the original image is segmented using the second annotation information to obtain an image of the plug-in assembly unit;

[0047] The copper component area image is obtained by segmenting the image of the socket assembly unit using the first annotation information.

[0048] Optionally, the first acquisition unit is configured to:

[0049] The copper component region is located in each original image of the original image set using an object detection weight model.

[0050] Optionally, the model training unit is used for:

[0051] The network weight model is trained using the small sample reference set to obtain a classification small sample weight model.

[0052] The target detection weight model and the classification few-sample weight model are tested on a preset validation set to obtain test question data;

[0053] The test problem data is added to the small sample reference set, and the classification small sample weight model is trained to obtain the plastic shell defect detection model.

[0054] Fourthly, this application provides a plastic shell defect detection device for detecting plastic shell defects in a sleeve assembly unit, the sleeve assembly unit including a copper component and a plastic shell disposed around the copper component, the plastic shell defect detection device comprising:

[0055] Image acquisition unit, used to acquire image of the socket to be identified;

[0056] The defect detection unit is used to input the image of the socket to be identified into the plastic shell defect detection model to obtain the plastic shell defect detection result. The plastic shell defect detection result is either a plastic shell defect category or a plastic shell no-defect category. The plastic shell defect detection model is any one of the plastic shell defect detection models described in the first aspect.

[0057] Fifthly, this application provides a computer device, the computer device comprising:

[0058] One or more processors;

[0059] Memory; and

[0060] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the training method of the plastic shell defect detection model as described in any one of the first aspects or the plastic shell defect detection method as described in any one of the second aspects.

[0061] Sixthly, this application provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to perform the steps of the training method for the plastic shell defect detection model as described in the first aspect or the plastic shell defect detection method as described in the second aspect.

[0062] This application provides a training method, a method for detecting plastic shell defects, and an apparatus for detecting plastic shell defects. The plastic shell defect detection model is used to detect defects in plastic shells within a sleeve assembly unit. The sleeve assembly unit includes a copper component and a plastic shell surrounding the copper component. The training method for the plastic shell defect detection model includes: acquiring a training set of the copper component region, which includes multiple images of the copper component region and corresponding first classification labels, including categories for defective and non-defective copper components; acquiring a small sample reference set, which includes multiple images of sleeve assembly units and corresponding second classification labels, including categories for defective and non-defective plastic shells; and performing small sample learning on the pre-trained model based on the copper component region training set and the small sample reference set to obtain the plastic shell defect detection model. This application does not directly use a small sample reference set to train the neural network model. Instead, it combines the copper part region training set and the small sample reference set to perform small sample learning on the model and uses a pre-trained model. This avoids the problem of low model prediction accuracy caused by directly using a small sample reference set to train the neural network model. It can improve the prediction accuracy of the trained plastic shell defect detection model, thereby improving the accuracy of plastic shell defect detection. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a schematic diagram of a scenario for the plastic shell defect detection system provided in an embodiment of this application;

[0065] Figure 2 This is a schematic flowchart of an embodiment of the training method for the plastic shell defect detection model provided in this application.

[0066] Figure 3 This is a schematic diagram of the process for obtaining a training set of the copper part region in one embodiment of the training method for the plastic shell defect detection model provided in this application.

[0067] Figure 4 This is a flowchart illustrating the training method of the plastic shell defect detection model provided in this application, which performs small-sample learning on the pre-trained model based on the copper part region training set and the small-sample reference set to obtain the plastic shell defect detection model.

[0068] Figure 5 This is a schematic diagram of the original image of the plastic shell with defect categories in one embodiment of the training method of the plastic shell defect detection model provided in this application;

[0069] Figure 6 This is a schematic diagram of the copper part area and the overall area of ​​the socket assembly unit marked in the original image in one embodiment of the training method for the plastic shell defect detection model provided in this application.

[0070] Figure 7 This is a schematic diagram of the network structure of the YOLOv6-n model;

[0071] Figure 8 This is a schematic flowchart of an embodiment of the plastic shell defect detection method provided in this application.

[0072] Figure 9 This is a schematic diagram of an embodiment of the training device for the plastic shell defect detection model provided in this application.

[0073] Figure 10 This is a schematic diagram of an embodiment of the plastic shell defect detection device provided in this application.

[0074] Figure 11 This is a schematic diagram of an embodiment of the computer device provided in this application. Detailed Implementation

[0075] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0076] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0077] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0078] This application provides a training method for a plastic shell defect detection model, a plastic shell defect detection method, and an apparatus, which will be described in detail below.

[0079] Please see Figure 1 , Figure 1 This is a schematic diagram of a plastic shell defect detection system provided in an embodiment of this application. The plastic shell defect detection system may include a computer device 100.

[0080] In this embodiment, the computer device 100 can be a standalone server, a server network, or a server cluster. For example, the computer device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0081] In this embodiment, the computer device 100 described above can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device 100 can be a desktop computer, a portable computer, a network server, a handheld computer (Personal Digital Assistant, PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, etc. This embodiment does not limit the image category of the computer device 100.

[0082] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include more than one application scenario. Figure 1 The number of computer devices shown is more or less, for example Figure 1Only one computer device is shown in the diagram. It is understood that the plastic shell defect detection system may also include one or more other computer devices capable of processing data, which are not specifically limited here.

[0083] In addition, such as Figure 1 As shown, the plastic shell defect detection system may also include a memory 200 for storing data.

[0084] It should be noted that, Figure 1 The schematic diagram of the plastic shell defect detection system shown is merely an example. The plastic shell defect detection system and scenario described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of plastic shell defect detection systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0085] First, this application provides a training method for a plastic shell defect detection model. The training method includes: acquiring a copper part region training set, which includes multiple images of copper part regions and corresponding first classification labels, the first classification labels including a defective category and a defect-free category for the copper part region; acquiring a small sample reference set, which includes multiple images of socket assembly units and corresponding second classification labels, the second classification labels including a defective category and a defect-free category for the plastic shell; and performing small sample learning on the pre-trained model based on the copper part region training set and the small sample reference set to obtain the plastic shell defect detection model.

[0086] like Figure 2 As shown, Figure 2 This is a schematic flowchart of an embodiment of the training method for the plastic shell defect detection model provided in this application. The training method for the plastic shell defect detection model includes the following steps S201 to S203:

[0087] S201. Obtain the training set of the copper component area.

[0088] In this embodiment, the socket assembly unit includes a copper component and a plastic shell surrounding the copper component. The socket assembly unit can be an x86 type socket, an x100 type socket, etc., depending on the specific requirements. This application uses an x86 type socket as an example for illustration.

[0089] The training set for the copper component area includes multiple images of the copper component area and corresponding first-class labels. The first-class labels include categories for "defective" and "defect-free" copper component areas. The multiple images of the copper component area in the training set can be captured by a camera, and the first-class labels can be labeled by machine or manually. A copper component area can be the bounding rectangle of the area containing the copper component. If the copper component area has defects such as missing or incorrectly assembled parts, the first-class label is "defective." If the copper component area is defect-free, the first-class label is "defect-free."

[0090] In this embodiment, original images of the socket assembly unit of the 86-type socket are collected to obtain an original image set of the socket assembly unit. The original image set includes normal images (OK) of the copper part area without defects and original images (NG) of the copper part area with defects. The original image set also includes normal images (OK) of the plastic shell without defects and images (NG) of the plastic shell with defects. The copper part areas of each original image in the original image set are cropped to obtain multiple copper part areas and corresponding first classification labels. The entire area of ​​the socket assembly unit in each original image in the original image set is cropped to obtain multiple socket assembly unit images.

[0091] like Figure 5 As shown, Figure 5 This is a schematic diagram of the original image of the defective plastic shell category in this application. Figure 5 The damaged plastic casing frame 21 is marked in the middle. (For example...) Figure 6 As shown, Figure 6 This is a schematic diagram of the copper component area and the overall area of ​​the socket assembly unit as marked in the original image of this application. Figure 6 The area marked is the copper component area 22 and the overall area 23 of the socket assembly unit. The area between the copper component area 22 and the overall area 23 of the socket assembly unit is the plastic shell to be identified.

[0092] S202. Obtain a small sample reference set.

[0093] In this embodiment, the small sample reference set includes multiple images of socket assembly units and corresponding second classification labels. The second classification labels include categories for defective plastic shells and categories for defect-free plastic shells. The second classification labels can be labeled by machine or manually.

[0094] In this embodiment, original images of the socket assembly unit of the 86-type socket are collected to obtain an original image set of the socket assembly unit. The original image set includes normal images (OK) of the plastic shell without defects and defective images (NG) of the plastic shell with broken defects. The entire area of ​​the socket assembly unit in each original image in the original image set is cropped to obtain multiple socket assembly unit images and corresponding second classification labels.

[0095] Furthermore, small sample data refers to data samples with a very small number of samples, generally meaning data samples with fewer than 50 original collected data (without augmentation or other data processing). Since the amount of sample data for some categories is enormous, simply counting by the number of samples is not very applicable. Therefore, in this embodiment, small sample data can also be data samples with a value lower than a preset proportion of the same type of data (the preset proportion is less than or equal to 1 / 10). For example, data of type A and type B both belong to the same type of label data, where the amount of data of type A is less than 50, or the amount of data of type A is less than 1 / 10 of the amount of data of type B. Small sample label types indicate uncommon label types.

[0096] Since there are few samples of plastic shell damage, the prediction effect of directly training a deep learning classification model is not good. Deep learning algorithms have difficulty learning and accurately identifying these categories with a small sample size. Therefore, the accuracy of the trained plastic shell defect detection model can be improved by using the small sample learning method.

[0097] In one specific embodiment, obtaining a small sample reference set includes: acquiring multiple original socket assembly unit images and their corresponding second classification labels. Pixels within the copper component areas of the multiple original socket assembly unit images are set to 0, resulting in multiple socket assembly unit images and their corresponding second classification labels.

[0098] S203. Based on the training set of the copper part area and the small sample reference set, perform small sample learning on the pre-trained model to obtain the plastic shell defect detection model.

[0099] For further details, please refer to [link / reference]. Figure 3 Obtain the training set of the copper component regions, including S301-S304:

[0100] S301. Obtain the original image set of the socket assembly unit.

[0101] Specifically, original images of the socket assembly unit of the 86-type socket were collected to obtain a set of original images of the socket assembly unit. The original image set includes normal images (OK) and images of defects such as broken plastic shells (NG).

[0102] S302. Locate the copper component area in each original image in the original image set.

[0103] In this embodiment of the application, the Labelimg annotation tool is used to represent the area where the main copper parts are located, i.e., the copper part area and the overall area of ​​the socket assembly unit, by drawing rectangles on the original image set, such as... Figure 6 As shown.

[0104] In this embodiment, an object detection weight model is obtained, and this model is used to locate the overall region of the socket assembly unit and the copper part region in each original image of the original image set. In a specific embodiment, obtaining the object detection weight model includes:

[0105] (1) Obtain the original image.

[0106] (2) Adjust the size of the original image to obtain the adjusted image.

[0107] (3) Input the adjusted image into the preset network model to predict the target location information.

[0108] The preset network model can be YOLOv6, Nanodet, MobileDets, MocroYOLO, etc. Preferably, the preset network model is YOLOv6-n.

[0109] The YOLOv6-n model consists of a backbone network, a neck, and a head. A schematic diagram of the YOLOv6-n model's network structure is shown below. Figure 7 As shown.

[0110] Backbone: An efficient, reparameterizable backbone, denoted as EfficientRep, is designed to decouple the multi-branch topology during training from the standard architecture during inference, achieving a better speed-accuracy tradeoff. The main component of the backbone is the RepBlock during training. Each RepBlock is transformed into a stack of 3×3 convolutional layers (denoted as RepConv) with ReLU activation functions during inference. Typically, 3×3 convolutions are highly optimized for mainstream GPUs and CPUs and offer higher computational density. Therefore, EfficientRep's backbone fully leverages the hardware's computational power, significantly reducing inference latency while enhancing representational capabilities.

[0111] Neck: The PAN structure with multi-scale feature fusion capabilities from a previous version of YOLO is used as the basis for the detection neck. The CSPBlock used in the PAN structure is replaced with RepBlock, and the width and depth are adjusted accordingly. The Neck of YOLOv6-n is represented as Rep-PAN.

[0112] Head: YOLOv6 employs a hybrid channel strategy to construct a more efficient decoupled head. Specifically, the number of intermediate 3×3 convolutional layers is reduced to only one. The head width is scaled by a multiplier of the widths of the backbone and neck. These modifications further reduce computational costs, resulting in lower inference latency.

[0113] (4) Calculate the loss value and prediction accuracy using the preset loss function and the annotation information of the original images obtained in advance.

[0114] The annotation information of the original image includes the overall area of ​​the socket assembly unit and the copper part area in the original image.

[0115] (5) Based on the preset optimizer, the weight parameters of the preset network model are updated slightly along the gradient direction. The forward propagation and backward propagation processes are repeated and iterated until the weight parameter model with the minimum loss value or the highest accuracy is obtained, and it is determined as the target detection weight model.

[0116] S303. Determine the first annotation information for each copper part area based on the copper part area.

[0117] Specifically, the copper parts area is marked in each original image to obtain the first annotation information for each copper parts area.

[0118] S304. Segment the corresponding original image according to each first annotation information to obtain the training set of the copper part region.

[0119] In one specific embodiment, the overall area of ​​the socket assembly unit is located in each original image in the original image set; the second annotation information of the overall area of ​​each socket assembly unit is determined according to the overall area of ​​the socket assembly unit; the overall area of ​​the socket assembly unit in the original image is segmented using the second annotation information to obtain the socket assembly unit image; the copper part area image is obtained by segmenting the socket assembly unit image using the first annotation information.

[0120] Specifically, the entire area of ​​the socket assembly unit is marked in each original image to obtain the second annotation information of the entire area of ​​each socket assembly unit; the copper part area is marked in each original image to obtain the first annotation information of each copper part area.

[0121] An XML-formatted annotation file is automatically generated based on the second annotation information of the overall area of ​​the socket assembly unit and the first annotation information of the copper component area. The second annotation information includes the annotation rectangle of the overall area of ​​the socket assembly unit in the original image; the first annotation information includes the annotation rectangle of the copper component area in the original image. The left endpoint of the annotation rectangle is represented as (x...). min ,y min ), the right endpoint is represented as (x max ,y max Based on the annotation detection boxes of each second annotation information, the corresponding original images are cropped to obtain multiple images of the plug-in assembly units; based on the annotation detection boxes of each first annotation information, the corresponding images of the plug-in assembly units are cropped to obtain images of the copper parts area.

[0122] For further details, please refer to [link / reference]. Figure 4 Based on the training set of the copper parts region and the small sample reference set, the pre-trained model is trained using small sample learning to obtain the plastic shell defect detection model, including S401-S402:

[0123] S401. Train the pre-trained model for image classification based on the training set of the copper parts region to obtain the network weight model.

[0124] Specifically, using Triplet Loss as the loss function, the pre-trained model is trained on the copper area training set for image classification to obtain the network weight model.

[0125] The pre-trained model can be a model trained on a general image training set. For example, it could be a model trained on the ImageNet dataset. The pre-trained model can be a Convolutional Neural Network (CNN), specifically, for example, the EfficientV2 network model. The pre-trained model can be replaced with other lightweight algorithms, such as ShuffleNetV2 or SqueezeNet.

[0126] Training the pre-trained model requires a large classification dataset, in which each image is labeled, and each class contains many images. Considering deployment requirements, the pre-trained model primarily uses the lightweight network model EfficientNetV2. The general image training set uses the ImageNet dataset.

[0127] When using the Triplet loss function to build the training set for the pre-trained model based on the ImageNet dataset, three images need to be selected from the ImageNet dataset each time. First, one image is randomly selected from the ImageNet dataset as an anchor. Then, another image is randomly selected from the category of the anchor image as a positive sample. Next, the category of the anchor image is excluded, and one image is randomly selected from the dataset as a negative sample. The anchor image, positive sample image, and negative sample image are then input into the constructed convolutional neural network used to extract image features, resulting in three feature vectors, f(+), f(a), and f(-). The squares of the L2 norms of f(+) and f(a), and f(-) and f(a), and d+ and d- are calculated respectively. The Triplet loss function is defined as follows:

[0128] Loss(x a ,x + ,x - )=max{0,d + +α-d-}

[0129] Here, α is called the margin, which is a hyperparameter. After determining the loss function, we can calculate the gradient of the loss function with respect to the model parameters and use stochastic gradient descent to update the model parameters.

[0130] S402. Train the network weight model with small samples based on the small sample reference set to obtain the plastic shell defect detection model.

[0131] In this embodiment of the application, the network weight model is trained using a small sample reference set to obtain a plastic shell defect detection model, which may include:

[0132] (1) The network weight model is trained with small samples based on the small sample reference set to obtain the classification small sample weight model.

[0133] The network weight model is used as the initial weight model. It is trained using a predetermined number of interlocking unit images on a small sample reference set. The normalized exponential function is used as the classifier. The cross-entropy loss function of all images in the small sample reference set is summed as the objective function and minimized by gradient descent. The weight parameters and biases of the network weight model are optimized to obtain the classification small sample weight model.

[0134] The preset number of images can be several or dozens. The initial weight model is obtained by combining an ImageNet pre-trained model and a copper region training set. Training is performed using a preset number of splice assembly unit images (several or dozens) and second-class labels from a small-sample reference set. Softmax is used as the classifier, and the objective function is the sum of the CrossEntropy values ​​of all samples in the small-sample reference set, minimized using gradient descent. This optimizes the weight parameters and biases in the Softmax classifier, resulting in the final small-sample classification weight model. Typically, the parameters W of the Softmax classifier are initialized to the reference set class representation matrix M (a vector matrix representing the class formed by the vectors of each class sample after feature extraction, meanization, and normalization), and b is initialized as an all-zero vector. Since the small-sample reference set is very small, a regularization term needs to be added to the objective function to prevent overfitting. In the field of small-sample learning, entropy regularization is a very suitable regularization method.

[0135] (2) Test the target detection weight model and the classification small sample weight model on the preset validation set to obtain test question data.

[0136] In this embodiment of the application, the test problem data includes sample images with incorrect positioning or incorrect category.

[0137] In one specific embodiment, multiple socket assembly unit images and their corresponding second classification labels are divided into a small sample reference set and a preset validation set according to a preset ratio. A data preprocessing script is then used to integrate the labeled small sample reference set and the preset validation set into two JSON files (train.json and validate.json). The preset ratio can be 9:1.

[0138] (3) Add the test problem data to the small sample reference set, train the classification small sample weight model, and obtain the plastic shell defect detection model.

[0139] For further details, please refer to [link / reference]. Figure 8 This application also provides a method for detecting defects in plastic shells, the method comprising S501-S502:

[0140] S501. Obtain the image of the socket to be identified.

[0141] S502. Input the image of the socket to be identified into the plastic shell defect detection model to obtain the plastic shell defect detection result.

[0142] The defect detection result for the plastic casing is categorized as either "defective" or "defect-free". The defect detection model for the plastic casing is any one of the defect detection models described in the above embodiments.

[0143] Furthermore, the image of the socket to be identified is input into the target detection weight model to obtain the overall area of ​​the socket assembly unit and the copper part area. The image of the overall area of ​​the socket assembly unit is cropped and the pixels of the copper part area are set to 0 to obtain the preprocessed socket image. The preprocessed socket image is input into the plastic shell defect detection model to obtain the plastic shell defect detection result.

[0144] To better implement the training method for the plastic shell defect detection model in the embodiments of this application, based on the training method for the plastic shell defect detection model, the embodiments of this application also provide a training device for the plastic shell defect detection model, such as... Figure 9 As shown, the training device for the plastic shell defect detection model includes:

[0145] The first acquisition unit 801 is used to acquire a copper part area training set, which includes multiple copper part area images and corresponding first classification labels. The first classification labels include copper part area defective category and copper part area non-defective category.

[0146] The second acquisition unit 802 is used to acquire a small sample reference set, which includes multiple images of socket assembly units and corresponding second classification labels. The second classification labels include categories of defective plastic shells and categories of defect-free plastic shells.

[0147] The model training unit 803 is used to perform small-sample learning on the pre-trained model based on the copper part area training set and the small-sample reference set to obtain the plastic shell defect detection model.

[0148] Optionally, the model training unit is used for:

[0149] The pre-trained model is trained for image classification based on the training set of the copper parts region to obtain the network weight model;

[0150] The network weight model is trained using the small sample reference set to obtain the plastic shell defect detection model.

[0151] Optionally, the first acquisition unit is configured to:

[0152] Obtain the original image set of the socket assembly unit;

[0153] The copper component area is located in each original image of the original image set.

[0154] Determine the first annotation information for each of the copper component regions based on the copper component regions;

[0155] The original image is segmented according to each of the first annotation information to obtain the training set of the copper part region.

[0156] Optionally, the first acquisition unit is configured to:

[0157] The overall area of ​​each original image positioning socket assembly unit in the original image set;

[0158] The second labeling information for each overall area of ​​the socket assembly unit is determined based on the overall area of ​​the socket assembly unit;

[0159] The entire area of ​​the plug-in assembly unit in the original image is segmented using the second annotation information to obtain an image of the plug-in assembly unit;

[0160] The copper component area image is obtained by segmenting the image of the socket assembly unit using the first annotation information.

[0161] Optionally, the first acquisition unit is configured to:

[0162] The copper component region is located in each original image of the original image set using an object detection weight model.

[0163] Optionally, the model training unit is used for:

[0164] The network weight model is trained using the small sample reference set to obtain a classification small sample weight model.

[0165] The target detection weight model and the classification few-sample weight model are tested on a preset validation set to obtain test question data;

[0166] The test problem data is added to the small sample reference set, and the classification small sample weight model is trained to obtain the plastic shell defect detection model.

[0167] To better implement the plastic shell defect detection method in the embodiments of this application, based on the plastic shell defect detection method, the embodiments of this application also provide a plastic shell defect detection device, such as... Figure 10 As shown, the plastic shell defect detection device includes:

[0168] Image acquisition unit 901 is used to acquire an image of the socket to be identified;

[0169] The defect detection unit 902 is used to input the image of the socket to be identified into the plastic shell defect detection model to obtain the plastic shell defect detection result. The plastic shell defect detection result is either a plastic shell defect category or a plastic shell no-defect category. The plastic shell defect detection model is any one of the plastic shell defect detection models described in the first aspect.

[0170] This application also provides a computer device that integrates a training device or a plastic shell defect detection device for any of the plastic shell defect detection models provided in this application. The computer device includes:

[0171] One or more processors;

[0172] Memory; and

[0173] One or more applications, wherein the applications are stored in memory and configured to be executed by a processor as steps of the training method for the plastic shell defect detection model in any of the embodiments described above.

[0174] like Figure 11 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:

[0175] The computer device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will understand that the computer device structure shown in the figures does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0176] Processor 601 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in memory 602, and by calling data stored in memory 602, thereby providing overall monitoring of the computer device. Optionally, processor 601 may include one or more processing cores; processor 601 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Preferably, processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may not be integrated into processor 601.

[0177] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0178] The computer device also includes a power supply 603 that supplies power to the various components. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 603 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0179] The computer device may also include an input unit 604, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0180] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the computer device loads the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 runs the application programs stored in the memory 602 to realize various functions, as follows:

[0181] A training set for the copper component region is obtained, which includes multiple images of the copper component region and corresponding first-class labels, including categories for defective and non-defective copper component regions. A small-sample reference set is obtained, which includes multiple images of socket assembly units and corresponding second-class labels, including categories for defective and non-defective plastic shells. The pre-trained model is then trained using the copper component region training set and the small-sample reference set to obtain a plastic shell defect detection model.

[0182] Alternatively, obtain an image of the socket to be identified;

[0183] The image of the socket to be identified is input into the plastic shell defect detection model to obtain the plastic shell defect detection result. The plastic shell defect detection result is either a plastic shell defect category or a plastic shell no-defect category. The plastic shell defect detection model is any one of the plastic shell defect detection models described in the first aspect.

[0184] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0185] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a disk, or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the training methods for a plastic shell defect detection model provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps:

[0186] A training set for the copper component region is obtained, which includes multiple images of the copper component region and corresponding first-class labels, including categories for defective and non-defective copper component regions. A small-sample reference set is obtained, which includes multiple images of socket assembly units and corresponding second-class labels, including categories for defective and non-defective plastic shells. The pre-trained model is then trained using the copper component region training set and the small-sample reference set to obtain a plastic shell defect detection model.

[0187] Alternatively, obtain an image of the socket to be identified;

[0188] The image of the socket to be identified is input into the plastic shell defect detection model to obtain the plastic shell defect detection result. The plastic shell defect detection result is either a plastic shell defect category or a plastic shell no-defect category. The plastic shell defect detection model is any one of the plastic shell defect detection models described in the first aspect.

[0189] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0190] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0191] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0192] The training method, detection method and apparatus for plastic shell defects provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A training method for a plastic shell defect detection model, characterized in that, The plastic shell defect detection model is used to detect plastic shell defects in the socket assembly unit, which includes a copper component and a plastic shell surrounding the copper component. The training method for the plastic shell defect detection model includes: Obtain a training set of copper parts regions, which includes multiple images of copper parts regions and corresponding first classification labels. The first classification labels include categories of copper parts regions with defects and categories of copper parts regions without defects. Obtain a small sample reference set, which includes multiple images of socket assembly units and corresponding second classification labels, the second classification labels including defective plastic shell category and non-defective plastic shell category; A plastic shell defect detection model is obtained by performing few-shot learning on the pre-trained model based on the copper component region training set and the few-shot reference set. Specifically, this involves: training the pre-trained model for image classification using the copper component region training set to obtain a network weight model; training the network weight model using few-shot learning on the few-shot reference set to obtain the plastic shell defect detection model; and further, training the network weight model using few-shot learning on the few-shot reference set to obtain the plastic shell defect detection model. The next step involves: training the network weight model using few-shot learning on the few-shot reference set to obtain a classification few-shot weight model; testing the target detection weight model and the classification few-shot weight model on a preset validation set to obtain test question data; adding the test question data to the few-shot reference set to train the classification few-shot weight model to obtain the plastic shell defect detection model. The target detection weight model is used to locate the copper component region to obtain the copper component region training set.

2. The training method for the plastic shell defect detection model according to claim 1, characterized in that, The acquisition of the copper component region training set includes: Obtain the original image set of the socket assembly unit; The copper component area is located in each original image of the original image set. Determine the first annotation information for each of the copper component regions based on the copper component regions; The original image is segmented according to each of the first annotation information to obtain the training set of the copper part region.

3. The training method for the plastic shell defect detection model according to claim 2, characterized in that, The step of segmenting the corresponding original image according to each of the first annotation information to obtain the copper part region training set includes: The overall area of ​​each original image positioning socket assembly unit in the original image set; The second labeling information for each overall area of ​​the socket assembly unit is determined based on the overall area of ​​the socket assembly unit; The entire area of ​​the plug-in assembly unit in the original image is segmented using the second annotation information to obtain an image of the plug-in assembly unit; The copper component area image is obtained by segmenting the image of the socket assembly unit using the first annotation information.

4. The training method for the plastic shell defect detection model according to claim 2, characterized in that, The location of the copper component area in each original image of the original image set includes: The copper component region is located in each original image of the original image set using an object detection weight model.

5. A method for detecting defects in plastic shells, characterized in that, The method for detecting defects in plastic shells includes: Obtain the image of the socket to be identified; The image of the socket to be identified is input into the plastic shell defect detection model to obtain the plastic shell defect detection result. The plastic shell defect detection result is either a plastic shell defect category or a plastic shell no-defect category. The plastic shell defect detection model is the plastic shell defect detection model according to any one of claims 1-4.

6. A training device for a plastic shell defect detection model, characterized in that, The plastic shell defect detection model is used to detect plastic shell defects in a socket assembly unit, the socket assembly unit including a copper component and a plastic shell surrounding the copper component, and the training device for the plastic shell defect detection model includes: The first acquisition unit is used to acquire a copper component area training set, which includes multiple copper component area images and corresponding first classification labels. The first classification labels include copper component area defective category and copper component area non-defective category. The second acquisition unit is used to acquire a small sample reference set, which includes multiple images of socket assembly units and corresponding second classification labels. The second classification labels include categories of defective plastic shells and categories of defect-free plastic shells. The model training unit is used to perform few-shot learning on a pre-trained model based on the copper component region training set and the few-shot reference set to obtain a plastic shell defect detection model. Specifically, the few-shot learning of the pre-trained model based on the copper component region training set and the few-shot reference set to obtain the plastic shell defect detection model includes: image classification training of the pre-trained model based on the copper component region training set to obtain a network weight model; few-shot training of the network weight model based on the few-shot reference set to obtain the plastic shell defect detection model; and few-shot training of the network weight model based on the few-shot reference set to obtain the plastic shell defect detection model, which includes: few-shot training of the network weight model based on the few-shot reference set to obtain a classification few-shot weight model; testing the target detection weight model and the classification few-shot weight model on a preset validation set to obtain test question data; adding the test question data to the few-shot reference set to train the classification few-shot weight model to obtain the plastic shell defect detection model. The target detection weight model is used to locate the copper component region to obtain the copper component region training set.

7. A plastic shell defect detection device, characterized in that, For detecting defects in the plastic shell of a socket assembly unit, the socket assembly unit including a copper component and a plastic shell surrounding the copper component, the plastic shell defect detection device includes: Image acquisition unit, used to acquire image of the socket to be identified; A defect detection unit is used to input the image of the socket to be identified into a plastic shell defect detection model to obtain a plastic shell defect detection result. The plastic shell defect detection result is either a plastic shell defect category or a plastic shell no-defect category. The plastic shell defect detection model is the plastic shell defect detection model according to any one of claims 1-4.

8. A computer device, characterized in that, The computer device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the training method of the plastic shell defect detection model according to any one of claims 1 to 4 or to implement the plastic shell defect detection method according to claim 5.

9. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to execute the training method of the plastic shell defect detection model according to any one of claims 1 to 4 or to implement the steps of the plastic shell defect detection method according to claim 5.

Images