Method, device and computer equipment for detecting scratch defects of electronic components

Abstract

The present application relates to a method, device, computer equipment, storage medium and computer program product for detecting scratch defects of electronic components. The method includes: obtaining a sample image set of sample electronic components, where the sample image set includes sample images with scratch defects and sample images without scratch defects; training a scratch defect detection model based on the average of the average precisions and the sample image set, adjusting the parameters in the scratch defect detection model, and obtaining a trained scratch defect detection model; and detecting scratch defects of target electronic components according to the trained scratch defect detection model. Using this method can improve the detection accuracy of the scratch defect detection model.

Description

Technical Field

[0001] The present application relates to the field of deep learning technologies, and particularly to a method, device, and computer device for detecting scratch defects of electronic components. Background Art

[0002] With the development of information technology, electronic components, as the basic underlying hardware for information transmission and processing, play a crucial role in the production and application of electronic products. Due to the continuous evolution of the demand for electronic products, electronic components are constantly developing in the direction of small size, thinness, chip type, miniaturization, and modularization. Although these characteristics improve the quality of electronic products, they bring great difficulties to testing institutions and testing personnel. In addition, during the production and processing of electronic components, complex process treatments are required. Under multiple process treatments, damage will inevitably occur on the surface of electronic components, which will cause surface defects of electronic components and directly affect whether the products are qualified. Therefore, the detection of surface defects of electronic components has become an essential process in the production, processing, and reliability analysis of electronic products.

[0003] In related technologies, the mainstream detection method is generally manual visual inspection. However, due to its low detection efficiency, poor real-time performance, and susceptibility to the subjective factors of testing personnel, it can no longer meet the growing detection requirements. Therefore, there is an urgent need for a method for detecting scratch defects of electronic components at present. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a method, device, computer device, computer-readable storage medium, and computer program product for detecting scratch defects of electronic components.

[0005] In a first aspect, the present application provides a method for detecting scratch defects of electronic components. The method includes:

[0006] Obtain a sample image set of sample electronic components, where the sample image set includes sample images with scratch defects and sample images without scratch defects;

[0007] Train a scratch defect detection model based on the mean Average Precision (mAP) and the sample image set, adjust the parameters in the scratch defect detection model, and obtain a trained scratch defect detection model;

[0008] Detect scratch defects of target electronic components according to the trained scratch defect detection model.

[0009] In one of the embodiments, the process of obtaining the sample image set includes:

[0010] Obtain multiple original sample images, perform a preset process on each original sample image to obtain multiple enhanced sample images, where the preset process includes translation, mirror flipping, brightness enhancement, or rotation by 90 degrees;

[0011] Through the ground truth boxes, label the scratch defects in multiple original sample images and multiple enhanced sample images respectively, determine the position, size of the ground truth boxes, and the defect categories framed by the ground truth boxes; based on the multiple enhanced sample images and multiple original sample images, obtain a training set, and form a sample image set from the training set.

[0012] In one embodiment, the scratch defect detection model is constructed based on the YOLOv3 model; among them, the YOLOv3 model includes three prediction layers, and the three prediction layers are respectively used as the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer. Among them, the grid division scales corresponding to the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer decrease in sequence; each prediction layer corresponds to at least one size of prior box, and the parameters in the YOLOv3 model include the target prior box size corresponding to the second-scale prediction layer; correspondingly, based on the mean Average Precision (mAP) and the training sample set, train the scratch defect detection model, adjust the parameters of the scratch defect detection model, and obtain a trained scratch defect detection model, including:

[0013] Determine the size range based on the prior box size corresponding to the second-scale prediction layer and the prior box size corresponding to the third-scale prediction layer;

[0014] Obtain multiple values within the size range, use each value as the value of the target prior box size, and determine the set of prior boxes corresponding to each training sample in the training sample set for each value; among them, for any value and any training sample, the set of prior boxes corresponding to any training sample for any training sample includes all the prior boxes output by any training sample through the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer for any value;

[0015] Determine the prediction box corresponding to each training sample for each value from the set of prior boxes corresponding to each training sample for each value, and based on the prediction box corresponding to each training sample for each value, calculate the mean Average Precision (mAP) of the training sample set for each value;

[0016] Determine the value corresponding to the maximum mean Average Precision (mAP), and use it as the final value of the target prior box size.

[0017] In one embodiment, determining a prediction box corresponding to each training sample at each value from the set of prior boxes corresponding to each training sample at each value includes:

[0018] Calculating the intersection over union (IoU) between each prior box in the set of prior boxes corresponding to each training sample at each value and the ground truth box corresponding to each training sample, and taking the prior box with the maximum IoU value in the set of prior boxes corresponding to each training sample at each value as the prediction box corresponding to each training sample at each value.

[0019] In one embodiment, the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer each correspond to at least one initial prior box size; the process of determining the initial prior box size includes:

[0020] Performing clustering on the ground truth box sizes corresponding to all sample images in the sample image set through the K-means algorithm to obtain the initial prior box size.

[0021] In one embodiment, according to the trained scratch defect detection model, performing scratch defect detection on a target electronic component includes:

[0022] Obtaining a target image of the target electronic component;

[0023] Inputting the target image into the trained scratch defect detection model to output a plurality of prior boxes for indicating the scratch positions in the target image;

[0024] Determining a confidence threshold for the prior boxes;

[0025] Determining a set of target prior boxes according to the confidence threshold and the confidences of the plurality of prior boxes;

[0026] Screening the set of target prior boxes by using the non-maximum suppression screening method to determine the final prediction box.

[0027] In a second aspect, the present application further provides an electronic component scratch defect detection device. The device includes:

[0028] A first acquisition module, configured to acquire a sample image set of sample electronic components, where the sample image set includes sample images with scratch defects and sample images without scratch defects;

[0029] A training module, configured to train a scratch defect detection model based on the mean average precision (mAP) and the sample image set, adjust parameters in the scratch defect detection model, and obtain a trained scratch defect detection model;

[0030] A detection module, configured to perform scratch defect detection on a target electronic component according to a trained scratch defect detection model.

[0031] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0032] Obtain a sample image set of sample electronic components, where the sample image set includes sample images with scratch defects and sample images without scratch defects;

[0033] Train a scratch defect detection model based on the mean of the average precision (mAP) and the sample image set, adjust the parameters in the scratch defect detection model, and obtain a trained scratch defect detection model;

[0034] Perform scratch defect detection on a target electronic component according to the trained scratch defect detection model.

[0035] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0036] Obtain a sample image set of sample electronic components, where the sample image set includes sample images with scratch defects and sample images without scratch defects;

[0037] Train a scratch defect detection model based on the mean of the average precision (mAP) and the sample image set, adjust the parameters in the scratch defect detection model, and obtain a trained scratch defect detection model;

[0038] Perform scratch defect detection on a target electronic component according to the trained scratch defect detection model.

[0039] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0040] Obtain a sample image set of sample electronic components, where the sample image set includes sample images with scratch defects and sample images without scratch defects;

[0041] Train a scratch defect detection model based on the mean Average Precision (mAP) and a sample image set, adjust the parameters in the scratch defect detection model, and obtain the trained scratch defect detection model;

[0042] According to the trained scratch defect detection model, perform scratch defect detection on the target electronic component.

[0043] The above-mentioned electronic component scratch defect detection method, device, computer device, storage medium, and computer program product obtain a sample image set of sample electronic components, where the sample image set includes sample images with scratch defects and sample images without scratch defects; train a scratch defect detection model based on the mean Average Precision and the sample image set, adjust the parameters in the scratch defect detection model, and obtain the trained scratch defect detection model; perform scratch defect detection on the target electronic component according to the trained scratch defect detection model, thereby improving the detection accuracy of the scratch defect detection model. Brief Description of the Drawings

[0044] Figure 1 It is an application environment diagram of the electronic component scratch defect detection method in an embodiment;

[0045] Figure 2 It is a flowchart of the electronic component scratch defect detection method in an embodiment;

[0046] Figure 3 It is a comparison diagram of artificially made scratch defects and surface scratch defects of electronic components caused by other reasons such as instruments during the production process in an embodiment;

[0047] Figure 4 It is a structural block diagram of the YOLOv3 model in an embodiment;

[0048] Figure 5 It is a structural block diagram of the electronic component scratch defect detection device in an embodiment;

[0049] Figure 6 It is an internal structure diagram of a computer device in an embodiment. Detailed Description of the Embodiments

[0050] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] It can be understood that the terms "first", "second", etc. used in this application can be used in this text to describe various technical terms. However, unless otherwise specified, these technical terms are not limited by these terms. These terms are only used to distinguish one technical term from another. For example, without departing from the scope of this application, the third preset threshold and the fourth preset threshold can be the same or different.

[0052] The method for detecting scratch defects of electronic components provided by the embodiments of this application can be applied to an application environment as shown in Figure 1 In the application environment shown. Among them, the terminal 101 communicates with the server 102 through a network. The data storage system can store the data that the server 102 needs to process. The data storage system can be integrated on the server 102, or can be placed in the cloud or on other network servers. After obtaining the sample image set, the terminal 101 can train the sample image set to obtain a trained scratch defect detection model, and then use the trained scratch defect detection model to detect the scratch defects of the target electronic components.

[0053] Among them, the terminal 101 can be but is not limited to various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, imaging devices, and portable wearable devices. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 102 can be implemented by an independent server or a server cluster composed of multiple servers.

[0054] In one embodiment, as shown in Figure 2 A method for detecting scratch defects of electronic components is provided. In this embodiment, an example is given where this method is applied to a terminal. It can be understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0055] 201. Obtain a sample image set of sample electronic components, where the sample image set includes sample images with scratch defects and sample images without scratch defects;

[0056] 202. Train the scratch defect detection model based on the mean average precision (mAP) and the sample image set, adjust the parameters in the scratch defect detection model, and obtain a trained scratch defect detection model;

[0057] 203. According to the trained scratch defect detection model, detect the scratch defects of the target electronic components.

[0058] In the above step 201, electronic components refer to electronic elements and components of small machines and instruments. They are often composed of several parts and can be used interchangeably in similar products. For the specific types of electronic components, the embodiments of the present invention do not make specific limitations, including but not limited to: resistors, capacitors, inductors, potentiometers, electron tubes, radiators, electromechanical components, connectors, semiconductor discrete devices, electroacoustic devices, laser devices, electronic display devices, optoelectronic devices, sensors, power supplies, switches, micro special motors, electronic transformers, relays, printed circuit boards, special materials for electronic functional processes, electronic glue (tape) products, and electronic chemical materials and parts, etc.

[0059] Regarding the source of the sample image set, the embodiments of the present invention do not make specific limitations, including but not limited to: self-made sample image sets. For example, a SCARA robot can be used to drive a lens to collect images of electronic components in a tray.

[0060] In addition, since the surface quality of finished electronic components is relatively high and the number of electronic components with scratch defects is small, it is necessary to artificially damage the surface of the electronic components to create scratch defects. Therefore, the scratches in the sample images with scratch defects can be artificially created, which is called artificially created scratch defects; or they can be caused by other reasons such as instruments during the production process of the electronic components, which is called real scratch defects. A comparison diagram of artificially created scratch defects and surface scratch defects caused by other reasons such as instruments during the production process of electronic components is as Figure 3 shown.

[0061] In the above step 202, the mean Average Precision (mAP) is a measure for evaluating the performance of the target detection model. Before determining the mAP, it is necessary to first determine the Precision (P), then calculate the Average Precision (AP) based on the Precision, and finally calculate the mAP based on the AP. In addition, the adjustment of the parameters in the above step 202 requires manual adjustment.

[0062] It is worth mentioning that in addition to the parameters in the above step 202, the parameters in the scratch defect detection model also include the parameters in the loss function and other parameters that affect the scratch defect detection model. During the training process of the scratch defect detection model, some parameters in the loss function and some other parameters of the scratch defect detection model are also being adjusted. Among them, the loss function plays a supervisory role for the scratch defect detection model. The loss function includes the center coordinate loss function y1, the width and height coordinate loss function y2, the predicted category loss function y3, and the confidence loss function y4.

[0063] Specifically, the process of training the scratch defect detection model based on the mean Average Precision (mAP) and the sample image set is as follows: Each time the parameters in the scratch defect detection model are adjusted, an average value of the average precision is obtained. And each average value of the average precision is obtained by averaging multiple average precisions. Therefore, the number of times the sample image set is trained is related to the number of times the parameters in the scratch defect detection model are adjusted. In addition, the number of average precision values is related to the number of training sample images in the sample image set. For example, if there are 100 training sample images in the sample image set and the parameters are adjusted manually 10 times, then 100 average precisions will be obtained each time the parameters are adjusted, and 10 average values of the average precision will be obtained after 10 parameter adjustments.

[0064] The method provided by the embodiments of the present invention can determine the parameters in the scratch defect detection model that are most suitable for detecting small targets by adjusting the parameters in the scratch defect detection model and according to the average value of the average precision, so that the detection accuracy of the scratch defect detection model is high, which is conducive to the demand, promotion and application of the surface defect detection mechanism of electronic components.

[0065] Combined with the content of the above embodiments, in one embodiment, the process of obtaining the sample image set includes:

[0066] 301. Obtain a plurality of original sample images, and perform a preset process on each original sample image to obtain a plurality of enhanced sample images, where the preset process includes translation, mirror flipping, brightness enhancement, or rotation by 90 degrees;

[0067] 302. Mark the scratch defects in the plurality of original sample images and the plurality of enhanced sample images respectively through the ground truth boxes, and determine the position, size of the ground truth boxes and the defect categories framed by the ground truth boxes; Based on the plurality of enhanced sample images and the plurality of original sample images, obtain a training set, and the training set constitutes the sample image set.

[0068] In the above step 301, the original sample images refer to those obtained by collecting the appearance of electronic components through an image hardware acquisition device, and the corresponding image quality is relatively high. Among them, the relatively high image quality is mainly reflected in two aspects: image clarity and accuracy.

[0069] Generally, in order to make the performance of the deep learning model better, a large amount of data is required to train it. However, the electronic components in the embodiments of the present invention have problems such as difficult sampling of sample images, harsh image acquisition conditions, and small amount of original sample images. In order to improve the detection accuracy of the scratch defect detection model and ensure the real-time performance during the detection process, it is necessary to perform a preset process on each original sample image in the original sample image set to expand the number of sample images.

[0070] In addition, before performing the preset processing on each original sample image in the original sample image set, it is necessary to first perform size normalization processing on all the original sample images, and then perform image standardization processing on the original sample images after size normalization processing. Therefore, the processing process of all the original sample images is as follows: first perform size normalization processing. For example, the original sample images can be normalized to images with a size of 200×200 pixels. Then perform image standardization processing, and finally perform image preset processing.

[0071] Regarding the pixel size of the images after size normalization processing and image standardization processing, the embodiments of the present invention do not make specific limitations on it, including but not limited to: after size normalization processing, the pixels of all sample images can be 200*200, or can be 300*300; after image standardization processing, the pixels of all sample images can be 316*316, or can be 416*416.

[0072] Among them, the image preset processing includes: translation, mirror flipping, brightness enhancement or rotation by 90 degrees. Regarding the range of translation, the embodiments of the present invention do not make specific limitations on it, including but not limited to: translating the entire sample image 15% to the right, translating the entire sample image 15% to the left, translating the entire sample image 15% upward, and translating the entire sample image 15% downward.

[0073] Among them, the image preset processing means that each sample image will undergo translation, mirror flipping, brightness enhancement or rotation by 90 degrees to obtain 4 new sample images, and then these 4 new sample images are screened to select suitable sample images to be used together with the original sample images as the final sample image set. For example, there are a total of 176 original sample images with scratch defects, and a total of 226 original sample images without scratch defects. After image preset processing, there are 633 new sample images with scratch defects and 690 new sample images without scratch defects. The final sample image set includes a total of 1725 images. In addition, in order to simulate the brightness change in the real environment, the Gamma correction method is also used to change the brightness of the sample images.

[0074] In the above step 302, the true box refers to after obtaining the final sample image set, a labeling software will be used to label the scratch defect position and category of the sample images with scratch defects, determine the size of the true box, and save information such as the scratch defect position, the category label of the true box, the coordinates of the true box, and the size of the true box. Regarding the labeling software, the embodiments of the present invention do not make specific limitations on it, including but not limited to: Labelme software.

[0075] Specifically, the final sample image set includes multiple enhanced sample images and multiple original sample images. The final sample image set is divided into a validation set and a training set, and the sample image set is composed of the validation set and the training set. Among them, the training set is used to train the scratch defect detection model, and the validation set is used to verify the trained scratch defect detection model. In addition, the training set can also be used to verify the trained scratch defect detection model.

[0076] The method provided by the embodiment of the present invention can increase the number of samples in the training set by performing image enhancement processing on the original sample images, thereby avoiding the overfitting phenomenon of the scratch defect detection model caused by the small number of sample images, and further improving the detection accuracy of the scratch defect detection model.

[0077] Combined with the content of the above embodiments, in one embodiment, the scratch defect detection model is constructed based on the yolov3 model. Among them, the yolov3 model includes three prediction layers, and the three prediction layers are respectively used as the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer. Among them, the grid division scales corresponding to the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer decrease in turn. Each prediction layer corresponds to at least one size of prior box, and the parameters in the yolov3 model include the target prior box size corresponding to the second-scale prediction layer. Correspondingly, training the scratch defect detection model based on the mean Average Precision (mAP) and the training sample set, and adjusting the parameters of the scratch defect detection model to obtain the trained scratch defect detection model, including:

[0078] 401. Determine the size range based on the prior box size corresponding to the second-scale prediction layer and the prior box size corresponding to the third-scale prediction layer;

[0079] 402. Obtain multiple values in the size range, use each value as the value of the target prior box size, and determine the prior box set corresponding to each training sample in the training sample set under each value. Among them, for any value and any training sample, the prior box set corresponding to any training sample under any training sample includes all the prior boxes output by any training sample through the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer under any value;

[0080] 403. Determine the prediction box corresponding to each training sample under each value from the prior box set corresponding to each training sample under each value, and calculate the mean Average Precision (mAP) of the training sample set under each value based on the prediction box corresponding to each training sample under each value;

[0081] 404. Determine the value corresponding to the mean of the maximum mean average precision (mAP), and use it as the final value of the target prior box size.

[0082] Among them, the structure diagram of the yolov3 model is as Figure 4 shown. The yolov3 model is a model algorithm that uses the DarkNet-53 convolutional network as the feature extraction network. The DarkNet-53 convolutional network contains 53 convolutional layers, and each layer is followed by a batch normalization (BN) layer and a leaky ReLU activation layer. And a convolutional layer with a stride of 2 is used to replace the pooling layer for the downsampling process of the feature map. Such an improvement can effectively prevent the loss of low-level features caused by the pooling layer.

[0083] Moreover, the yolov3 model is a regression-based object detection model. The prior detection system of this model reuses the classifier or locator to perform the detection task, and applies the model to multiple positions and scales of the image, and finally regards the region with a higher score as the detection result. In addition, the yolov3 model only uses convolutional layers to make it a fully convolutional network (FCN).

[0084] It is worth mentioning that the yolov3 model has two important features:

[0085] (1) Using the Residual network, the residual convolution in the DarkNet-53 convolutional network first performs a convolution with a kernel size of 3×3 and a stride of 2 to compress the width and height of the input image. At this time, a feature layer can be obtained and named layer. Then, a 1×1 convolution kernel and a 3×3 convolution are performed on this feature layer, and the result is added to layer. At this time, the residual structure is formed. Then, through continuous stacking of 1×1 convolutions, 3×3 convolutions, and residual edges. Among them, the characteristic of the residual network is that it is easy to optimize and can improve the accuracy by increasing the appropriate network depth. In addition, the residual blocks inside the DarkNet-53 convolutional network use skip connections, which alleviates the problem of gradient disappearance caused by increasing the depth in the deep neural network.

[0086] (2) Each convolutional part in the DarkNet-53 convolutional network uses a unique DarkNetConv2D structure. L2 regularization is performed during each convolution, and batch normalization and the activation function (Leaky ReLU) are performed when the convolution is completed. The activation function (Leaky ReLU) layer assigns all values a non-zero slope, and the assignment formula is shown in Equation (1):

[0087]

[0088] In Equation (1), y i represents the activation function, and a i is a fixed parameter belonging to the interval (1, +∞).

[0089] At the same time, the yolov3 model predicts results from features, and the process can be divided into two parts: (1) Construct a Feature Pyramid Networks (FPN) to enhance feature extraction. In the feature utilization part, the yolov3 model can extract three feature layers for object detection, and these three feature layers are located at different positions (the middle layer, the middle-lower layer, and the bottom layer) of the DarkNet-53 convolutional network. Subsequently, these feature layers are used to construct the FPN layer. The feature pyramid structure can fuse features of different-shaped feature layers, which is beneficial to extracting better features. (2) Use the yolov3 Head to predict the three feature layers, namely the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer. The essence of the yolov3 Head is a 3×3 convolution followed by a 1×1 convolution. Among them, the role of the 3×3 convolution is feature integration, and the role of the 1×1 convolution is to adjust the number of channels.

[0090] Compared with other object detection methods, the yolov3 model uses a single neural network to act on the entire image, divides the image into different regions, and thus predicts the bounding boxes and probabilities of each region. These bounding boxes are weighted by the predicted probabilities to obtain the final detection result.

[0091] In the actual use process, each of the three prediction layers corresponds to three sizes of prior boxes, that is, the three prediction layers correspond to a total of nine sizes of prior boxes. The fact that the grid division scales corresponding to the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer decrease in sequence means that the areas of the divided grids decrease in sequence; correspondingly, the number of grids in the feature map output by the first-scale prediction layer is the least, the number of grids in the feature map output by the third-scale prediction layer is the most, and the number of grids in the feature map output by the second-scale prediction layer is between the number of grids in the feature map output by the first-scale prediction layer and the number of grids in the feature map output by the third-scale prediction layer. Generally, the number of grids in the feature map output by the first-scale prediction layer is 13×13, the number of grids in the feature map output by the second-scale prediction layer is 26×26, and the number of grids in the feature map output by the third-scale prediction layer is 52×52.

[0092] In addition, the sizes of the three prior boxes corresponding to the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer decrease in sequence. During the training process of the yolov3 model, the sizes of the prior boxes corresponding to the first-scale prediction layer and the third-scale prediction layer will not change, and the size corresponding to the second-scale prediction layer will be adjusted accordingly.

[0093] Combined with the above content, since in actual use, the second-scale prediction layer corresponds to three sizes of prior boxes, the size ranges in step 401 above are also divided into three types. The three size ranges are determined according to the sizes of the prior boxes corresponding to the second-scale prediction layer and the sizes of the prior boxes corresponding to the third-scale prediction layer. The three sizes of the prior boxes corresponding to the second-scale prediction layer are sorted by size as A1, A2, and A3 respectively, and the three sizes of the prior boxes corresponding to the third-scale prediction layer are sorted by size as B1, B2, and B3 respectively. Then the three size ranges are: [A1 - B1], [A2 - B2], and [A3 - B3].

[0094] For example, the sizes of the prior boxes corresponding to the second-scale prediction layer are 150cm×160cm, 145cm×155cm, and 140cm×150cm respectively, and the sizes of the prior boxes corresponding to the third-scale prediction layer are 50cm×60cm, 45cm×55cm, and 40cm×50cm respectively. Therefore, the three size ranges are: [50cm×60cm - 150cm×160cm], [45cm×55cm - 145cm×155cm], and [40cm×50cm - 140cm×150cm].

[0095] Combined with the above content, regarding the value-taking methods within the three size ranges, the embodiments of the present invention do not make specific limitations, including but not limited to:

[0096] The sizes of the three prior boxes are taken upward from the minimum values in the corresponding size ranges respectively, and the interval ranges for each value-taking are the same, that is, the sizes of the three prior boxes change by the same amount each time. For example, the three size ranges are [40 cm × 50 cm - 140 cm × 150 cm], [45 cm × 55 cm - 145 cm × 155 cm], and [50 cm × 60 cm - 150 cm × 160 cm]. When taking values for the first time, the sizes of the three prior boxes are 42 cm × 52 cm, 47 cm × 57 cm, and 52 cm × 62 cm respectively. When taking values for the second time, the sizes of the three prior boxes are 44 cm × 54 cm, 49 cm × 59 cm, and 54 cm × 64 cm respectively. When taking values for the third time, the sizes of the three prior boxes are 46 cm × 56 cm, 51 cm × 61 cm, and 56 cm × 66 cm respectively. It is worth mentioning that neither the upper nor the lower limits of the three size ranges will be selected.

[0097] Specifically, after determining the sizes of the three target prior boxes of the second scale prediction layer from the three size ranges each time, the training set in step 302 above will be input into the scratch defect detection model for training. After each complete training of the training set, an average value of the mean Average Precision (mAP) will be obtained. The sizes of the three target prior boxes corresponding to the maximum average value of the mean average precision are taken as the final sizes of the three prior boxes of the second scale prediction layer in the scratch defect detection model.

[0098] The method provided by the embodiment of the present invention modifies the sizes of the prior boxes in the second scale prediction layer of the yolov3 model, making the sizes of the prior boxes smaller, thereby eliminating the problem of prior box waste caused by scale reasons. At the same time, since the number of prior boxes suitable for detecting small targets is increased, the accuracy of the scratch defect detection model in detecting scratch defects is improved.

[0099] Combined with the content of the above embodiments, in one embodiment, determining the prediction boxes corresponding to each training sample at each value-taking from the set of prior boxes corresponding to each training sample at each value-taking includes:

[0100] Calculating the intersection over union (IoU) of each prior box in the set of prior boxes corresponding to each training sample at each value-taking with the ground truth box corresponding to each training sample, and taking the prior box with the maximum intersection over union (IoU) value in the set of prior boxes corresponding to each training sample at each value-taking as the prediction box corresponding to each training sample at each value-taking.

[0101] Specifically, after each sample image in the training set is input into the scratch defect detection model, a set of prior boxes composed of multiple prior boxes will be output. Then, the intersection over union (IoU) of each predicted box and the corresponding ground truth box is calculated. The prior box corresponding to the maximum IoU value in the set of predicted boxes corresponding to each training sample at each value is used as the predicted box corresponding to each training sample at each value. The center coordinate loss, width and height coordinate loss, predicted class loss, and confidence loss between the predicted box and the ground truth box are calculated through the loss function, and the parameters of the scratch defect detection model are adjusted through the four loss values.

[0102] Among them, the loss function includes the center coordinate loss function y1, the width and height coordinate loss function y2, the predicted class loss function y3, and the confidence loss function y4. The expression of the loss function is as follows:

[0103] (1) Center coordinate loss function y1,

[0104]

[0105] In formula (2), S 2 is the total number of grids, B is the number of prior boxes corresponding to each grid, i and j are the index values of S 2 and B, (x i , y i ) and (x i ', y i ) represent the center coordinates of the predicted box and the ground truth box. When the j-th candidate box in the i-th grid cell is responsible for the detection target (it is defined that the IoU value between the prior box and the ground truth box is the largest, which means it is responsible), Otherwise

[0106] (2) Width and height coordinate loss function y2,

[0107]

[0108] In formula (3), S 2 is the total number of grids, B is the number of prior boxes corresponding to each grid, (w i , h i ) and (w i ', h i ) are the width and height of the predicted box and the width and height of the ground truth box. When the j-th candidate box in the i-th grid cell is responsible for the detection target (it is defined that the IoU value between the prior box and the ground truth box is the largest, which means it is responsible), Otherwise

[0109] (3) Predicted class loss function y3,

[0110]

[0111] In formula (4), S 2 is the total number of grids, p i is the classification probability. When the prior box contains the detection target, p i = 1; otherwise, p i = 0. c represents the category, and classes represents the category set. When the j-th candidate box in the i-th grid cell is responsible for the detection target (it is defined that the IoU value between the prior box and the ground truth box is the largest to be responsible), Otherwise

[0112] (4) Confidence loss function y4

[0113]

[0114] In formula (5), S 2 is the total number of grids, c i is the confidence. When the detection box is responsible for the detection target, c i = 1; otherwise, c i = 0. λ represents the weight. When the j-th candidate box in the i-th grid cell is responsible for the detection target (it is defined that the IoU value between the prior box and the ground truth box is the largest to be responsible), Otherwise

[0115] The method provided by the embodiment of the present invention selects the prediction box with the highest overlap degree with the ground truth box from all prediction boxes as the final prediction box by calculating the intersection over union of the prediction box and the corresponding ground truth box, calculates the loss between the prediction box and the ground truth box, thereby adjusting the parameters of the scratch defect detection model, and further improving the accuracy of the scratch defect detection model.

[0116] Combined with the content of the above embodiments, in one embodiment, each of the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer corresponds to at least one initial prior box size; the determination process of the initial prior box size includes:

[0117] By using the K-means algorithm, cluster the ground truth box sizes corresponding to all sample images in the sample image set to obtain the initial prior box size.

[0118] Specifically, each of the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer corresponds to three initial prior box sizes, that is, the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer together correspond to nine initial prior box sizes. These nine initial prior box sizes are obtained by clustering the ground truth box sizes corresponding to all sample images in the sample image set through the K-means algorithm.

[0119] In addition, during the training process, the three initial prior box sizes corresponding to the first scale prediction layer and the third scale prediction layer will not be adjusted. What needs to be adjusted is the initial prior box size corresponding to the second scale prediction layer.

[0120] The method provided by the embodiment of the present invention can determine the three initial prior box sizes corresponding to the first scale prediction layer, the second scale prediction layer, and the third scale prediction layer respectively by clustering the true box sizes corresponding to all sample images in the sample image set through the K-means algorithm, avoiding the redundant training of the scratch defect detection model due to uncertain prior box sizes, thereby improving the detection speed of the scratch defect detection model.

[0121] Combined with the content of the above embodiments, in one embodiment, according to the trained scratch defect detection model, performing scratch defect detection on a target electronic component includes:

[0122] 501. Obtain a target image of the target electronic component;

[0123] 502. Input the target image into the trained scratch defect detection model, and output a plurality of prior boxes for indicating the scratch positions in the target image;

[0124] 503. Determine a confidence threshold for the prior boxes; according to the confidence threshold and the confidences of the plurality of prior boxes, determine a set of target prior boxes;

[0125] 504. Screen the set of target prior boxes by using the non-maximum suppression screening method to determine the final prediction box.

[0126] In step 502 above, the trained scratch defect detection model refers to that the sizes of the three prior boxes corresponding to the second scale prediction layer have been determined, and in addition, other parameters of the scratch defect detection model have also been adjusted.

[0127] Specifically, after obtaining the target image of the target electronic component, inputting the target image into the trained scratch defect detection model will obtain a plurality of prior boxes. Compare the confidences of all prior boxes with the confidence threshold, select the prior boxes with confidences greater than the confidence threshold to form a set of target prior boxes, and then perform score sorting and screening on the set of target prior boxes by using the non-maximum suppression screening method to determine the final prediction box and output the detection result.

[0128] The method provided by the embodiment of the present invention can remove redundant prediction boxes by screening the set of target prior boxes through the non-maximum suppression screening method, thereby improving the accuracy of the detection result.

[0129] Combined with the content of the above embodiments, in one embodiment, an electronic component scratch defect detection method, the method further includes:

[0130] A scratch defect detection algorithm is constructed based on data augmentation technology and a deep learning network, and a defect self-learning model under small samples and strong interference is established. Among them, the data augmentation technology refers to the preset processing of each original sample image in the above step 301.

[0131] First, aiming at the problem of too small sample size of samples with scratch defects in the dataset, scratch defects are artificially created; secondly, an image hardware acquisition device is used to collect the appearance images of electronic components and establish a corresponding high-quality scratch defect image set; then, since the feature learning of the deep network is driven by image data, the collected image database is also augmented; finally, the features of the input original image are extracted by means of the DarkNet-53 network, and after processing the extracted feature maps, the parameters of the electronic component scratch defect detection model are adjusted.

[0132] The method provided by the embodiment of the present invention can improve the clarity of the collected images by collecting the appearance images of electronic components through an image hardware acquisition device, thereby improving the accuracy of image annotation; at the same time, a surface scratch defect detection model for electronic components is obtained through training, and the detection accuracy and real-time performance of the scratch defect detection model are improved.

[0133] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0134] Based on the same inventive concept, the embodiment of the present application also provides an electronic component scratch defect detection device for implementing the above-mentioned electronic component scratch defect detection method. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the following electronic component scratch defect detection device can refer to the limitations on the electronic component scratch defect detection method in the above text, and will not be repeated here.

[0135] In one embodiment, as Figure 5As shown in the figure, an electronic component scratch defect detection device is provided, including: a first acquisition module 501, a training module 502, and a detection module 503, where:

[0136] The first acquisition module 501 is used to acquire a sample image set of sample electronic components, where the sample image set includes sample images with scratch defects and sample images without scratch defects;

[0137] The training module 502 is used to train a scratch defect detection model based on the average value of the average precision and the sample image set, adjust the parameters in the scratch defect detection model, and obtain a trained scratch defect detection model;

[0138] The detection module 503 is used to detect scratch defects of target electronic components according to the trained scratch defect detection model.

[0139] In one embodiment, the first acquisition module 501 includes:

[0140] An acquisition unit is used to acquire a plurality of original sample images, perform preset processing on each original sample image to obtain a plurality of enhanced sample images, and the preset processing includes translation, mirror flipping, brightness enhancement, or rotation by 90 degrees;

[0141] A labeling unit is used to label the scratch defects in a plurality of original sample images and a plurality of enhanced sample images respectively through a ground truth box, determine the position, size of the ground truth box, and the defect category framed by the ground truth box; based on the plurality of enhanced sample images and the plurality of original sample images, obtain a training set, and the training set constitutes a sample image set.

[0142] In one embodiment, the training module 502 includes:

[0143] A first determination unit is used to determine a size range based on the prior box size corresponding to the second scale prediction layer and the prior box size corresponding to the third scale prediction layer;

[0144] A second determination unit is used to obtain a plurality of values in the size range, use each value as the value of the target prior box size, and determine the prior box set corresponding to each training sample in the training sample set for each value; where, for any value and any training sample, the prior box set corresponding to any training sample under any training sample includes all prior boxes output by any training sample through the first scale prediction layer, the second scale prediction layer, and the third scale prediction layer for any value;

[0145] A calculation unit, configured to determine, from the set of prior boxes corresponding to each training sample at each value, the prediction box corresponding to each training sample at each value, and calculate the average value of the average precision of the training sample set at each value based on the prediction boxes corresponding to each training sample at each value;

[0146] A third determination unit, configured to determine the value corresponding to the maximum average value of the average precision and use it as the final value of the target prior box size.

[0147] In one embodiment, the calculation unit includes:

[0148] A calculation subunit, configured to calculate the intersection over union (IoU) between each prior box in the set of prior boxes corresponding to each training sample at each value and the ground truth box corresponding to each training sample, and use the prior box with the maximum IoU value in the set of prior boxes corresponding to each training sample at each value as the prediction box corresponding to each training sample at each value.

[0149] In one embodiment, an electronic component scratch defect detection device further includes:

[0150] A clustering unit, configured to cluster the ground truth box sizes corresponding to all sample images in the sample image set by the K-means algorithm to obtain the initial prior box sizes.

[0151] In one embodiment, an electronic component scratch defect detection device further includes:

[0152] A second acquisition module, configured to acquire a target image of a target electronic component;

[0153] An output module, configured to input the target image into the trained scratch defect detection model and output a plurality of prior boxes for indicating the scratch positions in the target image;

[0154] A determination module, configured to determine a confidence threshold of the prior boxes; determine a target prior box set according to the confidence threshold and the confidences of the plurality of prior boxes;

[0155] A screening module, configured to screen the target prior box set by using the non-maximum suppression screening method to determine the final prediction boxes.

[0156] Each module in the above electronic component scratch defect detection device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above respective modules.

[0157] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in Figure 6 . The computer device includes a processor, a memory, and a communication interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method for detecting scratch defects of electronic components.

[0158] Those skilled in the art can understand that Figure 6 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0159] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0160] Obtain a sample image set of sample electronic components, where the sample image set includes sample images with scratch defects and sample images without scratch defects;

[0161] Train a scratch defect detection model based on the average value of average precision and the sample image set, adjust the parameters in the scratch defect detection model, and obtain a trained scratch defect detection model;

[0162] According to the trained scratch defect detection model, perform scratch defect detection on the target electronic components.

[0163] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0164] Obtain a plurality of original sample images, perform a preset process on each original sample image to obtain a plurality of enhanced sample images, where the preset process includes translation, mirror flipping, brightness enhancement, or rotation by 90 degrees;

[0165] Using the ground truth boxes, label the scratch defects in multiple original sample images and multiple enhanced sample images respectively, and determine the position, size of the ground truth boxes and the defect categories framed by the ground truth boxes; based on the multiple enhanced sample images and multiple original sample images, obtain a training set, and form a sample image set from the training set.

[0166] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0167] Determine the size range based on the prior box sizes corresponding to the second scale prediction layer and the prior box sizes corresponding to the third scale prediction layer;

[0168] Obtain multiple values within the size range, use each value as the value of the target prior box size, and determine the set of prior boxes corresponding to each training sample in the training sample set for each value; wherein, for any value and any training sample, the set of prior boxes corresponding to any training sample under any training sample includes all the prior boxes output by any training sample through the first scale prediction layer, the second scale prediction layer and the third scale prediction layer for any value;

[0169] Determine the prediction box corresponding to each training sample for each value from the set of prior boxes corresponding to each training sample for each value, and calculate the average value of the average precision of the training sample set for each value based on the prediction box corresponding to each training sample for each value;

[0170] Determine the value corresponding to the largest average value of the average precision, and use it as the final value of the target prior box size.

[0171] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0172] Calculate the intersection over union of each prior box in the set of prior boxes corresponding to each training sample for each value with the ground truth box corresponding to each training sample, and use the prior box corresponding to the largest intersection over union value in the set of prior boxes corresponding to each training sample for each value as the prediction box corresponding to each training sample for each value.

[0173] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0174] Through the K-means algorithm, cluster the ground truth box sizes corresponding to all sample images in the sample image set to obtain the initial prior box sizes.

[0175] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0176] Obtain the target image of the target electronic component;

[0177] Input the target image into the trained scratch defect detection model, and output multiple prior boxes for indicating the positions of scratches in the target image;

[0178] Determine the confidence threshold of the prior boxes; according to the confidence threshold and the confidences of the multiple prior boxes, determine the set of target prior boxes;

[0179] Use the non-maximum suppression screening method to screen the set of target prior boxes to determine the final prediction boxes.

[0180] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0181] Obtain a set of sample images of sample electronic components, where the set of sample images includes sample images with scratch defects and sample images without scratch defects;

[0182] Train the scratch defect detection model based on the average value of the average precision and the set of sample images, adjust the parameters in the scratch defect detection model, and obtain the trained scratch defect detection model;

[0183] According to the trained scratch defect detection model, perform scratch defect detection on the target electronic components.

[0184] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0185] Obtain a plurality of original sample images, perform preset processing on each original sample image to obtain a plurality of enhanced sample images, and the preset processing includes translation, mirror flipping, brightness enhancement, or rotation by 90 degrees;

[0186] Use the ground truth boxes to label the scratch defects in the plurality of original sample images and the plurality of enhanced sample images respectively, determine the positions, sizes of the ground truth boxes, and the defect categories framed by the ground truth boxes; based on the plurality of enhanced sample images and the plurality of original sample images, obtain a training set, and form the set of sample images from the training set.

[0187] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0188] Determine the size range based on the prior box sizes corresponding to the second-scale prediction layer and the prior box sizes corresponding to the third-scale prediction layer;

[0189] Obtain multiple values within a size range, and use each value as the value of the target prior box size to determine the set of prior boxes corresponding to each training sample in the training sample set for each value; wherein, for any value and any training sample, the set of prior boxes corresponding to any training sample for any training sample includes all the prior boxes output by any training sample through the first scale prediction layer, the second scale prediction layer, and the third scale prediction layer for any value.

[0190] Determine the prediction box corresponding to each training sample for each value from the set of prior boxes corresponding to each training sample for each value, and calculate the average value of the average precision of the training sample set for each value based on the prediction box corresponding to each training sample for each value.

[0191] Determine the value corresponding to the maximum average value of the average precision, and use it as the final value of the target prior box size.

[0192] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0193] Calculate the intersection over union of each prior box in the set of prior boxes corresponding to each training sample for each value with the ground truth box corresponding to each training sample, and use the prior box corresponding to the maximum intersection over union value in the set of prior boxes corresponding to each training sample for each value as the prediction box corresponding to each training sample for each value.

[0194] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0195] Cluster the ground truth box sizes corresponding to all sample images in the sample image set through the K-means algorithm to obtain the initial prior box size.

[0196] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0197] Obtain the target image of the target electronic component;

[0198] Input the target image into the trained scratch defect detection model, and output multiple prior boxes for indicating the scratch positions in the target image;

[0199] Determine the confidence threshold of the prior box; determine the target prior box set according to the confidence threshold and the confidence of the multiple prior boxes;

[0200] Screen the target prior box set using the non-maximum suppression screening method to determine the final prediction box.

[0201] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0202] Obtain a sample image set of sample electronic components, where the sample image set includes sample images with scratch defects and sample images without scratch defects;

[0203] Train a scratch defect detection model based on the average value of average precision and the sample image set, adjust the parameters in the scratch defect detection model, and obtain a trained scratch defect detection model;

[0204] According to the trained scratch defect detection model, perform scratch defect detection on the target electronic components.

[0205] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0206] Obtain a plurality of original sample images, perform preset processing on each original sample image to obtain a plurality of enhanced sample images, where the preset processing includes translation, mirror flipping, brightness enhancement, or rotation by 90 degrees;

[0207] Mark the scratch defects in the plurality of original sample images and the plurality of enhanced sample images respectively through the ground truth boxes, determine the positions, sizes of the ground truth boxes, and the defect categories framed by the ground truth boxes; based on the plurality of enhanced sample images and the plurality of original sample images, obtain a training set, and form a sample image set from the training set.

[0208] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0209] Determine a size range based on the prior box sizes corresponding to the second scale prediction layer and the prior box sizes corresponding to the third scale prediction layer;

[0210] Obtain a plurality of values within the size range, use each value as the value of the target prior box size, and determine the set of prior boxes corresponding to each training sample in the training sample set for each value; where, for any value and any training sample, the set of prior boxes corresponding to any training sample for any training sample includes all the prior boxes output by any training sample through the first scale prediction layer, the second scale prediction layer, and the third scale prediction layer for any value;

[0211] Determine the predicted box corresponding to each training sample for each value from the set of prior boxes corresponding to each training sample for each value, and calculate the average value of the average precision of the training sample set for each value based on the predicted box corresponding to each training sample for each value;

[0212] Determine the value corresponding to the maximum average value of average precision and use it as the final value of the target prior box size.

[0213] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0214] Calculate the intersection over union (IoU) between each prior box in the set of prior boxes corresponding to each training sample at each value and the ground truth box corresponding to each training sample, and use the prior box corresponding to the maximum IoU value in the set of prior boxes corresponding to each training sample at each value as the predicted box corresponding to each training sample at each value.

[0215] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0216] Cluster the sizes of the ground truth boxes corresponding to all sample images in the sample image set by the K-means algorithm to obtain the initial prior box sizes.

[0217] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0218] Obtain the target image of the target electronic component;

[0219] Input the target image into the trained scratch defect detection model, and output multiple prior boxes for indicating the scratch positions in the target image;

[0220] Determine the confidence threshold of the prior boxes; according to the confidence threshold and the confidences of the multiple prior boxes, determine the set of target prior boxes;

[0221] Screen the set of target prior boxes by the non-maximum suppression screening method to determine the final predicted boxes.

[0222] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0223] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0224] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0225] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for detecting scratch defects of electronic components, characterized in that, The method includes: Obtaining a sample image set of sample electronic components, where the sample image set includes sample images with scratch defects and sample images without scratch defects; Training a scratch defect detection model based on the average of the average precisions and the sample image set, adjusting the parameters in the scratch defect detection model, and obtaining a trained scratch defect detection model; the scratch defect detection model is constructed based on the YOLOv3 model; wherein, the YOLOv3 model includes three prediction layers, and the three prediction layers are respectively used as the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer, and the grid division scales corresponding to the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer decrease in sequence; each prediction layer corresponds to at least one size of prior box, and the parameters in the YOLOv3 model include the size of the target prior box corresponding to the second-scale prediction layer; Wherein, the training of the scratch defect detection model based on the average of the average precisions and the sample image set, adjusting the parameters in the scratch defect detection model, and obtaining a trained scratch defect detection model includes: Determining a size range based on the size of the prior box corresponding to the second-scale prediction layer and the size of the prior box corresponding to the third-scale prediction layer; Obtaining multiple values in the size range, taking each value as the value of the target prior box size, and determining the prior box set corresponding to each training sample in the sample image set under each value; wherein, for any value and any training sample, the prior box set corresponding to the any training sample under the any training sample includes all the prior boxes output by the any training sample through the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer under the any value; Determining the prediction box corresponding to each training sample under each value from the prior box set corresponding to each training sample under each value, and calculating the average of the average precisions of the sample image set under each value based on the prediction box corresponding to each training sample under each value; Determining the value corresponding to the maximum average of the average precisions and using it as the final value of the target prior box size; Performing scratch defect detection on the target electronic component according to the trained scratch defect detection model.

2. The method according to claim 1, characterized in that, The process of obtaining the sample image set includes: Obtaining a plurality of original sample images, performing a preset process on each original sample image to obtain a plurality of enhanced sample images, where the preset process includes translation, mirror flipping, brightness enhancement, or rotation by 90 degrees; Labeling the scratch defects in the plurality of original sample images and the plurality of enhanced sample images respectively through the ground truth boxes, determining the positions, sizes, and defect categories framed by the ground truth boxes; obtaining a training set based on the plurality of enhanced sample images and the plurality of original sample images, and forming the sample image set from the training set.

3. The method according to claim 1, characterized in that, The determining the prediction box corresponding to each training sample under each value from the prior box set corresponding to each training sample under each value includes: Calculate the intersection over union (IoU) between each prior box in the set of prior boxes corresponding to each training sample at each value and the ground truth box corresponding to each training sample. The prior box with the maximum IoU value in the set of prior boxes corresponding to each training sample at each value is used as the predicted box corresponding to each training sample at each value.

4. The method according to claim 1, characterized in that, The first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer each correspond to at least one initial prior box size; The process for determining the initial prior box size includes: Using the K-means algorithm to cluster the ground truth box sizes corresponding to all sample images in the sample image set to obtain the initial prior box size.

5. The method according to claim 1, characterized in that, The scratch defect detection of the target electronic component according to the trained scratch defect detection model includes: Obtaining the target image of the target electronic component; Inputting the target image into the trained scratch defect detection model to output multiple prior boxes for indicating the scratch positions in the target image; Determining the confidence threshold of the prior boxes; determining the set of target prior boxes according to the confidence threshold and the confidences of the multiple prior boxes; Screening the set of target prior boxes using the non-maximum suppression screening method to determine the final predicted box.

6. A device for detecting scratch defects of electronic components, characterized in that, The device includes: A first acquisition module for acquiring a sample image set of sample electronic components, the sample image set including sample images with scratch defects and sample images without scratch defects; A training module, which is used to train a scratch defect detection model based on the average of the average precisions and the sample image set, adjust the parameters in the scratch defect detection model, and obtain a trained scratch defect detection model; the scratch defect detection model is constructed based on the YOLOv3 model; wherein, the YOLOv3 model includes three prediction layers, and the three prediction layers are respectively used as a first-scale prediction layer, a second-scale prediction layer, and a third-scale prediction layer, and the grid division scales corresponding to the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer decrease in sequence; each prediction layer corresponds to at least one size of prior box, and the parameters in the YOLOv3 model include the target prior box size corresponding to the second-scale prediction layer; wherein, the training of the scratch defect detection model based on the average of the average precisions and the sample image set, adjusting the parameters in the scratch defect detection model, and obtaining a trained scratch defect detection model includes: determining a size range based on the prior box size corresponding to the second-scale prediction layer and the prior box size corresponding to the third-scale prediction layer; obtaining multiple values in the size range, taking each value as the value of the target prior box size, and determining the prior box set corresponding to each training sample in the sample image set under each value; wherein, for any value and any training sample, the prior box set corresponding to the any training sample under the any training sample includes all the prior boxes output by the any training sample through the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer under the any value; determining the prediction box corresponding to each training sample under each value from the prior box set corresponding to each training sample under each value, and calculating the average of the average precisions of the sample image set under each value based on the prediction box corresponding to each training sample under each value; determining the value corresponding to the maximum average of the average precisions and using it as the final value of the target prior box size; A detection module, which is used to detect the scratch defect of the target electronic component according to the trained scratch defect detection model.

7. The device according to claim 6, characterized in that, The device further includes a clustering unit, which is used for: Clustering the real box sizes corresponding to all the sample images in the sample image set through the K-means algorithm to obtain initial prior box sizes; the first-scale prediction layer, the second-scale prediction layer, and the third-scale prediction layer each correspond to at least one initial prior box size.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images