Electronic component defect detection method based on improved YOLOv5 model

Abstract

The application discloses an electronic component defect detection method based on an improved YOLOv5 model, collects an electronic component image, processes the image into an electronic component image dataset, and performs pretreatment; acquires an ImageNet dataset, a weight file and a standard YOLOv5 network structure, performs pre-training, and obtains pre-training weights; the standard YOLOv5 network structure is improved to obtain an improved YOLOv5 network structure; the pre-processed electronic component image dataset and the improved YOLOv5 network structure are used for training, and the weights obtained after training are evaluated, hyperparameter optimization is performed, and training is repeated until the recognition rate reaches more than 95%, at which time the weights are the optimal weights, and a final YOLOv5 model is obtained; the final YOLOv5 model is used for defect detection of electronic components, and corresponding detection results are obtained. The application increases the richness of training data, enhances the robustness of the model, and improves the detection speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of deep learning and object detection, and particularly relates to an electronic component defect detection method based on an improved YOLOv5 model. BACKGROUND

[0002] YOLO (You Only Look Once) is a single-stage object detection algorithm. YOLO is based on deep learning technology, which utilizes the ability of CNN (Convolutional Neural Network) to automatically learn feature representations and process image data, allowing for better capture of object context and semantic information. YOLO converts the object detection task into a regression problem by performing convolution and pooling operations on the entire image and introducing a fully connected layer at the end for object classification and localization. YOLOv5 is a popular version of the YOLO series of object detection algorithms, which balances speed and accuracy, adopts a lightweight network structure, and achieves good performance in object detection tasks.

[0003] However, the traditional YOLOv5 algorithm still has the following defects: the use of a single fixed input size results in relatively poor detection performance for smaller targets; when targets appear in dense areas of the image, during the process of matching each prediction box with the true box in the calculation of the loss function, multiple prediction boxes may be matched with the same true box, resulting in some targets being missed and the occurrence of missed detection. The backbone network of YOLOv5 is composed of CSPDarknet53, which adopts the Darknet53 structure. Although it has certain receptive field and semantic information capture ability at a lower network level, its feature extraction capability is relatively weak compared to some more advanced network structures (such as ResNet, EfficientNet), and the feature map has a long propagation path in the network, resulting in less smooth information flow and possible information bottlenecks. This may limit the network's perception ability and target representation ability, affecting the accuracy of object detection. SUMMARY

[0004] To solve the above technical problems, the present application proposes an electronic component defect detection method based on an improved YOLOv5 model, which effectively improves the precision of the model in detecting electronic component defects and improves the detection speed, meeting the requirements of high precision, high efficiency, and real-time performance in industrial electronic component defect detection.

[0005] To solve the above technical problems, the present application adopts the following technical solutions:

[0006] The electronic component defect detection method based on the improved YOLOv5 model comprises the following steps:

[0007] S1, collect electronic component images, and make the images into an electronic component image dataset, and divide the dataset into a training set and a validation set.

[0008] S2, pre-process the electronic component image dataset.

[0009] S3, obtain an ImageNet dataset and a weight file, and obtain a standard YOLOv5 network structure for pre-training to obtain pre-training weights.

[0010] S4, improve the standard YOLOv5 network structure to obtain an improved YOLOv5 network structure, and realize multi-scale feature fusion.

[0011] The improved YOLOv5 network structure comprises a feature backbone network, a feature fusion module and a detection head from top to bottom; wherein the feature backbone network comprises an EfficientNet module and a Squeeze-and-Excitation (Squeeze-and-Excitation) module, the result output by the EfficientNet module is input into the Squeeze-and-Excitation module for extracting the features of the input image; the feature fusion module comprises a BiFPN (Bi-directional Feature Pyramid Network) module for feature fusion, which enhances the feature information of different scales; the detection head comprises a softer NMS (Soft Non-Maximum Suppression) algorithm for predicting part of the target bounding box, processing and decoding the extracted features to produce the final target detection result.

[0012] S5, use the pre-training weights as initial weights, use the pre-processed electronic component image dataset to train the improved YOLOv5 network structure, evaluate the weights obtained after training, optimize the hyperparameters, and repeat the training until the recognition rate reaches more than 95%, at which time the weights are the optimal weights, and the final YOLOv5 model is obtained.

[0013] S6, use the final YOLOv5 model to detect defects of electronic components to obtain corresponding detection results.

[0014] Further, in step S1, making the electronic component image dataset comprises the following contents:

[0015] An industrial camera is used to collect electronic component images, including images of normal electronic components and different types of defective electronic components under different angles and different backgrounds, and each defective electronic component image is labeled with a corresponding error label to obtain an electronic component image dataset.

[0016] Further, in step S2, the preprocessing of the electronic component image dataset includes the following sub-steps:

[0017] S201, image cropping and scaling: performing scaling operation on the electronic component image dataset, and cropping or padding the image according to actual application requirements.

[0018] S202, image enhancement: improving robustness and generalization ability through image enhancement technology, adjusting the brightness, contrast and saturation of the images in the electronic component image dataset, and performing random flipping, rotation and scaling operation to expand the dataset.

[0019] S203, image normalization operation: normalizing the numerical range of the pixels in the images in the electronic component image dataset, scaling the pixels from the range [0, 255] to the range [0, 1] or [-1, 1].

[0020] Further, in step S3, obtaining the pre-trained weights includes the following sub-steps:

[0021] S301, obtaining the ImageNet dataset and weight file, adjusting the image size of the ImageNet dataset to match the size of the images in the preprocessed electronic component image dataset.

[0022] S302, obtaining the standard YOLOv5 network structure.

[0023] S303, learning rate scheduling: using a predefined learning rate scheduling strategy to adjust the initial learning rate to 0.000001; gradually increasing the learning rate, with a higher learning rate in the early stage to quickly converge, and the learning rate in this stage being in the range of [0.01, 0.007]; with a lower learning rate in the later stage for fine correction, and the learning rate in this stage being in the range of [0.001, 0.003]; and dynamically adjusting the learning rate according to the performance indicators in the training process.

[0024] S304, weight initialization: using the weights in step S301 as the initial weights.

[0025] S305, using the adjusted ImageNet dataset, pre-training the standard YOLOv5 network structure using the predefined learning rate scheduling strategy to obtain the pre-trained weights.

[0026] Further, in step S4, the network depth, network width and input resolution of the EfficientNet module are adjusted to achieve efficient and accurate network structure design, and the specific formula is:

[0027] depth:D=αφ

[0028] width:W=βφ

[0029] resolution:R=γφ

[0030] wherein depth represents the network depth; width represents the network width; resolution represents the input resolution; D represents the abbreviation of the network depth; W represents the abbreviation of the network width; R represents the abbreviation of the input resolution; a represents the scaling factor of the network depth; β represents the scaling factor of the network width; γ represents the scaling factor of the input resolution; and φ represents the scaling factor, which controls the number of network structure layers, including convolution layers, batch normalization layers, activation function layers, etc.

[0031] Further, in step S4, the Squeeze-and-Excitation module includes the following contents:

[0032] The Squeeze-and-Excitation module is an attention mechanism module for improving the performance of a convolutional neural network, mainly including two steps: compression and excitation.

[0033] (1) Compression (Squeeze): The Squeeze-and-Excitation module performs a global average pooling operation on the feature map output by the EfficientNet module, and compresses the spatial dimension into a feature vector. If the size of the feature map is HxWxC, wherein H and W represent the height and width, and C represents the number of channels, then the compression operation will obtain a feature vector with a size of 1x1xC. The specific formula is:

[0034] z=GlobalAvgPool(X)

[0035] wherein X represents the input feature map, z represents the output feature vector, and GlobalAvgPool represents the global average pooling operation.

[0036] (2) Excitation (Excitation): The feature vector is subjected to a nonlinear transformation to learn the correlation between channels for calculating channel weights.

[0037] First, the feature vector z output by the compression stage is input to FC1 (Fully Connected 1, first fully connected layer) for mapping, so that the model can more flexibly learn feature combinations, and the specific formula is:

[0038] f=FC1(z)=W1z+b1

[0039] Wherein, FC1 represents the first full connection layer, used for mapping the feature vector; f represents the feature vector after FC1 mapping; W1 and b1 represent the weight and bias of FC1 respectively.

[0040] Then, the non-linear activation function RELU is used to increase the non-linearity of the feature vector f, and the specific formula is:

[0041] a=RELU(f)

[0042] Wherein, a represents the feature vector after the activation function processing.

[0043] Then, a is input into the second full connection layer FC2 for linear transformation again, and the non-linear activated feature vector a is mapped into the channel weight vector s, and the specific formula is:

[0044] s=FC2(a)=W2z+b2

[0045] Wherein, FC2 represents the second full connection layer, used for mapping the channel weight feature vector; W2 and b2 represent the weight and bias of FC2 respectively.

[0046] The channel weight vector s describes the importance of each channel to the corresponding position in the feature map. By using two full connection layers, the correlation between channels can be learned more accurately, and the channel weight suitable for the target task can be found.

[0047] Finally, the sigmoid activation function is used to limit the value of the channel weight vector s to [0, 1], which represents the importance of the channel, and the specific formula is:

[0048] δ=sigmod(s)

[0049] Wherein, δ represents the channel weight vector after the Sigmoid activation function processing.

[0050] Further, in step S4, the feature fusion module includes the following contents:

[0051] The BiFPN module includes an up-sampler and a down-sampler, which can gradually reduce and improve the resolution of the feature map.

[0052] The down-sampler converts the feature map with high resolution into a feature map with lower resolution but richer semantic information through down-sampling operation, so as to capture more extensive context information.

[0053] The up-sampler enlarges the feature map with low resolution but rich semantic information through an up-sampling operation to a feature map with high resolution but relatively less context information, which helps to provide more accurate location information.

[0054] The BiFPN module constructs a top-down and bottom-up feature fusion network by introducing bidirectional paths; by introducing bidirectional connections and feature fusion operations, the BiFPN can solve the problems of feature information loss and redundancy that the FPN (Feature Pyramid Network) may encounter in the target detection task.

[0055] In the BiFPN module, both horizontal and vertical connections are introduced to facilitate cross-level feature interaction and information transmission. The horizontal connection fuses the feature maps of different levels together through up-sampling and down-sampling operations, and fuses the detailed information of the bottom layer into the high-level features. The vertical connection fuses the features of the bottom and high layers together through skip connection between different scale feature maps to realize multi-level feature joint of the feature map. Through the BiFPN module with multiple iterations, the improved YOLOv5 network structure can generate feature maps with rich semantic information and accurate location information on different scales to perform subsequent target detection and recognition.

[0056] The channel weight vector δ processed by the Sigmoid activation function is input into the feature fusion module, the up-sampler upsamples the vector to the same size as the feature map of the previous level, and the feature fusion is performed to obtain the fused feature, which is input into the down-sampler to down-sample to the same size as the feature map of the next level, and the feature fusion is performed to obtain the output result of the BiFPN.

[0057] Further, in step S4, the detection head includes the following contents:

[0058] In the detection head, the introduction of the softer NMS algorithm can reduce the conflict between overlapping bounding boxes and realize post-processing optimization.

[0059] First, the softer NMS algorithm is used to generate a set of bounding boxes and corresponding confidence scores, and the calculation formula of the confidence score is:

[0060] score=Pr(object)*Pr(class|object)*IoU

[0061] Wherein, score represents the confidence score; Pr(object) represents the probability that the defect image is contained in the bounding box; Pr(class|object) represents the probability that the defect image belongs to a certain specific class given that the defect image is contained in the bounding box; and IoU represents the intersection over union between the predicted bounding box and the real bounding box.

[0062] All the bounding boxes are sorted in descending order of the confidence score to obtain a sorted bounding box list, the IoU (Intersection over Union) overlap value of each bounding box with the bounding box with a lower confidence score is calculated, if the IoU overlap value is higher than a threshold, the low confidence score is attenuated, and the bounding box with the low confidence score is retained; the overlapping bounding boxes are continuously traversed and attenuated until all the bounding boxes are traversed.

[0063] Finally, the retained bounding boxes are sorted again according to the attenuated confidence scores to obtain the de-overlapped bounding boxes.

[0064] Further, in step S5, obtaining the final YOLOv5 model includes the following sub-steps:

[0065] S501, taking the pre-training weight as the initial weight of the improved YOLOv5 network structure, configuring the hyperparameters of the model according to the requirements of identifying electronic components, including input image size, anchor box setting, and class number.

[0066] S502, defining the loss function and additional loss items, selecting the optimizer and setting the learning rate.

[0067] S503, constructing a data loader to load the preprocessed electronic component image data set and the corresponding label information.

[0068] S504, iteratively traversing the training set, performing weight training and parameter updating on the improved YOLOv5 network structure, calculating the loss between the predicted result and the real label, and performing back propagation to update the weight. Evaluate the performance of the improved YOLOv5 network structure on the validation set, evaluate the weight, including calculating the precision, recall, and mAP (Mean Average Precision, average precision mean); until the recognition rate reaches more than 95%, the weight at this time is the final weight, and the final YOLOv5 model is obtained for subsequent deployment and application.

[0069] Further, in step S6, obtaining the corresponding detection result includes the following contents:

[0070] The final YOLOv5 model is deployed to a specific production or use environment as a target detection model for detecting electronic component defects, the target to be detected is input into the model for detection, and the confidence score obtained is used to judge the corresponding defect type.

[0071] Compared with the prior art, the present application has the beneficial effects that:

[0072] 1、The present application adopts a predefined learning rate scheduling strategy, and dynamically adjusts the learning rate according to the performance index in the training process, speeds up the training process of the model, provides good feature initialization for the target detection task, and improves the model generalization ability.

[0073] 2、The present application optimizes the YOLOv5 network structure, introduces EfficientNet as the backbone network of the network, balances the model complexity and accuracy, and lightens the module structure, reduces the calculation amount and memory occupation while maintaining high accuracy;And introduce Squeeze-and-Excitation module to add attention mechanism to the model, dynamically learn the importance between channels, and enhance the feature expression ability of the convolutional neural network through adaptive channel weight adjustment, which can improve the network performance without increasing additional calculation burden;At the same time, the BiFPN module is introduced instead of the original FPN module, which realizes multi-scale feature fusion and adaptive feature selection, effectively improves the robustness and efficiency of target detection;Finally, the softer NMS algorithm is introduced for post-processing, which reduces the confidence score of overlapping bounding boxes and retains more targets, making the target detection positioning process more accurate.

[0074] 3、The design method of the present application has universality, and the improved model has better generalization performance, which makes it easy to deploy to other target detection tasks, and provides application examples and solutions for small object target detection in industrial applications. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 is the overall implementation flowchart of the present application.

[0076] Figure 2 is the structure diagram of the EfficientNet module in the network structure of the present application.

[0077] Figure 3 is the structure diagram of the Squeeze-and-Excitation module in the network structure of the present application. DETAILED DESCRIPTION

[0078] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0079] The application provides an electronic component defect detection method based on an improved YOLOv5 model, as shown in the formula (I): Figure 1 The application provides an electronic component defect detection method based on an improved YOLOv5 model, as shown in the formula (I):

[0080] S1, collect electronic component images, and make the images into an electronic component image dataset, and divide the dataset into a training set and a validation set, and the specific content is:

[0081] An industrial camera is used to collect electronic component images on a factory assembly line, including images of normal electronic components and images of electronic components with different types of defects under different angles and different backgrounds, and each defective electronic component image is labeled with a corresponding error label, including short circuit, open circuit, missing hole, mouse bite, pseudo-copper and stray, to obtain an electronic component image dataset.

[0082] S2, pre-process the electronic component image dataset, and the specific content is:

[0083] S201, image cropping and scaling: scale the electronic component image dataset, and crop or fill the image to make the image size 640*640.

[0084] S202, image enhancement: improve the robustness and generalization ability through image enhancement technology, adjust the brightness, contrast and saturation of the images in the electronic component image dataset to 1.5 times the original, and perform random flipping, rotation and scaling operations to expand the dataset.

[0085] S203, image normalization operation: normalize the numerical range of the image pixels in the electronic component image dataset, and scale the pixels from the range [0, 255] to the range [0, 1] or [-1, 1].

[0086] S3, download the ImageNet dataset and weight file from the ImageNet website, and obtain the standard YOLOv5 network structure from the official website for pre-training to obtain the pre-training weight, and the specific content is:

[0087] S301, adjust the image size of the ImageNet dataset to 640*640 to match the size of the images in the pre-processed electronic component image dataset.

[0088] S302, obtain the standard YOLOv5 network structure from the YOLOv5 website.

[0089] S303, learning rate scheduling: using a predefined learning rate scheduling strategy, adjusting the initial learning rate to 0.000001; gradually increasing the learning rate, controlling the learning rate to be no more than 0.01, the early stage converges quickly with a higher learning rate, at this time the size of the learning rate is in the range of [0.01, 0.007]; the later stage is fine-tuned with a lower learning rate, at this time the size of the learning rate is in the range of [0.001, 0.003]; and dynamically adjusting the learning rate according to the performance index in the training process.

[0090] S304, weight initialization: the weight in step S301 is used as the initial weight.

[0091] S305, using the adjusted ImageNet dataset, using a predefined learning rate scheduling strategy to pre-train the standard YOLOv5 network structure, and obtaining pre-trained weights.

[0092] S4, improving the standard YOLOv5 network structure to obtain an improved YOLOv5 network structure, realizing multi-scale feature fusion;

[0093] The improved YOLOv5 network structure includes a feature backbone network, a feature fusion module and a detection head from top to bottom; wherein the feature backbone network includes an EfficientNet module and a Squeeze-and-Excitation module, the output of the EfficientNet module is input into the Squeeze-and-Excitation module; the feature fusion module includes a BiFPN module; the detection head includes a softer NMS algorithm. The specific content is:

[0094] As shown in Figure 2 , using the PyTorch tool of Python language, according to the size of the collected electronic component image, the structure of the EfficientNet module is constructed in turn, including input layer, convolution layer, weak connection layer, expansion convolution layer, regularization layer, pooling layer and full connection layer.

[0095] Adjust the network depth, network width and input resolution ratio of the EfficientNet module to realize efficient and accurate network structure design, the specific formula is:

[0096] depth:D=αφ

[0097] width:W=βφ

[0098] resolution:R=γφ

[0099] wherein, depth represents the network depth; width represents the network width; resolution represents the input resolution; D represents the abbreviation of the network depth; W represents the abbreviation of the network width; R represents the abbreviation of the input resolution; a represents the scaling factor of the network depth; b represents the scaling factor of the network width; g represents the scaling factor of the input resolution; and f represents the scaling factor, which controls the number of layers of the network structure, including the convolution layer, the batch normalization layer and the activation function layer.

[0100] In the embodiment, a=1.2, b=1.1, and g=1.15.

[0101] The Squeeze-and-Excitation module is an attention mechanism module for improving the performance of a convolutional neural network. As shown in FIG. 1, the Squeeze-and-Excitation module is constructed using the PyTorch tool of the Python language, including a squeeze module, an excitation module and a scaling module, wherein the squeeze module includes a global pooling operation, the excitation module includes two fully connected layers, a RELU function and a sigmoid function. Figure 3

[0102] (1) Squeeze: The Squeeze-and-Excitation module performs a global average pooling operation on the feature map output by the EfficientNet module, and compresses the spatial dimension into a feature vector. If the size of the feature map is HxWxC, wherein H and W represent the height and width, and C represents the number of channels, then the compression operation will obtain a feature vector with a size of 1x1xC. The specific formula is as follows:

[0103] z=GlobalAvgPool(X)

[0104] wherein, X represents the input feature map, z represents the output feature vector, and GlobalAvgPool represents the global average pooling operation.

[0105] (2) Excitation: The feature vector is subjected to a nonlinear transformation to learn the correlation between channels and calculate the channel weight.

[0106] First, the feature vector z output by the squeeze stage is input into the first fully connected layer FC1 for mapping, so that the model can more flexibly learn the feature combination. The specific formula is as follows:

[0107] f=FC1(z)=W1z+b1

[0108] wherein, FC1 represents the first fully connected layer for mapping the feature vector; f represents the feature vector after the mapping by FC1; W1 and b1 represent the weight and bias of FC1, respectively.​

[0109] Then, the nonlinearity of the feature vector f is increased by using the nonlinear activation function RELU, and the specific formula is:

[0110] a = RELU(f)

[0111] Wherein, a represents the feature vector after the activation function processing.

[0112] Then, the nonlinearity of the feature vector f is increased by using the nonlinear activation function RELU, and the specific formula is:

[0113] s = FC2(a) = W2z + b2

[0114] Wherein, FC2 represents the second fully connected layer, which is used to map the channel weight feature vector; W2 and b2 represent the weight and bias of FC2, respectively.

[0115] Finally, the value of the channel weight vector s is limited to [0, 1] by using the sigmoid activation function, which represents the importance of the channel, and the specific formula is:

[0116] δ = sigmod(s)

[0117] Wherein, δ represents the channel weight vector after the Sigmoid activation function processing.

[0118] The BiFPN module includes an up-sampler and a down-sampler, which can gradually reduce and improve the resolution of the feature map.

[0119] The BiFPN module introduces a top-down and bottom-up feature fusion network by introducing a bidirectional path; by introducing bidirectional connection and feature fusion operation, BiFPN can solve the problem of feature information loss and redundancy that FPN may encounter in target detection task.

[0120] In the BiFPN module, both horizontal and vertical connections are introduced to promote cross-level feature interaction and information transmission. The horizontal connection fuses the feature maps of different levels together through up-sampling and down-sampling operations, and fuses the detailed information of the bottom layer into the high-level features. The vertical connection fuses the features of the bottom and high layers together through skip connection between different scale feature maps to realize multi-level feature joint of feature maps. Through the BiFPN module with multiple iterations, the improved YOLOv5 network structure can generate feature maps with rich semantic information and accurate position information on different scales to perform subsequent target detection and recognition.

[0121] The channel weight vector δ after Sigmoid activation function processing is input into the feature fusion module. The upsampler upsamples the vector to the same size as the feature map of the previous level, performs feature fusion to obtain the fused feature, and inputs the feature into the downsampler to downsample it to the same size as the feature map of the next level, and performs feature fusion to obtain the output result of BiFPN.

[0122] In the detection head, the introduction of the softer NMS algorithm can reduce the conflicts between overlapping bounding boxes and achieve post-processing optimization.

[0123] First, a set of bounding boxes and corresponding confidence scores are generated using the softer NMS algorithm. The confidence score is calculated as follows:

[0124] score=Pr(object)*Pr(class|object)*IoU

[0125] Among them, score represents the confidence score; Pr(object) represents the probability that the bounding box contains a defect image; Pr(class|object) represents the probability that the defect image belongs to a specific class under the condition that the given bounding box contains a defect image; IoU represents the intersection over union ratio between the predicted bounding box and the true bounding box.

[0126] Sort all bounding boxes from high to low according to their confidence scores, obtain a sorted list of bounding boxes, calculate the IoU overlap value of each bounding box with the bounding box with a lower confidence score than it, and if the IoU overlap value is higher than the threshold, attenuate the low confidence score and retain the bounding box with the low confidence score; continue to traverse and attenuate the overlapping bounding boxes until all bounding boxes are traversed.

[0127] Finally, the retained bounding boxes are sorted again according to the attenuated confidence scores to obtain the de-overlapped bounding boxes.

[0128] In this embodiment, the threshold is set to 0.5.

[0129] S5. Use the pre-trained weights as the initial weights and the pre-processed electronic component image dataset to train the improved YOLOv5 network structure. Evaluate the weights obtained after training and perform hyperparameter optimization. Repeat the training until the recognition rate reaches more than 95%. The weights at this time are the final weights, and the final YOLOv5 model is obtained. The specific content is:

[0130] S501. Use the pre-trained weights as the initial weights of the improved YOLOv5 network structure. Configure the model's hyperparameters, including input image size, anchor box settings, and number of categories, based on the requirements for identifying electronic components.

[0131] S502, define a loss function suitable for the electronic component defect detection task, including GIoULoss (generalized intersection over union loss function) and FocalLoss (focal loss function). Define additional loss items, including the weights of class loss and box regression loss. Select an optimizer suitable for the task, including SGD (Stochastic Gradient Descent) and Adam (adaptive moment estimation), and set the learning rate. The specific formula of the loss function is:

[0132] GIoU = IoU - (C-Union) / C

[0133]

[0134] where C represents the minimum closed box of the predicted bounding box and the real bounding box; Union represents the union of the predicted bounding box and the real bounding box; pt represents the probability of the predicted class being a positive sample; is an adjustment factor set by a hyperparameter,

[0135] S503, use the PyTorch data loading framework DataLoader to build a data loader and load the preprocessed image data set and corresponding label information in the electronic component image data set.

[0136] S504, iterate through the training set, perform weight training and parameter update on the improved YOLOv5 network structure, calculate the loss between the predicted result and the real label, and perform back propagation to update the weights. According to the training strategy, adjust the learning rate to control the convergence and stability of the training process. Evaluate the performance of the improved YOLOv5 network structure on the validation set, evaluate the weights, including calculating the precision, recall, and mAP (mean average precision); until the recognition rate reaches more than 95%, the weight at this time is the best weight, and the final YOLOv5 model is obtained for subsequent deployment and application.

[0137] In this example, the training batch is set to 64. Observe the performance of the model on the validation set through the loss functions GIoULoss and FocalLoss. The smaller the value of GIoULoss, the higher the degree of overlap between the predicted bounding box and the real bounding box, and the more accurate the predicted bounding box of the model coincides with the real bounding box. The smaller the value of FocalLoss, the more attention the model pays to positive samples and negative samples in the classification problem.

[0138] S6, deploy the final YOLOv5 model as a target detection model for detecting electronic component defects to a specific production or use environment, input the target to be detected into the model for detection, and judge the corresponding defect type according to the obtained confidence score.

[0139] The above merely describes exemplary embodiments of the present application, and does not limit the patent protection scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, which is made by using the content of the specification and drawings, is also included in the patent protection scope of the present application.

Claims

1. An electronic component defect detection method based on an improved YOLOv5 model is characterized in that: include: S1. Collect electronic component images and create an electronic component image dataset from the images, and divide the dataset into a training set and a validation set; S2, preprocessing the electronic component image dataset; S3. Obtain the ImageNet dataset and weight file, and obtain the standard YOLOv5 network structure for pre-training to obtain pre-training weights; S4. Improve the standard YOLOv5 network structure to obtain an improved YOLOv5 network structure, which includes a feature backbone network, a feature fusion module, and a detection head from top to bottom; The feature backbone network includes the EfficientNet module and the Squeeze-and-Excitation module. The output of the EfficientNet module is input into the Squeeze-and-Excitation module. The feature fusion module includes the BiFPN module; the detection head includes the softer NMS algorithm; The Squeeze-and-Excitation module includes the following: The Squeeze-and-Excitation module performs a global average pooling operation on the feature map output by the EfficientNet module, compressing the spatial dimension into a feature vector. The specific formula is: z=GlobalAvgPool(X) Among them, X represents the input feature map, z represents the output feature vector, and GlobalAvgPool represents the global average pooling operation; The feature vector z is input into the first fully connected layer for mapping. The specific formula is: f=W1z+b1 Where f represents the feature vector after mapping by the first fully connected layer; W1 and b1 represent the weight and bias of the first fully connected layer respectively; The nonlinear activation function RELU is used to increase the nonlinearity of the feature vector f. The specific formula is: a=RELU(f) Among them, a represents the feature vector after being processed by the activation function; The feature vector a is input into the second fully connected layer for linear transformation and mapped into a channel weight vector. The specific formula is: s=W2z+b2 Where W2 and b2 represent the weight and bias of the second fully connected layer, respectively, and s represents the channel weight vector after linear transformation; The sigmoid activation function is used to limit the value of the channel weight vector s to [0,1]. The specific formula is: δ=sigmod(s) Among them, δ represents the channel weight vector after processing by the Sigmoid activation function; S5. Use the pre-trained weights as the initial weights and the pre-processed electronic component image dataset to train the improved YOLOv5 network structure. Then evaluate the weights obtained after training and perform hyperparameter optimization. Repeat the training until the recognition rate reaches more than 95%. The weights at this time are the final weights, and the final YOLOv5 model is obtained. S6. Use the final YOLOv5 model to perform defect detection on electronic components and obtain corresponding detection results.

2. The electronic component defect detection method based on the improved YOLOv5 model according to claim 1, characterized in that: In step S1, creating an electronic component image dataset includes the following: Industrial cameras are used to capture images of electronic components, including normal images and images with different types of defects at different angles and backgrounds. Each defective electronic component image is annotated with a corresponding error label to obtain an electronic component image dataset.

3. The electronic component defect detection method based on the improved YOLOv5 model according to claim 1, characterized in that: In step S2, preprocessing the electronic component image dataset includes the following sub-steps: S201, performing a scaling operation on an electronic component image dataset, and cropping or filling the image; S202, using image enhancement technology to adjust the brightness, contrast, and saturation of the images in the electronic component image dataset, and perform random flipping, rotation, and scaling operations; S203 , normalizing the numerical range of image pixels in the electronic component image dataset, and scaling the pixels from a range of [0, 255] to a range of [0, 1] or [-1, 1].

4. The electronic component defect detection method based on the improved YOLOv5 model according to claim 1, characterized in that: In step S3, obtaining the pre-training weights includes the following sub-steps: S301, obtaining an ImageNet dataset and a weight file, and adjusting the image size of the ImageNet dataset to match the size of the images in the preprocessed electronic component image dataset; S302, obtaining a standard YOLOv5 network structure; S303, using a predefined learning rate scheduling strategy, adjusting the initial learning rate to 0.000001; gradually increasing the learning rate, fast convergence at a higher learning rate in the early stage, at which time the learning rate is in the range of [0.01, 0.007]; fine-tuning at a lower learning rate in the later stage, at which time the learning rate is in the range of [0.001, 0.003]; S304, taking the weight in step S301 as the initial weight; S305. Use the adjusted ImageNet dataset and a predefined learning rate scheduling strategy to pre-train the standard YOLOv5 network structure to obtain pre-training weights.

5. The electronic component defect detection method based on the improved YOLOv5 model according to claim 1, characterized in that: In step S4, the ratio of the network depth, network width, and input resolution of the EfficientNet module is adjusted. The specific formula is: depth:D=α φ width:W=β φ resolution:R=γ φ Among them, depth represents the network depth, width represents the network width, resolution represents the input resolution, D represents the abbreviation of network depth, W represents the abbreviation of network width, R represents the abbreviation of input resolution, α represents the scaling factor of network depth, β represents the scaling factor of network width, γ represents the scaling factor of input resolution, and φ represents the scaling factor.

6. The electronic component defect detection method based on the improved YOLOv5 model according to claim 1, characterized in that: In step S4, the feature fusion module Includes the following: The BiFPN module includes an upsampler and a downsampler; The output result of the feature backbone network is input into the feature fusion module. The upsampler upsamples the vector to the same size as the feature map of the previous level, performs feature fusion to obtain the fused feature, and inputs the feature into the downsampler to downsample it to the same size as the feature map of the next level, and performs feature fusion to obtain the output result of BiFPN.

7. The electronic component defect detection method based on the improved YOLOv5 model according to claim 1, characterized in that: In step S4, the detection head includes the following contents: In the detection head, the softer NMS algorithm is used to generate a set of bounding boxes and corresponding confidence scores. The confidence score is calculated as follows: score=Pr(object)*Pr(class|object)*IoU Where score represents the confidence score; Pr(object) represents the probability that the bounding box contains a defect image; Pr(class|object) represents the probability that the defect image belongs to a specific class given that the bounding box contains a defect image; IoU represents the intersection over union ratio between the predicted bounding box and the true bounding box; Sort all bounding boxes by confidence score from high to low, obtain a sorted list of bounding boxes, calculate the IoU overlap value of each bounding box with the bounding box with a lower confidence score than its confidence score, and if the IoU overlap value is higher than the threshold, attenuate the low confidence score and retain the bounding box with the low confidence score; continue to traverse and attenuate the overlapping bounding boxes until all bounding boxes are traversed; The retained bounding boxes are sorted again according to the attenuated confidence scores to obtain the de-overlapped bounding boxes.

8. The electronic component defect detection method based on the improved YOLOv5 model according to claim 1, characterized in that: In step S5, obtaining the final YOLOv5 model includes the following sub-steps: S501, using the pre-trained weights as the initial weights of the improved YOLOv5 network structure, and configuring the model's hyperparameters, including input image size, anchor box settings, and number of categories; S502, define the loss function and additional loss items, select the optimizer and set the learning rate; S503: Construct a data loader to load the pre-processed images and corresponding annotation information from the electronic component image dataset; S504, iteratively traverse the training set, perform weight training and parameter update on the improved YOLOv5 network structure, calculate the loss between the prediction results and the true annotations, and perform backpropagation to update the weights; evaluate the performance of the improved YOLOv5 network structure on the validation set, and evaluate the weights, including calculating the accuracy, recall rate, and average precision mean; until the recognition rate reaches more than 95%, the weight at this time is the final weight, and the final YOLOv5 model is obtained.

9. The electronic component defect detection method based on the improved YOLOv5 model according to claim 1, characterized in that: In step S6, the corresponding detection results include the following: The final YOLOv5 model is used as the target detection model for detecting defects in electronic components. The target to be detected is input into the model for detection, and the corresponding defect type is determined based on the obtained confidence score.

Images