A PCB board surface defect detection method based on improved YOLOv5

Abstract

The present invention discloses a PCB board surface defect detection method based on improved YOLOv5. The feature fusion network PANet of the improved YOLOv5 is the BiFPN structure, and the SimAM attention mechanism is added. After optimizing the spatial pyramid pooling model SPPF, it becomes the SPPFCSPC model. Moreover, the P4 and P5 structures in the head for detecting medium and large targets are reduced, enabling it to more accurately extract and detect the defects of small targets to make up for the problem of insufficient defect detection scale. At the same time, in order to further solve the small target defect detection, the training method is optimized. The method of transfer learning is adopted to perform multiple transfer processing on the training weights and parameters of the training network, making the short-term training more sufficient.

Description

Technical Field

[0001] The present invention belongs to the technical field of pattern recognition, and particularly relates to a method for detecting surface defects of a PCB board. Background Art

[0002] Printed circuit boards are an important part of electronic products, and their quality inspection becomes a prerequisite for ensuring the long-term normal operation of electronic products. Printed circuit boards (PCBs) are mainly composed of two types of materials: insulating substrates and conductors, and play a role in supporting and interconnecting in various electronic products. Before the emergence of printed circuit boards, the interconnection between early electronic components was completed by directly connecting ordinary wires, which had problems such as large size, complex assembly process, and poor circuit stability. However, with the rapid development of industrial automation manufacturing technology, the manufacturing of electronic products has started to develop towards the direction of multi-function, integration, and intelligence. Among them, as a basic component of electronic products, the quality of printed circuit boards directly affects the performance of electronic products. However, the manufacturing process of PCBs is very complex and is inevitably affected by various factors during the entire production and manufacturing process. Quality inspection of them is a very important task. Although deep learning object detection methods have made great development in the field of PCB surface defect detection, there are still problems such as lack of labeled sample data, limited feature learning ability, difficult training of deep networks, and imbalance between detection accuracy and speed. The targets are small and difficult to detect, and the foreground and background are too close. By drawing on the advantages of COD camouflaged target detection and small target detection for fusion, attention helps better locate and identify small target defects. Summary of the Invention

[0003] In order to overcome the deficiencies of the prior art, the present invention provides a method for detecting surface defects of a PCB board based on improved YOLOv5. The feature fusion network PANet of YOLOv5 is improved to a BiFPN structure, and a SimAM attention mechanism is added. After optimizing the spatial pyramid pooling model SPPF, it becomes the SPPFCSPC model, and the P4 and P5 structures in the head for detecting medium and large targets are reduced, enabling it to more accurately extract and detect small target defects to make up for the problem of insufficient defect detection scale. At the same time, in order to further solve the problem of small target defect detection, the training method is optimized, and the method of transfer learning is adopted to perform multiple transfer processing on the training weights and parameters of the training network, making the degree of short-term training more sufficient.

[0004] The technical solutions adopted by the present invention to solve its technical problems include the following steps:

[0005] Step 1: Obtain a data set and perform data preprocessing;

[0006] Step 1-1: Intercept, flip, and map the obtained original dataset of PCB board surface defects, convert the dataset labels, and divide the dataset; divide the dataset into training, validation, and test sets;

[0007] Step 1-2: Adopt the mosaic data augmentation method, use the data of 4 pictures, and splice them in a way of random scaling, random cropping, and random distribution to combine 4 pictures into one picture;

[0008] Step 1-3: Uniformly scale the dataset images to the same size, and then send them into the detection network YOLOv5s_BSSs;

[0009] The specific description of the detection network YOLOv5s_BSSs is as follows;

[0010] Step 2: In the YOLOv5 backbone, replace the original Focus operation with a convolutional layer, add the SimAM attention mechanism, and optimize the spatial pyramid pooling structure SPPF to SPPFCSPC. The specific process is as follows:

[0011] Step 2-1: For the 0th layer, the module parameters args[64,6,2,2] of [-1,1,Conv,[64,6,2,2]] are parsed into [3,32,6,2,2]. 64 represents the number of convolutional kernels, and the final number of convolutional kernels is multiplied by width_multiple, which is 64×0.5 = 32, that is, the output dimension of the 0th layer; 6 is the convolutional kernel size, 2 is the convolutional kernel stride, and 3 is the input dimension of this layer because the input image is in RGB 3 channels of color;

[0012] Step 2-2: For the 1st layer, the module parameters [128,3,2] of [-1,1,Conv,[128,3,2]] are parsed into [32,64,3,2]. 32 is the number of channels, that is, the output dimension of the previous layer; 128 represents the number of convolutional kernels, and the final number of convolutional kernels is 128×0.5 = 64, that is, the output dimension of the 1st layer;

[0013] Step 2-3: For the 2nd layer, the module parameter

[128] of [-1,3,C3,

[128] ] is parsed into [64,64,1]. 64 is the number of channels, that is, the output dimension of the previous layer; 128 represents the number of convolutional kernels, and the final number of convolutional kernels is 128×0.5 = 64, that is, the output dimension of the 2nd layer;

[0014] Step 2-4: For the third layer, the module parameters [256, 3, 2] of [-1, 1, Conv, [256, 3, 2]] are parsed into [64, 128, 3, 2]. 64 is the number of channels, i.e., the dimension of the output of the previous layer; 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 256 × 0.5 = 128, which is the dimension of the output of the third layer.

[0015] Step 2-5: For the fourth layer, the module parameters

[256] of [-1, 6, C3,

[256] ] are parsed into [128, 128, 2]. 128 is the number of channels, i.e., the dimension of the output of the previous layer; 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 256 × 0.5 = 128, which is the dimension of the output of the fourth layer.

[0016] Step 2-6: For the fifth layer, the module parameters [512, 3, 2] of [-1, 1, Conv, [512, 3, 2]] are parsed into [128, 256, 3, 2]. 128 is the number of channels, i.e., the dimension of the output of the previous layer; 512 represents the number of convolutional kernels, and the final number of convolutional kernels is 512 × 0.5 = 256, which is the dimension of the output of the fifth layer.

[0017] Step 2-7: For the sixth layer, the module parameters

[512] of [-1, 9, C3,

[512] ] are parsed into [256, 256, 3]. 256 is the number of channels, i.e., the dimension of the output of the previous layer; 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 512 × 0.5 = 256, which is the dimension of the output of the sixth layer.

[0018] Step 2-8: For the seventh layer, the module parameters [1024, 3, 2] of [-1, 1, Conv, [1024, 3, 2]] are parsed into [256, 512, 3, 2]. 256 is the number of channels, i.e., the dimension of the output of the previous layer; 1024 represents the number of convolutional kernels, and the final number of convolutional kernels is 1024 × 0.5 = 512, which is the dimension of the output of the seventh layer.

[0019] Step 2-9: For the eighth layer, the module parameters

[1024] of [-1, 3, C3,

[1024] ] are parsed into [512, 512, 1]. 512 is the number of channels, i.e., the dimension of the output of the previous layer; 1024 represents the number of convolutional kernels, and the final number of convolutional kernels is 1024 × 0.5 = 512, which is the dimension of the output of the eighth layer.

[0020] Step 2-10: For the ninth layer, the module parameters

[1024] of [-1, 1, SimAM,

[1024] ] are parsed into [512, 512]. 512 is the number of channels, i.e., the dimension of the output of the previous layer; 1024 represents the number of convolutional kernels, and the final number of convolutional kernels is 1024 × 0.5 = 512, which is the dimension of the output of the ninth layer.

[0021] Step 2-11: For the 10th layer, the module parameters [1024, 5] of [-1, 1, SPPFCSPC, [1024, 5]] are parsed into [512, 512, 5]. 512 is the number of channels, which is the dimension of the output of the previous layer; 1024 represents the number of convolutional kernels, and the final number of convolutional kernels is 1024 × 0.5 = 512, which is the dimension of the output of the 10th layer.

[0022] Step 3: In the Neck of YOLOv5, replace the original Concat with BiFPN and remove the P4 and P5 structures to reduce the network scale. The specific process is as follows:

[0023] Step 3-1: For the 11th layer, the module parameters [512, 1, 1] of [-1, 1, Conv, [512, 1, 1]] are parsed into [512, 256, 1, 1]. 512 is the number of channels, which is the dimension of the output of the previous layer; 512 represents the number of convolutional kernels, and the final number of convolutional kernels is 512 × 0.5 = 256, which is the dimension of the output of the 11th layer.

[0024] Step 3-2: For the 12th layer, the module parameters of [-1, 1, nn.Upsample, [None, 2, nearest]] remain unchanged.

[0025] Step 3-3: For the 13th layer, the module parameters [256, 256] of [[-1, 6], 1, BiFPN_Add2, [256, 256]] remain unchanged.

[0026] Step 3-4: For the 14th layer, the module parameters [512, False] of [-1, 3, C3, [512, False]] are parsed into [256, 256, 1, False]. 256 is the number of channels, which is the dimension of the output of the previous layer; 512 represents the number of convolutional kernels, and the final number of convolutional kernels is 512 × 0.5 = 256, which is the dimension of the output of the 14th layer.

[0027] Step 3-5: For the 15th layer, the module parameters [256, 1, 1] of [-1, 1, Conv, [256, 1, 1]] are parsed into [256, 128, 1, 1]. 256 is the number of channels, which is the dimension of the output of the previous layer; 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 256 × 0.5 = 128, which is the dimension of the output of the 15th layer.

[0028] Step 3-6: For the 16th layer, the module parameters of [-1, 1, nn.Upsample, [None, 2, nearest]] remain unchanged.

[0029] Step 3-7: For the 17th layer, the module parameter [128, 128] of [[-1, 4], 1, BiFPN_Add2, [128, 128]] remains unchanged;

[0030] Step 3-8: For the 18th layer, the module parameter [256, False] of [-1, 1, C3, [256, False]] is parsed into [128, 128, 1, 1]. 125 is the number of channels, i.e., the dimension of the output of the previous layer; 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 256 × 0.5 = 128, which is the dimension of the output of the 18th layer;

[0031] Step 4: Use the output of the 18th layer as the input of the Detect layer to infer the predicted bounding boxes;

[0032] Object bounding box regression, the calculation formula is as follows:

[0033] b x = 2σ(t x ) - 0.5 + c x

[0034] b y = 2σ(t y ) - 0.5 + c y

[0035] b w = p w × (2σ(t w )) 2

[0036] b h = p h × (2σ(t h )) 2

[0037] where (b x , b y , b w , b h ) represents the center coordinates, width, and height of the predicted bounding box, (c x , c y ) represents the upper-left coordinates of the grid where the center of the predicted bounding box is located, (t x , t y ) represents the offset of the center of the predicted bounding box relative to the upper-left coordinates of the grid, (t w , t h ) represents the scaling ratio of the width and height of the predicted bounding box relative to the width and height of the anchor, (p w , p h ) represents the width and height of the prior box anchor;

[0038] Use the Sigmoid function to process the offset, constrain the center point of the predicted bounding box to the current grid, keep the predicted offset value within the range of (0, 1), and then according to the target box regression calculation formula, the offset of the center point coordinates of the predicted bounding box is kept within the range of (-0.5, 1.5);

[0039] Step 5: Positive and negative sample matching;

[0040] Each grid on each detection layer of YOLOv5 is preset with multiple anchor prior boxes. Screen the anchor prior boxes and divide them into positive and negative samples; adopt a matching strategy based on the width-to-height ratio. The specific steps are as follows:

[0041] Step 5-1: For each manually labeled ground truth, calculate the ratios of width to width (w1 / w2, w2 / w1) and height to height (h1 / h2, h2 / h1) between it and 9 different anchors respectively;

[0042] Step 5-2: Find the maximum value among the ratios of width to width (w1 / w2, w2 / w1) and height to height (h1 / h2, h2 / h1) between the ground truth and the anchor as the ratio between the ground truth and the anchor;

[0043] Step 5-3: If the ratio between the ground truth and the anchor is less than the set ratio threshold, the anchor is responsible for predicting this ground truth, that is, the predicted bounding box regressed by the anchor is called a positive sample, and all the remaining predicted bounding boxes are negative samples;

[0044] Step 6: Loss calculation;

[0045] Step 6-1: Total loss;

[0046] The loss of YOLOv5 includes three aspects: bounding box loss bbox loss 、classification loss cls loss 、confidence loss obj loss ; The expression of the total loss is:

[0047] Loss = box gain ×bbox loss +cls gain ×cls loss +obj gain ×obj loss

[0048] Among them, box gain 、cls gain 、objgain Correspond to different loss weights respectively;

[0049] Step 6-2: Bounding box loss;

[0050] Use CIoU to calculate the bounding box loss, and the penalty term of CIoU is:

[0051]

[0052] Among them, ρ 2 (b, b gt ) represents the Euclidean distance between the center points of the predicted box and the ground truth box, and c represents the diagonal distance of the smallest closed region that can contain both the predicted box and the ground truth box;

[0053] α is a weight parameter, and its expression is:

[0054]

[0055] v is used to measure the consistency of the aspect ratio, and its expression is:

[0056]

[0057] Among them, w gh represents the width of the ground truth box, h gt represents the height of the ground truth box, w represents the width of the predicted box, and h represents the height of the predicted box;

[0058] Therefore, the loss calculation formula of CIoU is:

[0059]

[0060] Step 6-3: Classification loss;

[0061] Use the binary cross-entropy function to calculate the classification loss, and the definition of the binary cross-entropy function is:

[0062]

[0063] Among them, y is the label corresponding to the input sample, the positive sample is 1, the negative sample is 0, and p is the probability that the model predicts that the input sample is a positive sample; Assume The definition of the cross-entropy function is simplified to:

[0064] L = -log p t

[0065] Step 6-4: Confidence loss;

[0066] YOLOv5 uses the binary cross - entropy function to calculate the confidence loss; for the confidence losses obtained on the detection layers of different scales, YOLOv5 assigns different weight coefficients. In the order from the largest to the smallest scale of the detection layer, the corresponding weight coefficients are 4.0, 1.0, and 0.4 respectively.

[0067] Step 7: Adopt the transfer learning training method to perform multiple transfer processing on the training weights and parameters of the training network to enhance the degree of short - term training.

[0068] Preferably, the values of box gain , cls gain , obj gain are 0.05, 0.5, and 1.0 respectively.

[0069] Preferably, the transfer learning method specifically includes the following steps:

[0070] (1) First, train the network on the COCO dataset to obtain the optimal training weights;

[0071] (2) Then, use the obtained optimal weights as the pre - trained weights for PCB board defect detection and train the network.

[0072] The beneficial effects of the present invention are as follows:

[0073] 1. Compared with the original YOLOv5 model, the present invention improves the feature fusion network PANet to BiFPN. The addition of the weighted bidirectional pyramid structure and the fusion mechanism can improve the detection effect of the network on targets of different scales;

[0074] 2. After the Backbone C3 module of the present invention, a SimAM attention detection mechanism is added, which enhances the network's attention to small - target defects and improves the accuracy of PCB board surface defect detection;

[0075] 3. Since the objects to be detected are PCB board surface defects, all of which are small targets, the present invention removes the P4 and P5 structures in the head used to detect medium and large targets, adjusts the number of multi - scale prediction layers, and cuts off redundant network layers to reduce the scale of the model, reduce the number of parameters and the amount of calculation, and improve the detection speed;

[0076] 4. On the basis of the SPPF structure, the present invention introduces the optimized model SPPFCSPC, which can effectively avoid problems such as image distortion caused by image region cropping and scaling operations, and at the same time solves the problem of repeated feature extraction of convolutional neural networks for images, greatly improving the speed of generating candidate boxes and saving costs;

[0077] 5. The present invention uses a transfer learning method to train a model. The deep transfer network has better performance than randomly initializing weights, which not only improves the detection effect but also accelerates the learning and optimization of the network. Description of the Drawings

[0078] Figure 1 is the flowchart of PCB board defect detection according to the present invention.

[0079] Figure 2 is the structural diagram of YOLOv5s_BSSs according to the present invention.

[0080] Figure 3 is the detection result diagram of the embodiment of the present invention.

[0081] Figure 4 is the P-R curve diagram of the embodiment of the present invention.

[0082] Figure 5 is the detection evaluation index diagram of the embodiment of the present invention. Detailed Embodiment

[0083] The present invention will be further described below with reference to the drawings and embodiments.

[0084] The object of the present invention is to solve the problem of low detection and recognition rate of small targets in the existing object detection technology, and provide an object detection method that performs image enhancement on the basis of the original YOLOv5 model, adds a BiFPN structure, a SimAM attention mechanism and an optimized SPPF structure, reduces the P4 and P5 structures and uses a transfer learning method to train, thereby improving the recognition rate of PCB board surface defects; by horizontally comparing the models with different mechanisms added, the detection and recognition rate of the models are improved, and finally the YOLOv5s_BSSs model with the best recognition effect is obtained.

[0085] As Figure 1 shown, a method for detecting PCB board surface defects based on improved YOLOv5 includes the following steps:

[0086] (1) Intercept, flip and map the obtained original data set to obtain a richer data set, convert the labels of the data set and divide the data set;

[0087] (2) Input the data set at the input end. During training, the mosaic data augmentation method is adopted. load_mosaic will randomly select 4 pictures to form a picture, and the output img_size is self.img_size*self.img_size. Re-select 4 images for mosaic augmentation, and perform mixup data augmentation on the image after mosaic augmentation and the data after previous mosaic augmentation;

[0088] (3) During network training, the network outputs prediction boxes based on the initial anchor boxes, and the anchor sizes corresponding to an input image with a resolution of 640×640 are preset. Since it is for the detection of small targets, only the small target detection boxes [10, 13, 16, 30, 33, 23] are set as the initial anchor boxes, which contain three sizes (length and width) of anchor boxes. Then, it is compared with the true groundtruth to calculate the difference between the two, and then updated in reverse to iterate the network parameters; during each training, the optimal anchor box values in different training sets are calculated adaptively;

[0089] (4) The original images are uniformly scaled to a standard size and then fed into the detection network;

[0090] (5) As Figure 2 shown, in the backbone, the original Focus operation is replaced by a convolutional layer, the SimAM attention mechanism is added, and the spatial pyramid pooling structure SPPF is optimized to SPPFCSPC. The specific process is as follows:

[0091] S1 For the 0th layer, the module parameters args[64, 6, 2, 2] of [-1, 1, Conv, [64, 6, 2, 2]] will be parsed into [3, 32, 6, 2, 2]. 64 represents the number of convolutional kernels, and the final number of convolutional kernels should be multiplied by width_multiple, that is, 64×0.5 = 32 (which is the output dimension of the 0th layer); 6 is the convolutional kernel size, 2 is the convolutional kernel stride, and 3 (the input dimension of this layer) because the input image is in RGB 3 channels in color;

[0092] S2 For the 1st layer, the module parameters [128, 3, 2] of [-1, 1, Conv, [128, 3, 2]] will be parsed into [32, 64, 3, 2]. 32 is the number of channels (from the output dimension of the previous layer); 128 represents the number of convolutional kernels, and the final number of convolutional kernels is 128×0.5 = 64 (which is the output dimension of the 1st layer);

[0093] S3 For the 2nd layer, for the 2nd layer, the module parameter

[128] of [-1, 3, C3,

[128] ] will be parsed into [64, 64, 1]. 64 is the number of channels (from the output dimension of the previous layer); 128 represents the number of convolutional kernels, and the final number of convolutional kernels is 128×0.5 = 64 (which is the output dimension of the 2nd layer);

[0094] S4 For the third layer, the module parameters [256, 3, 2] of [-1, 1, Conv, [256, 3, 2]] will be parsed into [64, 128, 3, 2]. 64 is the number of channels (the dimension of the output from the previous layer); 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 256 × 0.5 = 128 (which is the dimension of the output of the third layer);

[0095] S5 For the fourth layer, the module parameters

[256] of [-1, 6, C3,

[256] ] will be parsed into [128, 128, 2]. 128 is the number of channels (the dimension of the output from the previous layer); 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 256 × 0.5 = 128 (which is the dimension of the output of the fourth layer);

[0096] S6 For the fifth layer, the module parameters [512, 3, 2] of [-1, 1, Conv, [512, 3, 2]] will be parsed into [128, 256, 3, 2]. 128 is the number of channels (the dimension of the output from the previous layer); 512 represents the number of convolutional kernels, and the final number of convolutional kernels is 512 × 0.5 = 256 (which is the dimension of the output of the fifth layer);

[0097] S7 For the sixth layer, the module parameters

[512] of [-1, 9, C3,

[512] ] will be parsed into [256, 256, 3]. 256 is the number of channels (the dimension of the output from the previous layer); 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 512 × 0.5 = 256 (which is the dimension of the output of the sixth layer);

[0098] S8 For the seventh layer, the module parameters [1024, 3, 2] of [-1, 1, Conv, [1024, 3, 2]] will be parsed into [256, 512, 3, 2]. 256 is the number of channels (the dimension of the output from the previous layer); 1024 represents the number of convolutional kernels, and the final number of convolutional kernels is 1024 × 0.5 = 512 (which is the dimension of the output of the seventh layer);

[0099] S9 For the eighth layer, the module parameters

[1024] of [-1, 3, C3,

[1024] ] will be parsed into [512, 512, 1]. 512 is the number of channels (the dimension of the output from the previous layer); 1024 represents the number of convolutional kernels, and the final number of convolutional kernels is 1024 × 0.5 = 512 (which is the dimension of the output of the eighth layer);

[0100] S10 For the 9th layer, the module parameter

[1024] of [-1, 1, SimAM,

[1024] ] will be parsed into [512, 512]. Here, 512 is the number of channels (the dimension of the output from the previous layer); 1024 represents the number of convolutional kernels, and the final number of convolutional kernels is 1024 × 0.5 = 512 (which is the dimension of the output of the 9th layer).

[0101] S11 For the 10th layer, the module parameter [1024, 5] of [-1, 1, SPPFCSPC, [1024, 5]] will be parsed into [512, 512, 5]. Here, 512 is the number of channels (the dimension of the output from the previous layer); 1024 represents the number of convolutional kernels, and the final number of convolutional kernels is 1024 × 0.5 = 512 (which is the dimension of the output of the 10th layer).

[0102] (6) In the Neck, replace the original Concat with BiFPN, and remove the P4 and P5 structures originally used for detecting medium and large sizes to reduce the network scale. The specific process is as follows:

[0103] S1 For the 11th layer, the module parameter [512, 1, 1] of [-1, 1, Conv, [512, 1, 1]] will be parsed into [512, 256, 1, 1]. Here, 512 is the number of channels (the dimension of the output from the previous layer); 512 represents the number of convolutional kernels, and the final number of convolutional kernels is 512 × 0.5 = 256 (which is the dimension of the output of the 11th layer).

[0104] S2 For the 12th layer, the module parameters of [-1, 1, nn.Upsample, [None, 2, nearest]] remain unchanged;

[0105] S3 For the 13th layer, the module parameters [256, 256] of [[-1, 6], 1, BiFPN_Add2, [256, 256]] remain unchanged;

[0106] S4 For the 14th layer, the module parameter [512, False] of [-1, 3, C3, [512, False]] will be parsed into [256, 256, 1, False]. Here, 256 is the number of channels (the dimension of the output from the previous layer); 512 represents the number of convolutional kernels, and the final number of convolutional kernels is 512 × 0.5 = 256 (which is the dimension of the output of the 14th layer).

[0107] For the 15th layer, the module parameter [256, 1, 1] of [-1, 1, Conv, [256, 1, 1]] will be parsed into [256, 128, 1, 1]. 256 is the number of channels (the dimension of the output from the previous layer); 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 256 × 0.5 = 128 (which is the dimension of the output of the 15th layer).

[0108] For the 16th layer, the module parameter of [-1, 1, nn.Upsample, [None, 2, nearest]] remains unchanged.

[0109] For the 17th layer, the module parameter [128, 128] of [[-1, 4], 1, BiFPN_Add2, [128, 128]] remains unchanged.

[0110] For the 18th layer, the module parameter [256, False] of [-1, 1, C3, [256, False]] will be parsed into [128, 128, 1, 1]. 125 is the number of channels (the dimension of the output from the previous layer); 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 256 × 0.5 = 128 (which is the dimension of the output of the 18th layer).

[0111] (7) Use the output of the 18th layer as the input to the Detect layer to infer the predicted bounding boxes.

[0112] (8) Target box regression, with the calculation formula as follows:

[0113] b x = 2σ(t x ) - 0.5 + c x

[0114] b y = 2σ(t y ) - 0.5 + c y

[0115] b w = p w × (2σ(t w )) 2

[0116] b h = p h × (2σ(t h )) 2

[0117] where (b x , b y , b w , b h ) represents the center point coordinates, width, and height of the predicted bounding box, (cx , c y ) represents the upper left coordinates of the grid where the center point of the prediction box is located. (t x , t y ) represents the offset of the center point of the prediction box relative to the upper left coordinates of the grid. (t w , t h ) represents the scaling ratio of the width and height of the prediction box relative to the width and height of the anchor. (p w , p h ) represents the width and height of the prior box anchor.

[0118] To constrain the center point of the prediction box to the current grid, the Sigmoid function is used to process the offset, keeping the predicted offset value within the range of (0, 1). In this way, according to the target box regression calculation formula, the offset of the center point coordinates of the prediction box remains within the range of (-0.5, 1.5);

[0119] (9) Positive and negative sample matching. Each grid on each detection layer of YOLOv5 is preset with multiple anchor prior boxes, but not every grid contains a target, nor is every anchor suitable for regressing the current target. Therefore, these anchor prior boxes need to be screened and divided into positive and negative samples. The positive and negative samples in this article refer to the prediction boxes rather than the Ground Truth (manually annotated true boxes). YOLOv5 adopts a matching strategy based on the width-to-height ratio, and its general process is as follows:

[0120] S1 For each Ground Truth (manually annotated true box), calculate the ratio of width to width (w1 / w2, w2 / w1) and the ratio of height to height (h1 / h2, h2 / h1) between it and 9 different anchors respectively;

[0121] S2 Find the maximum value among the ratio of width to width (w1 / w2, w2 / w1) and the ratio of height to height (h1 / h2, h2 / h1) between the Ground Truth and the anchor as the ratio between the Ground Truth and the anchor;

[0122] S3 If the ratio between the Ground Truth and the anchor is less than the set ratio threshold (the default anchor_t = 4.0 in the hyperparameters), then this anchor is responsible for predicting this Ground Truth, that is, the prediction box regressed by this anchor is called a positive sample, and all the remaining prediction boxes are negative samples.

[0123] Through the above method, not only positive and negative samples are screened, but also for some Ground Truth, multiple anchors are matched at a single scale for prediction, generally increasing the number of positive samples to a certain extent. In addition, YOLOv5 also increases the number of positive samples through cross-grid expansion and cross-branch expansion methods, thereby accelerating the convergence speed.

[0124] (10) Loss calculation:

[0125] S1 Total loss: YOLOv5 makes predictions for each grid on the feature map, and compares the predicted information with the real information to guide the next convergence direction of the model. The role of the loss function is to measure the gap between the predicted information and the real information. If the predicted information is closer to the real information, the value of the loss function is smaller. The loss of YOLOv5 mainly includes three aspects: bounding box loss (bbox loss ), classification loss (cls loss ), and confidence loss (obj loss ). The expression of the total loss is:

[0126] Loss=box gain ×bbox loss +cls gain ×cls loss +obj gain ×obj loss

[0127] Among them, box gain , cls gain , and obj gain correspond to different loss weights, and the default values are 0.05, 0.5, and 1.0 respectively.

[0128] S2 Bounding box loss: CIoU is used to calculate the bounding box loss. CIoU is based on DIoU and further considers the aspect ratio of the Bounding Box. CIoU adds an influence factor α v to the penalty term of DIoU. This factor takes into account the aspect ratio of the predicted box and the real box, that is, the penalty term of CIoU is:

[0129]

[0130] Among them, ρ 2 (b,b gt ) represents the Euclidean distance between the centers of the predicted box and the real box, and c represents the diagonal distance of the smallest closed region that can contain both the predicted box and the real box;

[0131] Among them, α is a weight parameter, and its expression is:

[0132]

[0133] v is used to measure the consistency of the aspect ratio, and its expression is:

[0134]

[0135] Therefore, the loss calculation formula of CIoU is:

[0136]

[0137] S3 classification loss: The binary cross-entropy function is used to calculate the classification loss. The definition of the binary cross-entropy function is:

[0138]

[0139] where y is the label corresponding to the input sample (1 for positive samples and 0 for negative samples), and p is the probability that the model predicts the input sample as a positive sample. Assume The definition of the cross-entropy function can be simplified to:

[0140] L = -log p t

[0141] S4 confidence loss: The confidence of each prediction box represents the reliability of this prediction box. The larger the value, the more reliable the prediction box is, and it also means that it is closer to the true box. According to the CIoU between the prediction box corresponding to the grid and the true box as the confidence label of this prediction box, like calculating the classification loss, YOLOv5 defaults to using the binary cross-entropy function to calculate the confidence loss. In addition, for the confidence losses obtained on detection layers of different scales, YOLOv5 assigns different weight coefficients. In the order from large to small of the detection layer scales, the corresponding default weight coefficients are 4.0, 1.0, 0.4, that is, the loss weight coefficient on the large-scale feature map used to detect small targets is larger, so that the network focuses more on small targets during training.

[0142] (11) At the same time, in order to further solve the problem of small target defect detection, the training search method is optimized, and the method of transfer learning is adopted to perform multiple transfer processing on the training weights and parameters of the training network, making the degree of short-term training more sufficient.

[0143] Improve the PCB board surface defect detection model of YOLOv5, improve the feature fusion network to the BiFPN structure, and add the SimAM attention mechanism and the method of subtracting the P4 and P5 structures. The specific steps are as follows:

[0144] (1) Add a weighted bidirectional pyramid structure (BiFPN) to improve the detection effect:

[0145] S1. Build Concat_BiFPN in common.py;

[0146] S2. Register the BiFPN module in yolo.py;

[0147] S3. Modify the yaml file, and the head (feature fusion network) needs to be modified;

[0148] S4. Just change the yaml file in train.py to the one in this article.

[0149] (2) Add the SimAM attention mechanism, which can make the network pay more attention to the objects to be detected:

[0150] S1. Build the SimAM module in common.py;

[0151] S2. Register the SimAM module in yolo.py;

[0152] S3. Modify the yaml file. In the backbone, add the ECA module (the 9th layer) after the C3 module.

[0153] (3) Subtract the P4 and P5 modules in the head for detecting medium and large objects, thereby reducing the scale of the model, which can reduce the number of parameters and the amount of computation.

[0154] The specific steps of the transfer learning training method are as follows:

[0155] (1) First, train the network on the COCO dataset to obtain the optimal training weights;

[0156] (2) Then, use the obtained optimal weights as the pre-training weights for PCB defect detection and train the network. Specific embodiments:

[0158] 1. The specific steps for preparing the dataset of the defect detection method are as follows:

[0159] (1) Dataset acquisition and augmentation. The data comes from the public dataset provided by the Intelligent Robot Open Laboratory of Peking University, which contains six common defects. The dataset is intercepted, flipped, and mapped to obtain more datasets;

[0160] (2) Dataset format conversion. Convert the dataset labels from xml format to txt format through the dataset format conversion algorithm;

[0161] (3) Dataset division. Divide the dataset into training, validation, and test sets through the dataset division algorithm, and separate the corresponding training, validation, and test set labels.

[0162] 2. The network structure of the defect detection method is as follows:

[0163] (1) Input end: Data augmentation, adaptive anchor box calculation, adaptive image scaling;

[0164] S1 Mosaic data augmentation: Since the AP of small targets is generally much lower than that of medium and large targets, and the distribution of small targets is not uniform, 4 pictures of data are used and stitched in a way of random scaling, random cropping, and random distribution. While enriching the dataset, random scaling adds many small targets, making the network more robust, and can reduce the number of GPUs used. A single GPU can achieve good results;

[0165] S2 Adaptive anchor box calculation: In the YOLO algorithm, for different datasets, there are initial anchor boxes with set length and width. During network training, the network outputs prediction boxes based on the initial anchor boxes, and then compares them with the ground truth boxes to calculate the difference between the two, and then updates backward to iterate the network parameters. In Yolov5, this function is embedded in the code, and the optimal anchor box values in different training sets are calculated adaptively during each training;

[0166] S3 Adaptive image scaling: In common object detection algorithms, the lengths and widths of different images are different. Therefore, the common method is to uniformly scale the original image to a standard size and then send it into the detection network. The code of Yolov5 has improved this, which is also a good trick for the fast inference speed of Yolov5. In actual project use, the aspect ratios of many images are different. Therefore, after scaling and padding, the sizes of the black edges at both ends are different. If the padding is too much, there will be information redundancy, affecting the inference speed. Therefore, in the letterbox function of datasets.py in the Yolov5 code, it is modified to add the least amount of black edges to the original image adaptively. The black edges at both ends of the image height become smaller, and during inference, the calculation amount will also be reduced, that is, the object detection speed will be improved. The specific calculation steps in the algorithm are: calculate the scaling ratio, calculate the scaled size, and calculate the black edge padding value.

[0167] (2) Backbone: Conv structure, C3 structure, SimAM, SPPFCSPC;

[0168] S1 Replace the original Focus operation with a convolutional layer. The Conv module encapsulates three functions: convolution (Conv2d), Batch Normalization, and the Swish activation function, which is convenient for exporting the model. Replace Focus with an equivalent Conv (kernel = 6, stride = 2, padding = 2), and at the same time use autopad(k,p) to achieve the padding effect, reducing the number of parameters (params) and the amount of computation (FLOPs), reducing the consumption of CUDA video memory, thereby improving the speed of forward and backward propagation;

[0169] S2 Bottleneck1: The Bottleneck module draws on the residual structure of ResNet. One path first performs a 1×1 convolution to halve the number of channels of the feature map, thereby reducing the amount of computation, then extracts features through a 3×3 convolution, and doubles the number of channels. The number of input and output channels remains unchanged. The other path performs a residual connection through a shortcut and adds it to the output feature map of the first path to achieve feature fusion. The Bottleneck in Backbone1 defaults to having shortcut as True;

[0170] S3 C3 module: The C3 module replaces the early BottleneckCSP module. The structures and functions of these two are basically the same, both are CSP architectures, but they differ in the choice of correction units. The C3 module contains 3 standard convolutional layers and multiple Bottleneck modules. The difference between the C3 module and the BottleneckCSP module is that the Conv module after the output of the Bottleneck module is removed. However, although the Backbones of YOLOv4 and YOLOv5 draw on CSPNet, they do not actually divide the input feature map into two parts in the channel dimension as described in the original CSPNet paper. Instead, they directly transform the input feature map with two 1×1 convolutions;

[0171] S4 Introduce the SimAM attention mechanism: Existing studies have shown that BAM and CBAM respectively combine spatial attention and channel attention in parallel or in series. However, the two types of attention in the human brain often work together. Therefore, an attention module with unified weights is adopted. To better implement attention, it is necessary to evaluate the importance of each neuron. In neuroscience, neurons rich in information usually exhibit different firing patterns from surrounding neurons. Moreover, activating a neuron usually inhibits surrounding neurons, that is, spatial domain inhibition. In other words, neurons with spatial domain inhibition effects should be given higher importance. The simplest way to find important neurons is to measure the linear separability between neurons. Therefore, the following energy function is defined:

[0172]

[0173] Among them, Minimizing the above formula is equivalent to training the linear separability between neuron t and other neurons within the same channel. For simplicity, binary labels are adopted and a regularization term is added. The final energy function is defined as follows:

[0174]

[0175] Theoretically, each channel has M = H × W energy functions, but the above formula has the following analytical solution:

[0176]

[0177]

[0178] Among them,

[0179] Since all neurons on each channel follow the same distribution, the mean and variance can be calculated for the input features in the H and W dimensions first. The minimum energy is as follows:

[0180]

[0181] The above formula means that: the lower the energy, the greater the difference between neuron t and its surrounding neurons, and the higher the importance. Therefore, the importance of a neuron can be obtained through 1 / obtained.

[0182] After deriving the energy function and mining the importance of neurons, according to the definition of the attention mechanism, the features need to be enhanced. The whole process can be expressed as:

[0183]

[0184] S5 SPPFCSPC: On the basis of the SPPF structure, an optimized model SPPFCSPC is introduced. The SPP is the Spatial Pyramid Pooling structure throughout, which can effectively avoid problems such as image distortion caused by image region cropping and scaling operations. At the same time, it solves the problem of the convolutional neural network's extraction of duplicate features related to the graph, greatly improving the speed of generating candidate boxes and saving costs. The SPPF structure serially passes the input through multiple MaxPool layers of size 5x5. Simply put, it uses 2 convolutional operations of 5x5 to replace a convolutional operation of 9x9, and 3 convolutional operations of 5x5 to replace a convolutional operation of 13x13. The receptive field is the same, but the calculation time is much faster. Drawing on the idea of SPPCSPC, the SPPF is optimized to obtain the SPPFCSPC, achieving a speed improvement while maintaining the receptive field unchanged.

[0185] (3)Neck: Bottleneck2, BiFPN;

[0186] S1 Bottleneck2: In Bottleneck of Backbone1, the shortcut is default set to False;

[0187] S2 BiFPN structure: Replace part of Concat with the BiFPN_Add weighted bidirectional pyramid structure, which allows simple and fast multi-scale feature fusion and improves the detection effect at different scales;

[0188] (4)Head: The Head expands the number of channels of the feature maps of different scales obtained in the Neck through 1×1 convolutions respectively. The number of expanded feature channels is (the number of classes + 5) × the number of anchors on each detection layer. Here, 5 respectively correspond to the abscissa, ordinate, width, height and confidence of the center point of the prediction box. The confidence here represents the credibility of the prediction box, and the value range is (0,1). The larger the value, the more likely it is that there is a target in the prediction box. The Head originally contained 3 detection layers, corresponding to 3 different sizes (small, medium, large) of feature maps obtained in the Neck. Since the PCB board surface defect datasets used are all relatively small targets, the P4 and P5 structures for detecting medium and large targets in the Head are removed here to reduce the number of model modules and the computational load. The model divides the grid on the feature map according to the small size of the feature map, and presets anchors with the aspect ratio corresponding to the width and height of the small-size feature map for each grid on the feature map to predict and regress the target. The above channel dimension can be understood as saving all the position information and classification information of the anchor prior boxes on the channel dimension of the feature map.

[0189] 3. The specific steps of the transfer learning training method for the defect detection method are as follows:

[0190] (1) YOLOv5s model:

[0191] S1 trains the original YOLOv5s model, using the weights obtained from training on the COCO dataset as the pre-training weights, and obtains the training results;

[0192] S2 trains the YOLOv5s model by reducing the P4 and P5 structures in the Head for detecting medium and large targets, using the weights obtained from training on the COCO dataset as the pre-training weights, and obtains the training results.

[0193] (2) Adding the BiFPN structure to the YOLOv5s model:

[0194] S1 trains the YOLOv5s model with the BiFPN structure added, using the weights obtained from training on the COCO dataset as the pre-training weights, and obtains the training results;

[0195] S2 trains the YOLOv5s model with the BiFPN structure added and the P4 and P5 structures reduced, using the weights obtained from training on the COCO dataset as the pre-training weights, and obtains the training results.

[0196] (3) Adding the BiFPN structure and the SimAM attention mechanism to the YOLOv5s model:

[0197] S1 trains the YOLOv5s model with the BiFPN structure and the SimAM attention mechanism added, using the weights obtained from training on the COCO dataset as the pre-training weights, and obtains the training results;

[0198] S2 trains the YOLOv5s model with the BiFPN structure and the SimAM attention mechanism added and the P4 and P5 structures reduced, using the weights obtained from training on the COCO dataset as the pre-training weights, and obtains the training results.

[0199] (4) Adding the BiFPN structure, the SimAM attention mechanism and adding the SPPF to optimize the model SPPFCSPC to the YOLOv5s model;

[0200] S1 trains the YOLOv5s model with the BiFPN structure, the SimAM attention mechanism and the SPPF added to optimize the model SPPFCSPC, using the weights obtained from training on the COCO dataset as the pre-training weights, and obtains the training results;

[0201] S2 trains the YOLOv5s model with the BiFPN structure, the SimAM attention mechanism and the SPPF added to optimize the model SPPFCSPC and the P4 and P5 structures reduced, using the weights obtained from training on the COCO dataset as the pre-training weights, and obtains the training results.

[0202] (5) Horizontally compare the training results of the above models. For each type of defect, calculate the precision P and recall rate R, draw the P-R curve, and calculate the area under the P-R curve to obtain the average precision AP. mAP refers to the average precision of all defects.

[0203] The test results are as Figures 3 to 5 shown.

[0204] When implementing on a specific dataset:

[0205] S1 Intercept, flip, and map the original dataset to obtain a total of 10,668 datasets. Divide all the datasets, with 8,640 in the training set, 961 in the validation set, and 1,067 in the test set. Then convert the dataset labels to the txt format;

[0206] S2 First, train the COCO dataset in the system to obtain the best training weights as the pre-training weights for PCB defect detection;

[0207] S3 Perform transfer learning training on the PCB defect dataset using the pre-training weights;

[0208] S4 Train several different models respectively to obtain the transfer learning training results and evaluation metrics:

[0209] For each type of defect, calculate the precision P and recall rate R, draw the P-R curve, and calculate the area under the P-R curve to obtain the average precision AP. mAP refers to the average precision of all defects. The specific calculation formulas are as follows:

[0210]

[0211]

[0212]

[0213]

[0214] In the formula: TP represents the number of correctly judged defects among the detected defects; FP represents the number of misjudged defects; FN represents the number of undetected defects;

[0215] S5 After training, obtain the best training weights, use these weights to detect the test set, obtain the defect type labels and confidence levels of the test set, and horizontally compare the models.

Claims

1. A method for detecting surface defects of PCB boards based on improved YOLOv5, characterized in that, It includes the following steps: Step 1: Obtain a dataset and perform data preprocessing; Step 1-1: Intercept, flip, and map the original dataset of PCB board surface defects obtained, convert the labels of the dataset, and divide the dataset; divide the dataset into training, validation, and test sets; Step 1-2: Adopt the mosaic data augmentation method, use the data of 4 pictures, and splice them through random scaling, random cropping, and random distribution methods to combine 4 pictures into one picture; Step 1-3: Uniformly scale the images in the dataset to the same size, and then send them into the detection network YOLOv5s_BSSs; The specific description of the detection network YOLOv5s_BSSs is as follows; Step 2: In the YOLOv5 backbone, replace the original Focus operation with a convolutional layer, add the SimAM attention mechanism, and optimize the spatial pyramid pooling structure SPPF to SPPFCSPC. The specific process is as follows: Step 2-1: For the 0th layer, the module parameters args[64,6,2,2] of [-1,1,Conv,[64,6,2,2]] are parsed into [3,32,6,2,2]. 64 represents the number of convolutional kernels, and the final number of convolutional kernels is multiplied by width_multiple, which is 64×0.5 = 32, that is, the output dimension of the 0th layer; 6 is the convolutional kernel size, 2 is the convolutional kernel stride, and 3 is the input dimension of this layer because the input image is in RGB 3 channels of color; Step 2-2: For the 1st layer, the module parameters [128,3,2] of [-1,1,Conv,[128,3,2]] are parsed into [32,64,3,2]. 32 is the number of channels, that is, the output dimension of the previous layer; 128 represents the number of convolutional kernels, and the final number of convolutional kernels is 128×0.5 = 64, that is, the output dimension of the 1st layer; Step 2-3: For the 2nd layer, the module parameters [128] of [-1,3,C3,[128]] are parsed into [64,64,1]. 64 is the number of channels, that is, the output dimension of the previous layer; 128 represents the number of convolutional kernels, and the final number of convolutional kernels is 128×0.5 = 64, that is, the output dimension of the 2nd layer; Step 2-4: For the 3rd layer, the module parameters [256,3,2] of [-1,1,Conv,[256,3,2]] are parsed into [64,128,3,2]. 64 is the number of channels, that is, the output dimension of the previous layer; 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 256×0.5 = 128, that is, the output dimension of the 3rd layer; Step 2-5: For the 4th layer, the module parameters [256] of [-1,6,C3,[256]] will be parsed into [128,128,2]. 128 is the number of channels, that is, the output dimension of the previous layer; 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 256×0.5 = 128, that is, the output dimension of the 4th layer; Step 2-6: For the 5th layer, the module parameters [512, 3, 2] of [-1, 1, Conv, [512, 3, 2]] are parsed into [128, 256, 3, 2]. 128 is the number of channels, i.e., the dimension of the output of the previous layer; 512 represents the number of convolutional kernels, and the final number of convolutional kernels is 512×0.5 = 256, which is the dimension of the output of the 5th layer. Step 2-7: For the 6th layer, the module parameters [512] of [-1, 9, C3, [512]] are parsed into [256, 256, 3]. 256 is the number of channels, i.e., the dimension of the output of the previous layer; 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 512×0.5 = 256, which is the dimension of the output of the 6th layer. Step 2-8: For the 7th layer, the module parameters [1024, 3, 2] of [-1, 1, Conv, [1024, 3, 2]] are parsed into [256, 512, 3, 2]. 256 is the number of channels, i.e., the dimension of the output of the previous layer; 1024 represents the number of convolutional kernels, and the final number of convolutional kernels is 1024×0.5 = 512, which is the dimension of the output of the 7th layer. Step 2-9: For the 8th layer, the module parameters [1024] of [-1, 3, C3, [1024]] are parsed into [512, 512, 1]. 512 is the number of channels, i.e., the dimension of the output of the previous layer; 1024 represents the number of convolutional kernels, and the final number of convolutional kernels is 1024×0.5 = 512, which is the dimension of the output of the 8th layer. Step 2-10: For the 9th layer, the module parameters [1024] of [-1, 1, SimAM, [1024]] are parsed into [512, 512]. 512 is the number of channels, i.e., the dimension of the output of the previous layer; 1024 represents the number of convolutional kernels, and the final number of convolutional kernels is 1024×0.5 = 512, which is the dimension of the output of the 9th layer. Step 2-11: For the 10th layer, the module parameters [1024, 5] of [-1, 1, SPPFCSPC, [1024, 5]] are parsed into [512, 512, 5]. 512 is the number of channels, i.e., the dimension of the output of the previous layer; 1024 represents the number of convolutional kernels, and the final number of convolutional kernels is 1024×0.5 = 512, which is the dimension of the output of the 10th layer. Step 3: In the Neck of YOLOv5, replace the original Concat with BiFPN and remove the P4 and P5 structures to reduce the network scale. The specific process is as follows: Step 3-1: For the 11th layer, the module parameters [512, 1, 1] of [-1, 1, Conv, [512, 1, 1]] are parsed into [512, 256, 1, 1]. 512 is the number of channels, i.e., the dimension of the output of the previous layer; 512 represents the number of convolutional kernels, and the final number of convolutional kernels is 512×0.5 = 256, which is the dimension of the output of the 11th layer. Step 3-2: For the 12th layer, the module parameters of [-1, 1, nn.Upsample, [None, 2, nearest]] remain unchanged. Step 3-3: For the 13th layer, the module parameter [256, 256] of [[-1, 6], 1, BiFPN_Add2, [256, 256]] remains unchanged; Step 3-4: For the 14th layer, the module parameter [512, False] of [-1, 3, C3, [512, False]] is parsed into [256, 256, 1, False]. Here, 256 is the number of channels, which is the dimension of the output of the previous layer; 512 represents the number of convolutional kernels, and the final number of convolutional kernels is 512 × 0.5 = 256, which is the dimension of the output of the 14th layer; Step 3-5: For the 15th layer, the module parameter [256, 1, 1] of [-1, 1, Conv, [256, 1, 1]] is parsed into [256, 128, 1, 1]. Here, 256 is the number of channels, which is the dimension of the output of the previous layer; 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 256 × 0.5 = 128, which is the dimension of the output of the 15th layer; Step 3-6: For the 16th layer, the module parameter of [-1, 1, nn.Upsample, [None, 2, nearest]] remains unchanged; Step 3-7: For the 17th layer, the module parameter [128, 128] of [[-1, 4], 1, BiFPN_Add2, [128, 128]] remains unchanged; Step 3-8: For the 18th layer, the module parameter [256, False] of [-1, 1, C3, [256, False]] is parsed into [128, 128, 1, 1]. Here, 125 is the number of channels, which is the dimension of the output of the previous layer; 256 represents the number of convolutional kernels, and the final number of convolutional kernels is 256 × 0.5 = 128, which is the dimension of the output of the 18th layer; Step 4: Use the output of the 18th layer as the input of the Detect layer to infer the predicted bounding boxes; Object box regression, the calculation formula is as follows: b x = 2σ(t x ) - 0.5 + c x b y = 2σ(t y ) - 0.5 + c y b w = p w ×(2σ(t w )) 2 b h = p h ×(2σ(t h )) 2 Among them, (b x , b y , b w , b h ) represents the center point coordinates, width, and height of the prediction box. (c x , c y ) represents the upper left coordinates of the grid where the center point of the prediction box is located. (t x , t y ) represents the offset of the center point of the prediction box relative to the upper left coordinates of the grid. (t w , t h ) represents the scaling ratio of the width and height of the prediction box relative to the width and height of the anchor. (p w , p h ) represents the width and height of the prior box anchor; Use the Sigmoid function to process the offset, constrain the center point of the predicted bounding box to the current grid, keep the predicted offset value within the range of (0, 1), and then according to the object box regression calculation formula, the offset of the center point coordinates of the predicted bounding box is kept within the range of (-0.5, 1.5); Step 5: Positive and negative sample matching; Each grid on each detection layer of YOLOv5 is preset with multiple anchor prior boxes. Screen the anchor prior boxes and divide them into positive and negative samples; adopt a matching strategy based on the width-to-height ratio. The specific steps are as follows: Step 5-1: For each manually annotated ground truth box, calculate the width-to-width ratio (w1 / w2, w2 / w1) and height-to-height ratio (h1 / h2, h2 / h1) between it and 9 different anchors respectively; Step 5-2: Find the maximum value among the width-to-width ratio (w1 / w2, w2 / w1) and height-to-height ratio (h1 / h2, h2 / h1) between the ground truth and the anchor as the ratio between the ground truth and the anchor; Step 5-3: If the ratio of Ground Truth to the anchor is less than the set ratio threshold, then the anchor is responsible for predicting this Ground Truth, that is, the prediction box obtained by the regression of the anchor is called a positive sample, and all the remaining prediction boxes are negative samples; Step 6: Loss calculation; Step 6-1: Total loss; The losses of YOLOv5 include three aspects: the bounding box loss bbox loss , the classification loss cls loss , and the confidence loss obj loss ; the expression for the total loss is: Loss=box gain ×bbox loss +cls gain ×cls loss +obj gain ×obj loss Among them, box gain , cls gain , obj gain correspond to different loss weights respectively; Step 6-2: Bounding box loss; Use CIoU to calculate the bounding box loss, and the penalty term of CIoU is: Among them, ρ 2 (b, b gt ) represents the Euclidean distance between the center points of the predicted box and the ground truth box, and c represents the diagonal distance of the smallest closed region that can simultaneously contain the predicted box and the ground truth box; α is a weight parameter, and its expression is: v is used to measure the consistency of the aspect ratio, and its expression is: Among them, w gt represents the width of the ground truth box, and h gt represents the height of the ground truth box, w represents the width of the predicted box, and h represents the height of the predicted box; So the loss calculation formula of CIoU is: Step 6-3: Classification loss; Use the binary cross-entropy function to calculate the classification loss, and the definition of the binary cross-entropy function is: where y is the label corresponding to the input sample, with the positive sample being 1 and the negative sample being 0, and p is the probability that the model predicts the input sample as a positive sample; assume The definition of the cross-entropy function is simplified to: L = -log p t Step 6-4: Confidence loss; YOLOv5 uses the binary cross-entropy function to calculate the confidence loss; for the confidence losses obtained on the detection layers of different scales, YOLOv5 assigns different weight coefficients. In the order from the largest to the smallest scale of the detection layer, the corresponding weight coefficients are 4.0, 1.0, and 0.4 respectively; Step 7: Adopt the transfer learning training method to perform multiple transfer processing on the training weights and parameters of the training network to enhance the degree of short-term training.

2. The PCB board surface defect detection method based on improved YOLOv5 according to claim 1, characterized in that, The said box gain 、cls gain 、obj gain are 0.05, 0.5, and 1.0 respectively.

3. A method for detecting surface defects of a PCB board based on improved YOLOv5 according to claim 1, characterized in that, The specific steps of the transfer learning method are as follows: (1) First, train the network on the COCO dataset to obtain the optimal training weights; (2) Then use the obtained optimal weights as the pre-training weights for PCB defect detection and train the network.

Images