PCB defect detection method and system based on YOLOv8 network

Abstract

The invention relates to a PCB defect detection method and system based on a YOLOv8 network, and belongs to the technical field of target detection. Collecting and preprocessing a PCB sample image data set for training a PCB defect detection model; inputting a to-be-detected image into the trained defect detection model to obtain a detection result; the to-be-detected image is generated by collecting and processing image data of the to-be-detected PCB; wherein the defect detection model comprises a trunk network, a neck network and a head network, and the trunk network extracts features through a mobile reverse bottleneck convolution module and a C2f module to generate an intermediate feature map; and the neck network performs splicing fusion on the middle feature map to generate a spliced feature map, the head network performs re-weighting on the spliced feature map through a convolution kernel attention fusion module to generate a third feature map, and then the detection module outputs a detection result. The method not only improves the precision of the defect detection model in detecting the features of the tiny target, but also improves the detection speed and generalization ability, and reduces the calculation cost and complexity at the same time.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a PCB defect detection method and system based on a YOLOv8 network. Background Art

[0002] With the widespread application of target detection technology and the intensification of market competition, the recognition and detection algorithm based on the YOLOv8 algorithm is gradually becoming an important tool for enterprises to enhance their competitiveness. The continuous optimization and iteration of the algorithm have continuously improved its commercial value. With its high efficiency and accuracy, the YOLOv8 recognition and detection algorithm has shown wide application potential in multiple industries. For example, in intelligent monitoring systems, it can be used for face recognition, abnormal behavior detection, emotion recognition, etc.; in industrial detection scenarios, it can be used for rapid identification of product defects and quality control, etc.

[0003] The YOLOv8 algorithm uses a single forward propagation to predict all objects in the image, allowing it to run at extremely high speeds while maintaining high accuracy. This feature gives YOLOv8 a clear advantage in image detection speed and accuracy over traditional target detection algorithms. However, when dealing with small target detection tasks, especially when inspecting subtle defects on PCB boards, these defects often occupy small pixels on the image, resulting in limited feature information that can be extracted. This makes the traditional YOLOv8-based detection method face the problems of slow detection speed, low attention to defect feature information, and low detection accuracy. Summary of the invention

[0004] To this end, the technical problem to be solved by the present invention is to overcome the problems of slow detection speed, low attention to defect feature information and low detection accuracy when detecting tiny target features in the prior art, and to provide a PCB defect detection method and system based on a YOLOv8 network. The PCB defect detection model has high detection accuracy, fast detection speed and high generalization ability.

[0005] In the first aspect, in order to solve the above technical problems, the present invention provides a PCB defect detection method based on a YOLOv8 network, comprising:

[0006] Collecting a PCB sample image data set, performing a preprocessing operation on the sample image data set, and generating a preprocessing data set;

[0007] Inputting the preprocessed data set into a PCB defect detection model for training to obtain an optimized PCB defect detection model;

[0008] Inputting the image to be detected into the optimized PCB defect detection model to obtain a defect detection result of the PCB to be detected; the image to be detected is generated by collecting and processing image data of the PCB to be detected;

[0009] The PCB defect detection model includes a backbone network, a neck network and a head network. The backbone network includes an input image receiving layer and a feature extraction layer. The feature extraction layer includes a mobile reverse bottleneck convolution module and a C2f module. The mobile reverse bottleneck convolution module includes a dimension-raising convolution layer, a depth convolution layer, an SE module and a dimension-reducing convolution layer. The input image receiving layer receives the image to be detected and generates a first feature map after a convolution operation. The dimension-raising convolution layer receives the first feature map, performs dimension-raising processing, and then inputs it into the depth convolution layer for feature extraction, and then inputs it into the SE module for feature recalibration. Finally, it is input into the dimension-reducing convolution layer for dimension reduction to generate a second feature map. The C2f module receives the second feature map and generates an intermediate feature map.

[0010] The neck network receives the intermediate feature map and performs a splicing and fusion operation to generate a spliced ​​feature map. The head network includes a convolution kernel attention fusion module and a detection module. The convolution kernel attention fusion module receives the spliced ​​feature map and reweights it to generate a third feature map containing local features and global information. The detection module receives the third feature map and processes and outputs a fourth feature map.

[0011] In one embodiment of the present invention, the method of inputting the preprocessed data set into the PCB defect detection model for training is:

[0012] The network parameters of the PCB defect detection model are adjusted by the composite coefficient of EfficientNet-B7; the learning rate of the PCB defect detection model is adjusted according to the network structure of EfficientNet-B7;

[0013] The PCB defect detection model is trained using the preprocessed data set to obtain a training result, a value of an optimization loss function is calculated according to the training result, and the optimization loss function is iteratively trained using a back propagation algorithm, wherein the optimization loss function includes a function that penalizes misclassification of a small number of categories.

[0014] In one embodiment of the present invention, the penalty term formula of the optimization loss function is:

[0015]

[0016] Among them, IoU is the intersection over union ratio between the predicted bounding box and the true bounding box, ρ 2 (A, B) is the square of the Euclidean distance between the center point of the predicted box and the real box, c 2 is the square of the diagonal length of the minimum circumscribed rectangle of the two boxes, w gt and h gtare the width and height of the real bounding box, w and h are the width and height of the predicted bounding box, C w and C h are the side lengths of the minimum circumscribed rectangles of the two bounding boxes in the width and height directions, respectively. λ1, λ2 and λ3 are weight coefficients for adjusting the contribution of different loss terms in the overall optimization loss function.

[0017] In one embodiment of the present invention, the optimization loss function includes a CIoU Loss loss function, and the function for penalizing misclassification of a few categories is a Focal Loss loss function.

[0018] In one embodiment of the present invention, the CIoU Loss function is:

[0019]

[0020] Among them, v is the correction factor used to further adjust the loss function, α is the balance parameter used to adjust the weight of the v term in the loss function, and the calculation formula of α is:

[0021]

[0022] The calculation formula for v is:

[0023]

[0024] In one embodiment of the present invention, the Focal Loss function is:

[0025] L f-l =-α t (1-p t ) γ ×log(p t );

[0026] Among them, α t is the balance coefficient used to adjust the importance of positive and negative samples, p t is the predicted probability of the sample by the PCB defect detection model. When the true label of the sample is 1, p t is the probability that the model predicts the positive category, and γ is the modulation coefficient used to adjust the rate at which the weight of easy-to-classify samples decreases.

[0027] In one embodiment of the present invention, there are multiple feature extraction layers, at least one of which includes an SPPF module.

[0028] In one embodiment of the present invention, the input image receiving layer includes a first convolution module, wherein the first convolution module includes a two-dimensional convolution layer, a two-dimensional batch normalization layer and an activation function, the two-dimensional convolution layer performs a convolution operation on the input image, the two-dimensional batch normalization layer normalizes the output result of the two-dimensional convolution layer, and the activation function introduces a nonlinear factor to the output result of the two-dimensional batch normalization layer.

[0029] In one embodiment of the present invention, the convolution kernel attention fusion module receives the concatenated feature map and re-weights it to generate a third feature map containing local features and global information. The method is:

[0030] The local branch and the global branch of the convolution kernel attention fusion module independently perform convolution and weighting operations on the spliced ​​feature map to generate their own local feature maps and global information feature maps;

[0031] Inputting the local feature map and the global information feature map into the attention mechanism module of the convolution kernel attention fusion module to perform a weight dynamic adjustment operation to generate a weight map;

[0032] The local feature map and the global information feature map are weightedly fused according to the weight map to generate the third feature map.

[0033] In the second aspect, in order to solve the above technical problems, the present invention provides a PCB defect detection system based on a YOLOv8 network, comprising:

[0034] The data set acquisition and preprocessing module is used to acquire the PCB sample image data set, and perform preprocessing operations on the sample image data set to generate a preprocessed data set;

[0035] A PCB defect detection model optimization module is used to input the preprocessed data set into the PCB defect detection model for training to obtain an optimized PCB defect detection model;

[0036] A defect detection module, used for inputting the image to be detected into the optimized PCB defect detection model to obtain a defect detection result of the PCB to be detected; the image to be detected is generated by collecting and processing image data of the PCB to be detected;

[0037] The PCB defect detection model includes a backbone network, a neck network and a head network. The backbone network includes an input image receiving layer and a feature extraction layer. The feature extraction layer includes a mobile reverse bottleneck convolution module and a C2f module. The mobile reverse bottleneck convolution module includes a dimension-raising convolution layer, a depth convolution layer, an SE module and a dimension-reducing convolution layer. The input image receiving layer receives the image to be detected and generates a first feature map after a convolution operation. The dimension-raising convolution layer receives the first feature map, performs dimension-raising processing, and then inputs it into the depth convolution layer for feature extraction, and then inputs it into the SE module for feature recalibration. Finally, it is input into the dimension-reducing convolution layer for dimension reduction to generate a second feature map. The C2f module receives the second feature map and generates an intermediate feature map.

[0038] The neck network receives the intermediate feature map and performs a splicing and fusion operation to generate a spliced ​​feature map. The head network includes a convolution kernel attention fusion module and a detection module. The convolution kernel attention fusion module receives the spliced ​​feature map and reweights it to generate a third feature map containing local features and global information. The detection module receives the third feature map and processes and outputs a fourth feature map.

[0039] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0040] The present invention discloses a PCB defect detection method and system based on a YOLOv8 network, which integrates the neural network architecture EfficientNet-B7, and can significantly reduce the computational cost and model complexity while ensuring high detection accuracy. The mobile reverse bottleneck convolution (MB Conv) module of EfficientNet-B7 achieves precise focusing on the key areas of the image through its specific design, such as the deep convolution layer and the SE module, and significantly enhances the ability to extract small target features. In addition, the optimized loss function of the PCB defect detection model has a fine consideration of the center point distance and size difference, which can enable the detection model to achieve higher-precision bounding box positioning in PCB defect detection, and help to more accurately identify various small defects; and the optimized loss function has faster convergence than the traditional loss function, which can further improve the training efficiency of the model; at the same time, the optimized loss function also increases the minimum closed box size and other factors to further enhance the generalization ability of the network model for defect detection of different sizes and positions on the PCB, so that the model can better adapt to various complex PCB defect detection scenarios. Placing the convolution kernel attention fusion module before the detection module of the head network can enhance the feature extraction ability of the detection module. The convolution kernel attention fusion module re-weights the features through the self-attention mechanism, so that important features get more attention, while the attention of unimportant features is suppressed, which helps the defect detection model to identify target defects more easily; at the same time, the self-attention mechanism in the convolution kernel attention fusion module can be combined with the multi-scale feature extraction of the convolution operation in YOLOv8, further improving the detection module's adaptability to feature map detection of different scales. The present invention not only improves the accuracy of the PCB defect detection model in detecting tiny target features, but also improves the convergence speed and generalization ability of the model, while reducing the computational cost and complexity of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0042] Figure 1 It is a flow chart of a PCB defect detection method based on YOLOv8 network in a preferred embodiment of the present invention;

[0043] Figure 2 This is a network structure diagram of the improved YOLOv8 in the preferred embodiment of the present invention;

[0044] Figure 3 It is a structural diagram of a mobile reverse bottleneck convolution module in a preferred embodiment of the present invention;

[0045] Figure 4This is a structural diagram of the convolution kernel attention fusion module in the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0046] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0047] Embodiment 1

[0048] Reference Figure 1 As shown, a PCB defect detection method based on a YOLOv8 network provided by an embodiment of the present invention includes but is not limited to the following steps:

[0049] Collecting a PCB sample image data set, performing a preprocessing operation on the sample image data set, and generating a preprocessing data set;

[0050] Input the preprocessed data set into the PCB defect detection model for training to obtain an optimized PCB defect detection model;

[0051] The image to be detected is input into the optimized PCB defect detection model to obtain the defect detection result of the PCB to be detected; the image to be detected is generated by collecting and processing the image data of the PCB to be detected.

[0052] Among them, the PCB defect detection model includes a backbone network, a neck network and a head network. The backbone network includes an input image receiving layer and a feature extraction layer. The feature extraction layer includes a moving reverse bottleneck convolution module and a C2f module. The moving reverse bottleneck convolution module includes a dimensionality increase convolution layer, a depth convolution layer, an SE module and a dimensionality reduction convolution layer. The input image receiving layer receives the image to be detected and generates a first feature map after a convolution operation. The dimensionality increase convolution layer receives the first feature map for dimensionality increase processing and then inputs it to the depth convolution layer for feature extraction, and then inputs it to the SE module for feature recalibration. Finally, it is input to the dimensionality reduction convolution layer for dimensionality reduction to generate a second feature map. The C2f module receives the second feature map and generates an intermediate feature map.

[0053] The neck network receives the intermediate feature map and performs splicing and fusion operations to generate a spliced ​​feature map. The head network includes a convolution kernel attention fusion module and a detection module. The convolution kernel attention fusion module receives the spliced ​​feature map and reweights it to generate a third feature map containing local features and global information. The detection module receives the third feature map and processes and outputs a fourth feature map.

[0054] Because the mobile reverse bottleneck convolution (MB Conv) module can focus more on the key areas in the image through its specific design, such as deep convolution layer and SE module, it can more effectively extract the features of small targets. At the same time, the feature information of small targets is often distributed in multiple scales of the image. The MB Conv module may better capture these features through its internal multi-scale feature fusion mechanism, such as feature pyramid network FPN. This helps to improve the detection accuracy of PCB defect detection model for small targets.

[0055] In order to improve the detection accuracy and generalization ability of the PCB defect detection model, a PCB sample image data set is collected to train the PCB defect detection model. The PCB sample image data set of this embodiment includes a public data set and a self-collected PCB image data set. The use of the public data set can increase the diversity of the sample image data set and ensure that the PCB defect detection model in the embodiment of the present invention can better adapt to the PCB detection needs in different scenarios. At the same time, the self-collected PCB image data set further supplements the various complex and diverse situations that may be encountered in actual production, thereby enhancing the practicality and representativeness of the PCB sample image data set. Of course, the PCB sample image data set can only include the self-collected PCB image data set as needed.

[0056] Specifically, the public data set is preferably the Peking University PCB data set PCB_DATASET, which covers a variety of typical PCB defect types, such as missing holes, rat bites, open circuits, short circuits, mixed copper and fake copper. The self-collected PCB image data set is preprocessed and labeled in the VOC annotation mode using the LabelImg tool to obtain the xml label file corresponding to the PCB defect image file, and create a txt file to store the name of the annotation category.

[0057] Perform preprocessing operations on the PCB sample image dataset to generate a preprocessed dataset, which is divided into a training set, a validation set, and a test set in proportion. The specific steps include:

[0058] In step S11, the image is enhanced by adjusting parameters such as contrast and brightness of the image.

[0059] In step S12, the image is adjusted to a uniform scale to meet the input requirements of the model.

[0060] In step S13, the image format is converted from RGB to grayscale image.

[0061] In step S14, the diversity of the sample image data set is enhanced by rotating, cropping, scaling, etc., so as to improve the generalization ability of the PCB defect detection model.

[0062] In step S15, according to the characteristics of the PCB defects, appropriate features are selected for extraction, such as texture features, shape features and depth features.

[0063] In step S16, the extracted features are preprocessed to eliminate the impact of dimensional differences and numerical ranges on the detection model. The specific steps include: first, the feature marks are standardized to eliminate dimensional differences and numerical range differences between different features. Specifically, the average value of all features is changed to 0 and the variance is changed to 1. The specific calculation formula is:

[0064]

[0065] Among them, X is the original eigenvalue, μ is the mean of the feature, σ is the standard deviation of the feature, X std is the standardized eigenvalue; secondly, the features are normalized so that all features are in the same numerical range. Specifically, the eigenvalues ​​are scaled to a small specific interval. The specific calculation formula is:

[0066]

[0067] Among them, X is the original eigenvalue, X max and X min are the maximum and minimum values ​​of the feature, respectively, X ndh is the normalized eigenvalue; then, the label encoding method is used to convert discrete features into numerical features for easy model processing; finally, linear discriminant analysis is used to reduce the dimension of the features to reduce the complexity of the calculation and avoid overfitting.

[0068] In step S17, the preprocessed data set (i.e., the preprocessed data set) is randomly divided into a training set, a validation set, and a test set in proportion. In this embodiment, a ratio of 8:1:1 is preferably used for division, that is, 80% of the data is used for model training, 10% of the data is used for model validation, and 10% of the data is used for model testing. Of course, the choice of proportion should be determined according to the needs of the usage scenario and the actual situation of the data. 80% of the data is used for the training set, which can ensure that the model has enough samples to learn its inherent laws and characteristics, and improve the generalization ability and detection accuracy of the model. 10% of the data is used for the validation set, and the model performance can be evaluated at different stages of model training, so as to select hyperparameters for the model, such as the optimal number of iterations and learning rate. 10% of the data is used for the test set, which can provide an independent model performance evaluation that is not affected by the training process, representing the generalization ability of the detection model.

[0069] The preprocessed data set obtained after preprocessing the sample image data set is input into the PCB defect detection model for model learning and training. Through continuous iteration and optimization, the detection model can accurately identify various PCB defects from the image data.

[0070] Specifically, the PCB defect detection model is modeled by integrating the neural network architecture EfficientNet-B7 on the basic network model of YOLOv8. EfficientNet-B7 is an efficient convolutional neural network model. Its core lies in its hybrid dimensional scaling method, which simultaneously adjusts the depth, width and resolution of the network through a set of fixed scaling factors (α, β, γ). Here, α, β, and γ are constants determined by grid search to balance the scaling ratio between different dimensions. At the same time, B7 has significant advantages in the EfficientNet series. It can not only maintain high accuracy but also significantly reduce the computational cost and model complexity, thereby achieving network optimization. The specific implementation steps include:

[0071] In step S21, the EfficientNet-B7 network is integrated into the backbone network of YOLOv8, and the convolutional block in YOLOv8 is replaced with the MB Conv module of EfficientNet-B7.

[0072] In step S22, the residual connection and bottleneck structure of YOLOv8 are retained, and the composite coefficient φ of EfficientNet-B7 is used to adjust the network parameters.

[0073] In step S23, since the network structure of EfficientNet-B7 is deeper, the learning rate needs to be adjusted during training to avoid gradient explosion or training instability. Generally, a deeper network structure requires a smaller initial learning rate to ensure smooth training. Therefore, according to the network structure of EfficientNet-B7, the learning rate of the network model is adjusted so that its initial learning rate is smaller.

[0074] In step S24, the PCB defect detection model is trained using the preprocessed data set, the value of the optimized loss function is calculated according to the training results, and the optimized loss function is iteratively trained using the back propagation algorithm, and the model parameters are continuously adjusted to minimize the loss function to obtain the final optimized PCB defect detection model.

[0075] In the loss function, the CIoU Loss loss function introduces the aspect ratio of the predicted bounding box and the true bounding box, as well as the center point distance, based on the IoU loss, so that the model can more accurately capture the shape and position information of the target, thereby improving the accuracy of target detection. The Focal Loss loss function introduces a modulation factor to make the model training process pay more attention to samples that are difficult to classify, and can solve the problem of model performance degradation caused by category imbalance. Therefore, in this embodiment, the preferred loss function includes the CIoU Loss loss function and the Focal Loss loss function, that is, the Focal Loss loss function is added to the CIOU Loss loss function to penalize the misclassification of a few categories, thereby further improving the classification ability of the model.

[0076] The optimized loss function of the PCB defect detection model has a detailed consideration of the center point distance and size difference, which enables the model to achieve higher-precision bounding box positioning in PCB defect detection, helping to more accurately identify various small defects; and compared with the traditional loss function, the optimized loss function has faster convergence, which can further improve the training efficiency of the model; at the same time, the optimized loss function also adds factors such as the minimum enclosing box size to further enhance the generalization ability of the network model for defect detection of different sizes and positions on the PCB, so that the model can better adapt to various complex PCB defect detection scenarios. This design improves the accuracy of the PCB defect detection model in detecting small target features, achieves faster convergence speed and lower complexity, and increases its generalization ability.

[0077] Specifically, the calculation formula of CIoU Loss loss function is:

[0078]

[0079] Among them, IoU is the intersection over union ratio between two bounding boxes, which is the basic indicator for evaluating the degree of overlap of bounding boxes; ρ 2 (A, B) is the square of the Euclidean distance between the center point of the predicted box and the real box, which reflects the difference in the position of the center points of the two boxes; c 2 is the square of the diagonal length of the minimum circumscribed rectangle of the two boxes. This length is used to normalize the center point distance so that the loss function has scale invariance; v is a correction factor used to further adjust the loss function to consider the aspect ratio information of the bounding box; α is a balance parameter used to adjust the weight of the v term in the loss function. Its specific value is usually calculated dynamically according to IoU. The calculation formula of α is:

[0080]

[0081] The calculation formula for v is:

[0082]

[0083] Among them, w gt and h gt are the width and height of the true bounding box, w and h are the width and height of the predicted bounding box. This formula is calculated by calculating the square of the angle difference corresponding to the aspect ratio of the true bounding box and the predicted bounding box (and multiplying it by a constant scaled) to reflect the difference in shape between the two.

[0084] Specifically, the Focal Loss loss function calculation formula is:

[0085] L f-l =-α t (1-p t ) γ ×log(p t ),

[0086] Among them, α t is the balance coefficient, which is used to adjust the importance of positive and negative samples; p t is the predicted probability of the model for the sample. When the true label of the sample is 1, p t is the probability that the model predicts the positive category; γ is the modulation coefficient, which is used to adjust the weight reduction rate of easy-to-classify samples.

[0087] Specifically, the penalty term formula for optimizing the loss function is:

[0088]

[0089] Among them, C w and C h are the side lengths of the minimum circumscribed rectangles of the two bounding boxes in the width and height directions respectively; λ1, λ2, and λ3 are hyperparameters, also known as weight coefficients, which are used to adjust the contribution of different loss items in the total loss function.

[0090] The penalty term formula can be decomposed into: IoU loss, center point distance loss and edge length loss. The IoU loss is used to calculate the loss of the intersection-over-union ratio between the predicted bounding box and the true bounding box. The larger the value, the higher the overlap between the predicted bounding box and the true bounding box, and the better the performance of the target detection model. The specific formula is:

[0091] L IoU = 1-IoU;

[0092] The center point distance loss is used to calculate the Euclidean distance loss between the center point of the predicted bounding box and the center point of the true bounding box, which helps to make the center point of the predicted bounding box closer to the center point of the true bounding box during training. The specific formula is:

[0093]

[0094] The edge length loss is used to directly penalize the difference between the width and height of the predicted bounding box and the width and height of the true bounding box, which helps to solve the problem of incorrectly magnifying the edge length that may exist in the loss function. The specific formula is:

[0095]

[0096] By introducing penalty terms for the width and height of the predicted bounding box and the true bounding box, the optimized loss function can ensure that the predicted bounding box is closer to the true bounding box in shape, which helps to better match the shape and size of defects during PCB defect detection, thereby improving the detection accuracy of the PCB defect detection model.

[0097] By iteratively training the optimized loss function, the detection model parameters are continuously adjusted to minimize the value of the loss function to improve the model generalization ability and detection accuracy. In order to further enhance the local capture ability of the PCB defect detection model for image features, a convolution kernel attention fusion module is added before the YOLOv8 detection module. The introduction of this module can more accurately locate the target bounding box and predict the category. The specific steps include:

[0098] Add the code of the convolution kernel attention fusion module before the detection module of the YOLOv8 algorithm;

[0099] Modify the YOLOv8 configuration file and include the configuration information of the convolution kernel attention fusion module in the YOLOv8 large configuration file;

[0100] Add the calling code of the convolution kernel attention fusion module to the optimized YOLOv8 network structure code;

[0101] Adjust the training parameters of the PCB defect detection model, such as training rounds and learning rate, and train the model again using the preprocessed dataset;

[0102] Use the validation set to evaluate the performance of the model and compare it with the performance without the convolution kernel attention fusion module to determine whether the introduction of the attention mechanism can improve the performance of the detection model;

[0103] The optimized PCB defect detection model is tested using the test set to obtain the corresponding PCB defect detection results.

[0104] Placing the convolution kernel attention fusion module before the detection module of the head network can enhance the feature extraction capability of the detection module. The convolution kernel attention fusion module re-weights the features through the self-attention mechanism, so that important features receive more attention, while the attention of unimportant features is suppressed, which helps the defect detection model to more easily identify the target defects; at the same time, the self-attention mechanism in the convolution kernel attention fusion module can be combined with the multi-scale feature extraction of the convolution operation in YOLOv8, further improving the detection module's adaptability to feature map detection of different scales.

[0105] The image of the PCB to be inspected is input into the optimized PCB defect detection model to obtain the defect detection result of the PCB to be inspected.

[0106] Specifically, when collecting image data of the PCB to be inspected, since the specifications and sizes of each PCB may be different, it is impossible to use a unified standard to determine the number of images required. For some small or compactly designed PCBs, it may only be necessary to take one image to fully cover its surface information and ensure that all key areas and details are accurately recorded. However, for large, complex or PCBs with special layouts, a single image may not be able to fully display all of its surface details. At this time, it is necessary to take multiple images and splice them into a complete and complete image set through appropriate combinations to fully cover the entire surface of the PCB. In actual operation, the number of images required and the shooting method can be flexibly determined according to the specific specifications, sizes and inspection requirements of the PCB.

[0107] In an embodiment of the present invention, a PCB defect detection model includes a backbone network (Backbone), a neck network (Neck) and a head network (Head). Backbone is used to receive an image to be detected and extract features from the image to be detected. Neck is used to fuse feature maps from different stages of Backbone to improve the performance of the model. Through feature fusion, Neck can provide richer and more accurate feature representation. Head is used to output defect detection results based on the features extracted by Neck.

[0108] Specifically, Backbone includes a layer responsible for receiving an input image (Stem Layer) and a feature extraction layer (StageLayer). The Stem Layer includes a first convolution module (Conv module), wherein the Conv module includes a two-dimensional convolution layer (Conv2d layer), a two-dimensional batch normalization layer (BatchNorm2d layer) and an activation function. The Conv2d layer is used to complete the convolution operation and generate a feature map, and the BatchNorm2d layer is used to complete the batch normalization operation, which can improve the stability and convergence speed of the training. The activation function can be selected according to factors such as the task type, the characteristics of the training data set, and the expected model performance. Because the SiLU activation function has the advantages of smoothness, nonlinear characteristics, self-regularization effect, no zero output area, good gradient propagation performance, approximation performance, low computational complexity and strong adaptability. In this embodiment, the SiLU activation function is preferred, which can help avoid problems such as gradient disappearance during model training. The number of Stage Layer layers has no fixed standard in the YOLOv8 algorithm, and is determined according to the design goals of the algorithm, computing resource limitations, and model performance requirements. In this embodiment, four feature extraction layers are preferred. The first feature extraction layer (Stage Layer 1), the second feature extraction layer (Stage Layer 2) and the third feature extraction layer (Stage Layer 3) all contain MB Conv modules and C2f modules, and the fourth feature extraction layer (Stage Layer 4) contains MB Conv modules, C2f modules and SPPF modules. The MB Conv module extracts and transforms the input feature map through convolution operations to generate new feature maps. The C2f module is used to fuse feature maps of different scales to generate feature maps that contain both high resolution and rich semantic information, thereby improving the target detection performance of the network model. The SPPF module extracts and fuses image features by performing pooling operations on the input feature maps at different scales. This multi-scale feature representation helps the model capture rich information in the image and improve the accuracy of target detection. Compared with traditional spatial pyramid pooling, SPPF optimizes the algorithm and structure to improve the processing speed.

[0109] Specifically, refer to Figure 2 , the PCB defect image to be detected will pass through the Stem Layer and four Stage Layer layers in Backbone in sequence. The specific steps include:

[0110] In step S31, the PCB defect image to be inspected passes through a Conv module used to reduce the number of feature map channels and thus reduce computational complexity. First, the image to be inspected is convolved in a 1×1Conv2d layer. After the BatchNorm2d layer normalizes the output result of the Conv2d layer, the SiLU activation function introduces a nonlinear factor to the output result of the BatchNorm2d layer, and outputs a 256×256×64 first feature map (i.e., feature map C0).

[0111] In step S32, refer to Figure 3 , the feature map C0 is input to the MB Conv module of Stage Layer 1. In the MBConv module, the 1×1 dimensionality increase convolution layer receives the feature map C0 for dimensionality increase processing and then inputs it to the 3×3 deep convolution layer (DWConv) for feature extraction and increasing the receptive field. It is then input to the SE module for feature recalibration and finally input to the 1×1 dimensionality reduction convolution layer for dimensionality reduction to generate the second feature map. The subsequent MB Conv module processing steps are similar to this and will not be repeated. The C2f module receives the second feature map and generates the intermediate feature map C1.

[0112] In step S33, the intermediate feature map C1 is input to the MB Conv module of Stage Layer 2 for processing, and then input to the C2f module to generate the intermediate feature map C2, and the intermediate feature map C2 is divided into feature map C21 and feature map C22. The feature map C21 is sent to Neck for the final splicing operation.

[0113] In step S34, feature map C22 is input into the MB Conv module and C2f module of Stage Layer 3 to generate an intermediate feature map C3, and the intermediate feature map C3 is divided into feature map C31 and feature map C32. Feature map C31 is sent to Neck for the first splicing operation.

[0114] In step S35, the feature map C32 is input to the MB Conv module of Stage Layer 4 for processing, and then input to the C2f module, and generates an intermediate feature map C4 after passing through the SPPF module.

[0115] Specifically, Neck includes a C2f module, an UpSample module, and a Concat module. The UpSample module is used to enlarge the size of the feature map so that it can be fused with the low-level feature map. The Concat module is used to concatenate different feature maps to implement feature fusion.

[0116] Specifically, Head includes a convolution kernel attention fusion (CAFMAttention) module and a detection (Detect) module. The CAFMAttention module includes a local branch and a global branch, which are used to fuse convolution and attention mechanisms to capture local and global features. The local branch is implemented by convolution and channel rearrangement, and the global branch uses the attention mechanism to model long-range feature dependencies. The Detect module is used to receive the feature map of the CAFMAttention module and output the final detection result.

[0117] Reference Figure 2 , Neck performs adaptive fusion of feature maps at different stages and Head outputs detection results. The specific steps include:

[0118] In step 1, the intermediate feature map C4 is input to Neck to complete the first CBL convolution operation, and then the UpSample module performs the upsampling operation, and the first splicing operation is performed with the feature map C31 in the Concat module, and then the result after the splicing operation is input to the C2f module to generate the intermediate feature map C5, and the intermediate feature map C5 is divided into feature map C51 and feature map C52. After the feature map C51 completes the upsampling operation in the UpSample module, it is directly passed to the Concat module for the second splicing operation, and then passed to the C2f module to generate feature map C6 (i.e., splicing feature map), completing the feature fusion operation, and transmitting feature map C6 to the CAFMAttention module.

[0119] In step 2, refer to Figure 4 , the CAFMAttention module receives the feature map C6, first performs three 3×3 convolution operations (Conv) through the local branch to extract the local features of the image, then rearranges the channel order of the feature map (i.e., channel shuffling), and then performs three 3×3 convolution operations, and finally weights to generate a feature map containing local features (i.e., local feature map); at the same time, the global branch operates similarly to the local branch. After weighting to obtain a feature map containing global information (i.e., global information feature map), the output of the fully connected layer is converted into a probability distribution through the Softmax function, so that the sum of the predicted probabilities of each category is 1, and then the weights between the two branches are dynamically adjusted through the attention mechanism (Attention) module to generate a weight map so that the network can pay more attention to important feature information; finally, the network outputs the generated prediction result (i.e., the third feature map) through an output layer, and transmits the prediction result to the Detect module, which generates a 64×64×256 feature map (i.e., the fourth feature map).

[0120] In step three, feature map C52 is passed to the Concat module for the third splicing operation, and then passed to multiple Bottleneck blocks for further processing. The feature maps input to Bottleneck are processed through a series of convolutions, normalizations, and activations to generate feature maps that will complete the fourth splicing operation with the intermediate feature map C4 in the Concat module, and finally passed to the C2f module to generate feature map C7 to complete the feature fusion operation. Feature map C7 is transmitted to the CAFMAttention module. The processing steps after the CAFMAttention module receives the feature map C7 are similar to those for generating the third feature map, which will not be repeated here. The CAFMAttention module inputs the processed results to the Detect module, and the Detect module generates a 16×16×512 feature map.

[0121] In step 4, feature map C522 generates feature map C8 after passing through the C2f module once, and feature map C8 is input to the CAFMAttention module. The processing steps after the CAFMAttention module receives the feature map C8 are similar to those for generating the third feature map, which will not be repeated here. The CAFMAttention module transmits the processed results to the Detect module, and the Detect module generates a 32×32×512 feature map.

[0122] Embodiment 2

[0123] Based on the same inventive concept, this embodiment provides a PCB defect detection system based on the YOLOv8 network. The principle of solving the problem is similar to that of the PCB defect detection method based on the YOLOv8 network, and the repeated parts are not repeated here.

[0124] The embodiment of the present invention provides a PCB defect detection system based on a YOLOv8 network, including:

[0125] The data set acquisition and preprocessing module is used to acquire the PCB sample image data set, and perform preprocessing operations on the sample image data set to generate a preprocessed data set;

[0126] A PCB defect detection model optimization module is used to input the preprocessed data set into the PCB defect detection model for training to obtain an optimized PCB defect detection model;

[0127] The defect detection module is used to input the image to be detected into the optimized PCB defect detection model to obtain the defect detection result of the PCB to be detected; the image to be detected is generated by collecting and processing the image data of the PCB to be detected;

[0128] Among them, the PCB defect detection model includes a backbone network, a neck network and a head network. The backbone network includes a receiving input image layer and a feature extraction layer. The feature extraction layer includes a mobile reverse bottleneck convolution module and a C2f module. The mobile reverse bottleneck convolution module includes a dimension-increasing convolution layer, a depth convolution layer, an SE module and a dimension-reducing convolution layer. The receiving input image layer receives the image to be detected and generates a first feature map after a convolution operation. The dimension-increasing convolution layer receives the first feature map, performs dimension-increasing processing, and then inputs it into the depth convolution layer for feature extraction, and then inputs it into the SE module for feature recalibration. Finally, it is input into the dimension-reducing convolution layer for dimension reduction to generate a second feature map. The C2f module receives the second feature map and generates an intermediate feature map.

[0129] The neck network receives the intermediate feature map and performs splicing and fusion operations to generate a spliced ​​feature map. The head network includes a convolution kernel attention fusion module and a detection module. The convolution kernel attention fusion module receives the spliced ​​feature map and reweights it to generate a third feature map containing local features and global information. The detection module receives the third feature map and processes and outputs a fourth feature map.

[0130] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0131] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0134] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A PCB defect detection method based on YOLOv8 network, characterized in that: include: Collecting a PCB sample image data set, performing a preprocessing operation on the sample image data set, and generating a preprocessing data set; Inputting the preprocessed data set into a PCB defect detection model for training to obtain an optimized PCB defect detection model; Inputting the image to be detected into the optimized PCB defect detection model to obtain a defect detection result of the PCB to be detected; the image to be detected is generated by collecting and processing image data of the PCB to be detected; The PCB defect detection model includes a backbone network, a neck network and a head network. The backbone network includes an input image receiving layer and a feature extraction layer. The feature extraction layer includes a mobile reverse bottleneck convolution module and a C2f module. The mobile reverse bottleneck convolution module includes a dimension-raising convolution layer, a depth convolution layer, an SE module and a dimension-reducing convolution layer. The input image receiving layer receives the image to be detected and generates a first feature map after a convolution operation. The dimension-raising convolution layer receives the first feature map, performs dimension-raising processing, and then inputs it into the depth convolution layer for feature extraction, and then inputs it into the SE module for feature recalibration. Finally, it is input into the dimension-reducing convolution layer for dimension reduction to generate a second feature map. The C2f module receives the second feature map and generates an intermediate feature map. The neck network receives the intermediate feature map and performs a splicing and fusion operation to generate a spliced ​​feature map. The head network includes a convolution kernel attention fusion module and a detection module. The convolution kernel attention fusion module receives the spliced ​​feature map and reweights it to generate a third feature map containing local features and global information. The detection module receives the third feature map and processes and outputs a fourth feature map.

2. According to a PCB defect detection method based on YOLOv8 network according to claim 1, it is characterized in that: The method of inputting the preprocessed data set into the PCB defect detection model for training is: The network parameters of the PCB defect detection model are adjusted by the composite coefficient of EfficientNet-B7; the learning rate of the PCB defect detection model is adjusted according to the network structure of EfficientNet-B7; The PCB defect detection model is trained using the preprocessed data set to obtain a training result, a value of an optimization loss function is calculated according to the training result, and the optimization loss function is iteratively trained using a back propagation algorithm, wherein the optimization loss function includes a function that penalizes misclassification of a small number of categories.

3. A PCB defect detection method based on YOLOv8 network according to claim 2, characterized in that: The penalty term formula of the optimization loss function is: Among them, IoU is the intersection over union ratio between the predicted bounding box and the true bounding box, ρ 2 (A, B) is the square of the Euclidean distance between the center point of the predicted box and the real box, c 2 is the square of the diagonal length of the minimum circumscribed rectangle of the two boxes, w gt and h gt are the width and height of the real bounding box, w and h are the width and height of the predicted bounding box, C w and C h are the side lengths of the minimum circumscribed rectangles of the two bounding boxes in the width and height directions, respectively. λ1, λ2 and λ3 are weight coefficients for adjusting the contribution of different loss terms in the overall optimization loss function.

4. A PCB defect detection method based on YOLOv8 network according to claim 2, characterized in that: The optimization loss function includes the CIoU Loss loss function, and the function for penalizing misclassification of a few categories is the FocalLoss loss function.

5. A PCB defect detection method based on YOLOv8 network according to claim 4, characterized in that: The CIoU Loss loss function is: Among them, v is the correction factor used to further adjust the loss function, α is the balance parameter used to adjust the weight of the v term in the loss function, and the calculation formula of α is: The calculation formula for v is:

6. A PCB defect detection method based on YOLOv8 network according to claim 4, characterized in that , the Focal Loss loss function is: 50 f-l =-α t (1-p t ) γ ×log(p t ); Among them, α t is the balance coefficient used to adjust the importance of positive and negative samples, p t is the predicted probability of the sample by the PCB defect detection model. When the true label of the sample is 1, p t is the probability that the model predicts the positive category, and γ is the modulation coefficient used to adjust the rate at which the weight of easy-to-classify samples decreases.

7. A PCB defect detection method based on YOLOv8 network according to claim 1, characterized in that: There are multiple feature extraction layers, and at least one feature extraction layer includes an SPPF module.

8. A PCB defect detection method based on YOLOv8 network according to claim 1, characterized in that: The input image receiving layer includes a first convolution module, wherein the first convolution module includes a two-dimensional convolution layer, a two-dimensional batch normalization layer and an activation function, the two-dimensional convolution layer performs a convolution operation on the input image, the two-dimensional batch normalization layer normalizes the output result of the two-dimensional convolution layer, and the activation function introduces a nonlinear factor to the output result of the two-dimensional batch normalization layer.

9. A PCB defect detection method based on YOLOv8 network according to claim 1, characterized in that: The convolution kernel attention fusion module receives the concatenated feature map and re-weights it to generate a third feature map containing local features and global information. The method is: The local branch and the global branch of the convolution kernel attention fusion module independently perform convolution and weighting operations on the spliced ​​feature map to generate their own local feature maps and global information feature maps; Inputting the local feature map and the global information feature map into the attention mechanism module of the convolution kernel attention fusion module to perform a weight dynamic adjustment operation to generate a weight map; The local feature map and the global information feature map are weightedly fused according to the weight map to generate the third feature map.

10. A PCB defect detection system based on YOLOv8 network, characterized in that ,include: The data set acquisition and preprocessing module is used to acquire the PCB sample image data set, and perform preprocessing operations on the sample image data set to generate a preprocessed data set; A PCB defect detection model optimization module is used to input the preprocessed data set into the PCB defect detection model for training to obtain an optimized PCB defect detection model; A defect detection module, used for inputting the image to be detected into the optimized PCB defect detection model to obtain a defect detection result of the PCB to be detected; the image to be detected is generated by collecting and processing image data of the PCB to be detected; The PCB defect detection model includes a backbone network, a neck network and a head network. The backbone network includes an input image receiving layer and a feature extraction layer. The feature extraction layer includes a mobile reverse bottleneck convolution module and a C2f module. The mobile reverse bottleneck convolution module includes a dimension-raising convolution layer, a depth convolution layer, an SE module and a dimension-reducing convolution layer. The input image receiving layer receives the image to be detected and generates a first feature map after a convolution operation. The dimension-raising convolution layer receives the first feature map, performs dimension-raising processing, and then inputs it into the depth convolution layer for feature extraction, and then inputs it into the SE module for feature recalibration. Finally, it is input into the dimension-reducing convolution layer for dimension reduction to generate a second feature map. The C2f module receives the second feature map and generates an intermediate feature map. The neck network receives the intermediate feature map and performs a splicing and fusion operation to generate a spliced ​​feature map. The head network includes a convolution kernel attention fusion module and a detection module. The convolution kernel attention fusion module receives the spliced ​​feature map and reweights it to generate a third feature map containing local features and global information. The detection module receives the third feature map and processes and outputs a fourth feature map.

Images