PCB defect detection method based on improved YOLOv8

Abstract

The invention belongs to the technical field of industrial defect detection, and particularly relates to a PCB defect detection method based on improved YOLOv8, which comprises the following steps: 1) establishing a YOLOv8 model, improving an extraction part, a C2f module and a PAN-FPN module in the YOLOv8 model, and designing an input end of the YOLOv8 model to realize data enhancement of an image; and finally, an improved YOLOv8 model is constructed. 2) based on the YOLOv8 model improved in the step 1), inputting the acquired images of various PCB defect types into the input end of the improved YOLOv8 model through an industrial camera for data set training; in the training process, data enhancement is realized to promote the model to better utilize the correlation between different samples, so that the generalization ability and accuracy of the model are improved; and 3) in combination with a neural network structure for realizing different scale feature fusion in the improved YOLOv8 model, improving the detection capability of defects of different sizes and shapes of the PCB, outputting identified and positioned PCB defect areas, and generating a corresponding detection report.

Description

Technical Field

[0001] The invention belongs to the technical field of industrial defect detection, and specifically is a PCB defect detection method based on improved YOLOv8. Background Art

[0002] With the continuous development and popularization of electronic products, the manufacturing quality and efficiency of printed circuit boards (PCBs), as the core components of electronic devices, are directly related to the performance and market competitiveness of the entire product. PCB defect identification is of great significance to ensure the quality of electronic products. However, various defects often occur in the production process of PCB boards. Among them, the common types of PCB defects are divided into six categories: holes, gaps, open circuits, short circuits, burrs, and residual copper. Figure 1 As shown. If these defects are not discovered and repaired in time, they will have a serious impact on the performance and reliability of the product. Therefore, the development of efficient and accurate PCB board defect detection technology has become an urgent need in the electronics manufacturing industry.

[0003] Traditional manual inspection methods are not only inefficient, but also easily affected by subjective factors, leading to false detection and missed detection. Defect detection through image recognition technology can timely discover and repair defects on PCB boards, thereby improving the yield and quality of electronic products. At present, PCB board defect detection technology based on image recognition mainly includes traditional image processing methods and deep learning methods. These methods have certain effects in simple scenarios, but there are problems such as low detection accuracy and poor model generalization ability. Summary of the invention

[0004] The purpose of the present invention is to provide a PCB defect detection method improved based on YOLOv8, which optimizes the C2f module, data enhancement method and multi-scale fusion method, can reduce the amount of calculation, improve generalization ability and detection accuracy.

[0005] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: a PCB defect detection method based on improved YOLOv8, comprising the following steps:

[0006] 1) Establish a YOLOv8 model, and improve the extraction part, C2f module, and PAN-FPN module in the YOLOv8 model. At the same time, design the input end of the YOLOv8 model to achieve image data enhancement; finally, build an improved YOLOv8 model;

[0007] 2) Based on the improved YOLOv8 model in step 1), the images of various types of PCB defects collected by the industrial camera are input into the input end of the improved YOLOv8 model for data set training; during the training process, data enhancement is implemented to enable the model to better utilize the correlation between different samples, thereby improving the generalization ability and accuracy of the model;

[0008] 3) Combined with the neural network structure that realizes the fusion of features of different scales in the improved YOLOv8 model, the detection capability of PCB defects of different sizes and shapes is improved, the identified and located PCB defect areas are output, and the corresponding inspection report is generated.

[0009] The improved YOLOv8 model is constructed as follows:

[0010] 1-1) Use Z-Pool pooling layer, Conv convolution layer and Sigmoid activation function to establish the extraction part of YOLOv8 model;

[0011] 1-2) Add the Triplet attention mechanism after each Bottleneck structure in the original C2f module to form a C2f_T module;

[0012] 1-3) Apply the PredMix algorithm at the input of the established YOLOv8 model to achieve data enhancement;

[0013] 1-4) Replace the PAN-FPN module of the neural network structure used for target detection and semantic segmentation tasks in the YOLOv8 model with the BiFPN multi-scale module to improve detection accuracy.

[0014] The step 1-1) is specifically:

[0015] a. Perform pooling operation through Z-Pool pooling layer to reduce the input data dimension;

[0016] b. Extract features from the input data through a series of convolution kernels in the Conv convolution layer;

[0017] c. The Conv layer and the Z-Pool layer are used alternately to form the feature extraction part of the model;

[0018] d. The Sigmoid activation function is used to increase the nonlinear characteristics of the model and map the input data between 0 and 1 so that the model can fit more complex functional relationships.

[0019] In step 1-2), the Triplet attention analyzes the input data by introducing three different perspectives to jointly determine the most noteworthy part;

[0020] After rotation, pooling, convolution, normalization and activation function, the three branches of the Triplet attention mechanism network not only obtain the weights of the channel dimension and width dimension, but also obtain the weight information of the channel dimension, height dimension and spatial dimension;

[0021] The obtained weight information is fused to obtain a feature map of the same size as the original one, realizing the function of the attention mechanism for different dimensions.

[0022] In step 2), the input design of the YOLOv8 model implements data enhancement during training, including the following steps:

[0023] 2-1) Input multiple PCB defect images into the input end of the improved YOLOv8 model to obtain the annotated PCB label image, and train the entire data set to generate the predicted image of the training set;

[0024] 2-2) The PCB label image and the predicted image are mixed with random weight ratios to generate new training samples for training. The PredMix algorithm is used to simulate the overlap and occlusion of the target to generate new training samples for YOLOv8 model training.

[0025] In step 2-1), the entire data set is trained as follows:

[0026] The entire data set is divided into four parts: training set A, training set B, validation set and test set. Training is performed on training sets A and B respectively. The two trained network models A and B are used to cross-predict the other part of the training set. The detection results of each PCB training set image, that is, the specific coordinate position, are saved for subsequent data enhancement operations.

[0027] The step 2-2) is specifically:

[0028] The PredMix algorithm is used to randomly select a label part image (x i truth ,y i truth ) and a random predicted rectangular frame part image on another dataset image Combined to generate new training samples;

[0029] The size of the predicted rectangle and the label rectangle are inconsistent. You need to first j pred Resize to x i truth Consistent, the generated training samples (x, y) are used for model training, that is:

[0030]

[0031] y=y i truth

[0032] Among them, let x truth ∈R H×W×C Represents a label part of an image in training, y truth Represents a label; x pred ∈R H×W×C Represents a predicted rectangular box part of an image in the predicted image of the training set, y pred Represents the predicted classification.

[0033] The step 3) is specifically:

[0034] 3-1) Input features from feature maps of feature pyramid networks at different levels to the BiFPN multi-scale module in the improved YOLOv8 model;

[0035] 3-2) Top-down path:

[0036] In the BiFPN multi-scale module, upsampling operation is used to transfer high-level feature information to low-level ones.

[0037] 3-3) Bottom-up path:

[0038] In the BiFPN multi-scale module, pooling or convolution operations are used to transfer low-level feature information to high-level ones.

[0039] 3-4) Feature Fusion:

[0040] The BiFPN multi-scale module uses feature addition or feature concatenation to fuse features from the top-down path and the bottom-up path;

[0041] 3-5) After feature fusion is completed, feature adjustment operations are performed to further improve the representation ability of features and adapt to targets of different scales;

[0042] 3-6) The improved YOLOv8 model outputs fusion features of multiple scales, that is, outputs the identified and located PCB defect areas; at the same time, based on the identified and located PCB defect areas, the host computer automatically generates a corresponding inspection report.

[0043] The types of PCB defects include: holes, gaps, open circuits, short circuits, burrs and residual copper.

[0044] The present invention has the following beneficial effects and advantages: The present invention provides a PCB defect detection method based on YOLOv8 improvement, which can reduce the amount of calculation, improve the generalization ability and detection accuracy. Compared with other methods, the present invention has higher accuracy and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a schematic diagram of the principle of the improved YOLOv8 model of the present invention;

[0046] Figure 2 These are the six common PCB defects in the industry of the present invention;

[0047] Figure 3 This is the network result diagram of the Triplet attention mechanism of the present invention;

[0048] Figure 4 This is the network result diagram of the Triplet attention mechanism of the present invention;

[0049] Figure 5 It is the data set pre-operation process of the present invention;

[0050] Figure 6 This is the Bi-FPN structure diagram of the present invention. DETAILED DESCRIPTION

[0051] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the invention. Therefore, the present invention is not limited to the specific implementation disclosed below.

[0053] like Figure 1 As shown, it is a schematic diagram of the principle of the improved YOLOv8 model of the present invention. The present invention provides a PCB defect detection method based on the improved YOLOv8, comprising the following steps:

[0054] 1) Establish a YOLOv8 model, and improve the extraction part, C2f module, and PAN-FPN module in the YOLOv8 model. At the same time, design the input end of the YOLOv8 model to achieve image data enhancement; finally, build an improved YOLOv8 model;

[0055] like Figure 2As shown in the figure, it is the research object of the present invention. It lists six types of defects that may occur in the production process of PCB boards, such as holes, gaps, open circuits, short circuits, burrs, and residual copper.

[0056] 2) Based on the improved YOLOv8 model in step 1), the images of various types of PCB defects collected by the industrial camera are input into the input end of the improved YOLOv8 model for data set training; during the training process, data enhancement is implemented to enable the model to better utilize the correlation between different samples, thereby improving the generalization ability and accuracy of the model;

[0057] 3) Combined with the neural network structure that realizes the fusion of features of different scales in the improved YOLOv8 model, the detection capability of PCB defects of different sizes and shapes is improved, the identified and located PCB defect areas are output, and the corresponding inspection report is generated.

[0058] 3-1) Input features from feature maps of feature pyramid networks at different levels to the BiFPN multi-scale module in the improved YOLOv8 model;

[0059] 3-2) Top-down path:

[0060] In the BiFPN multi-scale module, upsampling operation is used to transfer high-level feature information to low-level ones.

[0061] 3-3) Bottom-up path:

[0062] In the BiFPN multi-scale module, pooling or convolution operations are used to transfer low-level feature information to high-level ones.

[0063] 3-4) Feature Fusion:

[0064] The BiFPN multi-scale module uses feature addition or feature concatenation to fuse features from the top-down path and the bottom-up path;

[0065] 3-5) After feature fusion is completed, feature adjustment operations are performed to further improve the representation ability of features and adapt to targets of different scales;

[0066] 3-6) The improved YOLOv8 model outputs fusion features of multiple scales, that is, outputs the identified and located PCB defect areas; at the same time, based on the identified and located PCB defect areas, the host computer automatically generates a corresponding inspection report.

[0067] The specific technical solution adopted by the present invention to achieve the above-mentioned purpose includes the following features:

[0068] 1) Algorithm framework:

[0069] Build the improved YOLOv8 model, specifically:

[0070] 1-1) Use Z-Pool pooling layer, Conv convolution layer and Sigmoid activation function to establish the extraction part of YOLOv8 model;

[0071] 1-2) Add the Triplet attention mechanism after each Bottleneck structure in the original C2f module to form a C2f_T module;

[0072] 1-3) Apply the PredMix algorithm at the input of the established YOLOv8 model to achieve data enhancement;

[0073] 1-4) Replace the PAN-FPN module of the neural network structure used for target detection and semantic segmentation tasks in the YOLOv8 model with the BiFPN multi-scale module to improve detection accuracy.

[0074] Considering that the defect points in PCB defect recognition and detection tasks are not only of various types but also have great differences in size, shape, and position, the detection is extremely difficult and the detection accuracy is not high. In order to enhance the feature extraction learning ability of the detection network, further improve the detection ability of small targets in PCB images, and reduce the amount of calculation, the Triplet attention mechanism can be introduced into the C2f module. The C2f structure is as follows: Figure 3 shown.

[0075] like Figure 3 The figure shows the original C2f module structure. It combines the advantages of YOLOv7, reduces a standard convolution layer, and makes full use of the bottleneck module to enhance the gradient branch. This method retains the lightweight characteristics and can capture richer gradient flow information, but the feature learning and extraction capabilities are not greatly improved.

[0076] The C2f_T module is designed to replace the C2f module as the main residual learning module. The design idea is to add a Triplet attention mechanism behind each Bottleneck structure in the C2f module. The Triplet attention mechanism analyzes the input data by introducing three different perspectives to jointly decide which parts are most worthy of attention. The C2f_T module aims to significantly enhance the model's capabilities in feature extraction and cross-layer feature integration, and the Triplet attention mechanism introduces almost no additional parameters, maintaining the efficiency and simplicity of the model. The network structure of the Triplet attention mechanism is as follows: Figure 4 shown.

[0077] like Figure 4As shown, the Triplet attention mechanism used in the present invention. It obtains corresponding weight information through a series of similar operations through 3 different branches, and performs fusion operations according to the weight information to realize the attention mechanism function of paying attention to multi-dimensional information. First, the pooling operation is performed through the Z-Pool layer, which can retain important feature information while reducing the dimension of the input data; then through the Conv layer, as one of the core components in the deep learning model, the convolution layer extracts features from the input data through a series of convolution kernels, and the alternating use of the Conv layer and the Z-Pool layer together constitutes the feature extraction part of the model; finally, the nonlinear characteristics of the model are increased through the Sigmoid activation function, and the input data is mapped between 0 and 1, so that the model can fit more complex functional relationships. The three branches of the Triplet attention mechanism network obtain the weights of the channel dimension and the width dimension through rotation, pooling, convolution, normalization and activation function, and also obtain the weight information of the channel dimension, height dimension and space dimension. Finally, according to the obtained weight information, fusion is performed to obtain a feature map of the same size as the original, realizing the function of the attention mechanism for different dimensions.

[0078] 2) Data enhancement:

[0079] Apply PredMix to the input of YOLOv8. The entire data enhancement method is divided into two steps:

[0080] Step 1: Generate prediction data for the training set. Divide the entire data set into 4 parts: training set A, training set B, validation set, and test set. Train on training set A and B separately, and use the two trained network models A and B to cross-predict the other part of the training set. Save the detection results (specific coordinates) of each PCB training set image for subsequent data enhancement operations. The specific operation method is as follows: Figure 5 shown.

[0081] In the second step, the PCB label image and the predicted image are mixed with random weight ratios to generate new training samples for training. By simulating the overlap and occlusion of targets through PredMix, the model's detection ability for dense targets and incomplete targets can be improved.

[0082] like Figure 5 As shown in FIG. 1 , the data enhancement method PredMix used in the present invention is shown in FIG. 1 . The entire data enhancement process is designed into two key steps:

[0083] Step 1: The focus is on generating prediction data for the training set. This step can provide a rich prediction data foundation for subsequent data enhancement.

[0084] Step 2: Focus on mixing the PCB label image with the predicted image at a random weight ratio to generate new training samples for model training. Specifically, let x truth ∈R H×W×C Represents a label part of an image in training, y truth Represents a label; x pred ∈R H×W×C Represents a predicted rectangular frame part of an image in the prediction result of step 1, y pred Represents prediction classification. PredMix aims to randomly select a label part image (x i truth ,y i truth ) and a random predicted rectangular frame part image on another dataset image Combined to generate new training samples. The size of the predicted rectangle and the label rectangle is usually inconsistent, so first Resize to x i truth The generated training samples (x, y) are used for model training. The specific operation definition is shown in the following formula.

[0085]

[0086] y=y i truth

[0087] 3) Multi-scale module:

[0088] Introducing a simple and efficient BiFPN to replace PAN-FPN in the YOLOv8 model can improve the detection accuracy. Its network structure is as follows Figure 6 As shown. The working principle of BiFPN is to assign weights to each input, and according to the calculated weight information, the features are fused accordingly. If the contribution of the node is low, the weight is reduced or even deleted; if the node is important, a higher weight is assigned, which effectively reduces the number of parameters and the complexity of the network, so that the network model focuses more on learning key feature information, thereby improving the learning ability of the network. Moreover, BiFPN does not introduce additional parameters and transformation operations during the feature fusion process, which helps to reduce information loss and retain more feature information of PCB images. Therefore, the use of BiFPN can achieve more efficient feature fusion and information transfer in the feature pyramid network, while reducing the amount of calculation and memory usage.

[0089] like Figure 6As shown, the BiFPN structure used in the present invention is used. It eliminates unilateral inputs, assigns weights to each input, and performs corresponding feature fusion according to the calculated weight information. For feature maps with a single input path, due to their low contribution, their weights are reduced or even eliminated, and no additional processing is applied; when the feature map is fused with two input paths, if the feature maps have the same scale, cross-level fusion needs to introduce a new path from the trunk feature map and assign a higher weight to it, which can effectively reduce the number of parameters and reduce the complexity of the network, without introducing additional parameters or transformation operations, so that the network model can focus more on learning key feature information.

[0090] In summary, the present invention provides a PCB defect detection method based on YOLOv8, which can reduce the amount of calculation, improve the generalization ability and detection accuracy. Compared with other methods, the present invention has higher accuracy and robustness.

[0091] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0092] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A PCB defect detection method based on improved YOLOv8, characterized in that: The following steps are involved: 1) Establish a YOLOv8 model, and improve the extraction part, C2f module, and PAN-FPN module in the YOLOv8 model. At the same time, design the input end of the YOLOv8 model to achieve image data enhancement; Finally, the improved YOLOv8 model is constructed; 2) Based on the improved YOLOv8 model in step 1), the images of various types of PCB defects collected by the industrial camera are input into the input end of the improved YOLOv8 model for data set training; During the training process, data enhancement is implemented to enable the model to better utilize the correlation between different samples, thereby improving the generalization ability and accuracy of the model; 3) Combined with the neural network structure that realizes the fusion of features of different scales in the improved YOLOv8 model, the detection capability of PCB defects of different sizes and shapes is improved, the identified and located PCB defect areas are output, and the corresponding inspection report is generated.

2. A PCB defect detection method based on improved YOLOv8 according to claim 1, characterized in that: The improved YOLO8 model is constructed as follows: 1-1) Use Z-Pool pooling layer, Conv convolution layer and Sigmoid activation function to establish the extraction part of YOLOv8 model; 1-2) Add the Triplet attention mechanism after each Bottleneck structure in the original C2f module to form a C2f_T module; 1-3) Apply the PredMix algorithm at the input of the established YOLOv8 model to achieve data enhancement; 1-4) Replace the PAN-FPN module of the neural network structure used for target detection and semantic segmentation tasks in the YOLOv8 model with the BiFPN multi-scale module to improve detection accuracy.

3. A PCB defect detection method based on improved YOLOv8 according to claim 2, characterized in that: The step 1-1) is specifically: a. Perform pooling operation through Z-Pool pooling layer to reduce the input data dimension; b. Extract features from the input data through a series of convolution kernels in the Conv convolution layer; c. The Conv layer and the Z-Pool layer are used alternately to form the feature extraction part of the model; d. The Sigmoid activation function is used to increase the nonlinear characteristics of the model and map the input data between 0 and 1 so that the model can fit more complex functional relationships.

4. A PCB defect detection method based on improved YOLOv8 according to claim 2, characterized in that: In step 1-2), the Triplet attention analyzes the input data by introducing three different perspectives to jointly determine the most noteworthy part; After rotation, pooling, convolution, normalization and activation function, the three branches of the Triplet attention mechanism network not only obtain the weights of the channel dimension and width dimension, but also obtain the weight information of the channel dimension, height dimension and spatial dimension; The obtained weight information is fused to obtain a feature map of the same size as the original one, realizing the function of the attention mechanism for different dimensions.

5. A PCB defect detection method based on improved YOLOv8 according to claim 1, characterized in that: In step 2), the input design of the YOLOv8 model implements data enhancement during training, including the following steps: 2-1) Input multiple PCB defect images into the input end of the improved YOLOv8 model to obtain the annotated PCB label image, and train the entire data set to generate the predicted image of the training set; 2-2) The PCB label image and the predicted image are mixed with random weight ratios to generate new training samples for training. The PredMix algorithm is used to simulate the overlap and occlusion of the target to generate new training samples for YOLOv8 model training.

6. A PCB defect detection method based on improved YOLOv8 according to claim 5, characterized in that: In step 2-1), the entire data set is trained as follows: The entire data set is divided into four parts: training set A, training set B, validation set and test set. Training is performed on training sets A and B respectively. The two trained network models A and B are used to cross-predict the other part of the training set. The detection results of each PCB training set image, that is, the specific coordinate position, are saved for subsequent data enhancement operations.

7. A PCB defect detection method based on improved YOLOv8 according to claim 5, characterized in that: The step 2-2) is specifically: The PredMix algorithm is used to randomly select a label part image (x i truth ,y i truth ) and a random predicted rectangular frame part image on another dataset image Combined to generate new training samples; The size of the predicted rectangle and the label rectangle are inconsistent. You need to first j pred Resize to x i truth Consistent, the generated training samples (x, y) are used for model training, that is: y=y i truth Among them, let x truth ∈R H×W×C Represents a label part of an image in training, y truth Represents a label; x pred ∈R H ×W×C Represents a predicted rectangular box part of an image in the predicted image of the training set, y pred Represents the predicted classification.

8. A PCB defect detection method based on improved YOLOv8 according to claim 1, characterized in that: The step 3) is specifically: 3-1) Input features from feature maps of feature pyramid networks at different levels to the BiFPN multi-scale module in the improved YOLOv8 model; 3-2) Top-down path: In the BiFPN multi-scale module, upsampling operation is used to transfer high-level feature information to low-level ones. 3-3) Bottom-up path: In the BiFPN multi-scale module, pooling or convolution operations are used to transfer low-level feature information to high-level ones. 3-4) Feature Fusion: The BiFPN multi-scale module uses feature addition or feature concatenation to fuse features from the top-down path and the bottom-up path; 3-5) After feature fusion is completed, feature adjustment operations are performed to further improve the representation ability of features and adapt to targets of different scales; 3-6) The improved YOLOv8 model outputs fusion features of multiple scales, that is, outputs the identified and located PCB defect areas; at the same time, based on the identified and located PCB defect areas, the host computer automatically generates a corresponding inspection report.

9. A PCB defect detection method based on improved YOLOv8 according to claim 1, characterized in that: The types of PCB defects include: holes, gaps, open circuits, short circuits, burrs and residual copper.

Images