Circuit board appearance defect visual recognition method and system based on improved YOLO algorithm

Abstract

The present invention discloses a circuit board appearance defect visual recognition method and system based on an improved YOLO algorithm, the method comprising step S1: collecting an image data set containing circuit board appearance defects, and annotating and preprocessing the data set; step S2: improving YOLOv8n as a basic algorithm, and constructing an improved YOLO algorithm model; step S3: using the preprocessed data set to train the improved YOLO algorithm model; step S4: inputting the circuit board image to be detected into the trained improved YOLO algorithm model for detection, and outputting the location and category information of the defect. The present invention enhances the model's recognition ability for tiny defects and complex shape defects, and improves detection accuracy and efficiency by constructing a DSXConv module and introducing a new loss function EAPIOU.

Description

Technical Field

[0001] The present invention belongs to the technical field of circuit board appearance defect detection, and in particular relates to a circuit board appearance defect visual recognition method and system based on an improved YOLO algorithm. Background Art

[0002] As the core component of electronic products, the quality of circuit boards directly affects the performance and service life of electronic products. As electronic components develop towards miniaturization and integration, the density of components on circuit boards continues to increase, and the types and complexity of appearance defects are also increasing, which greatly increases the difficulty of detecting circuit board appearance defects.

[0003] Traditional circuit board appearance defect detection mainly relies on manual visual inspection, which is not only inefficient, but also easily affected by the subjective factors of the inspectors, making it difficult to ensure the accuracy and consistency of the detection results. With the development of computer vision and deep learning technology, automatic detection methods based on image processing and machine learning have gradually become a research hotspot. However, the existing deep learning-based detection algorithms still have some shortcomings when dealing with circuit board appearance defect detection tasks, such as low detection accuracy for minor defects and slow detection speed. Therefore, it is of great practical significance to develop an efficient and accurate visual recognition method for circuit board appearance defects. Summary of the invention

[0004] The purpose of the present invention is to provide a circuit board appearance defect visual recognition method and system based on an improved YOLO algorithm, so as to solve the problems of low detection accuracy and slow detection speed for tiny defects existing in the prior art.

[0005] In order to solve the above technical problems, the present invention adopts the following solutions:

[0006] A visual recognition method for circuit board appearance defects based on an improved YOLO algorithm comprises the following steps:

[0007] Step S1: Data preparation: Collect image datasets containing circuit board appearance defects, annotate and preprocess the datasets, and the preprocessing includes image enhancement and normalization operations. The image enhancement operations include random cropping, flipping, rotation, and brightness adjustment.

[0008] Step S2: Model construction: Improve the YOLOv8n algorithm as the basic algorithm to build an improved YOLO algorithm model. The specific improvements include:

[0009] Step S2.1: Construct a DSXConv convolution module based on the dynamic snake convolution Dsconv, and use the DSXConv convolution module to replace the ordinary convolution module in the backbone network C2f to form a C2f-DSM module, which further improves the model's detection accuracy for specific defects.

[0010] Step S2.2: Propose a new loss function EAPIOU, which enhances the position, aspect ratio and angle matching penalty mechanism.

[0011] Step S3: Model training: Use the preprocessed dataset to train the improved YOLO algorithm model.

[0012] Step S4: Defect detection: Input the circuit board image to be detected into the trained improved YOLO algorithm model for detection, and output the location and category information of the defects in the circuit board image.

[0013] Further optimization, the process of constructing the DSXConv convolution module in step S2.1 includes:

[0014] Step S2.1.1: Use snake convolutions of different sizes to extract tiny defect features. Dynamic snake convolution Dsconv improves the convolution kernel by introducing continuity constraints and iterative strategies to enhance the model's ability to focus on slender and tortuous features and improve the recognition accuracy of complex-shaped targets.

[0015] Step S2.1.2: Introduce the multi-dimensional collaborative attention module MCA. By introducing the multi-dimensional collaborative attention module MCA into the backbone network, multi-scale feature extraction, feature fusion and cross calculation are used to effectively utilize multi-scale features and background information, and enhance the model's perception of defects of different scales.

[0016] Step S2.1.3: The extracted micro-defect features are fused with the MCA module to finally obtain the DSXConv convolution module.

[0017] The DSXConv convolution module described in the present invention utilizes snake convolutions of different sizes to extract tiny defect features, and integrates the multi-dimensional collaborative attention module MCA to obtain the fusion, thereby improving the detection accuracy of the model for specific defects.

[0018] Further optimization, in step S2.1, first, snake convolution with kernel sizes of 3×3, 5×5, 7×7, and 9×9 is used to extract tiny defects in the circuit board, that is, the input x After convolution with 4 different convolution kernels, feature extraction is performed to obtain the feature map x m1 、x m2、x m3 and x m4 , the specific formula is:

[0019] (1);

[0020] The number of branches per channel is ¼ of the input channels, and the branches are independent and do not interfere with each other.

[0021] Secondly, the features extracted by snake convolution are introduced into the MCA module for processing to generate attention maps. xc 1. xc 2. xc 3 and xc 4. The specific formula is:

[0022] (2);

[0023] Then, the Concat operation is used in the connection layer to merge the features of all branches and obtain the input quantity Y. The specific formula is:

[0024] (3);

[0025] Finally, the processed input Y is compared with the original input information x Add and use 1×1 convolution to fuse information and get the output feature quantity:

[0026] (4);

[0027] in, x Indicates the feature quantity at the time of input, DsConv i Indicates that the convolution kernel size is i × i Convolution operation, i is 3, 5, 7 and 9; Concat represents the channel fusion operation, and Conv1 represents the ordinary convolution operation with a convolution kernel of 1×1.

[0028] For further optimization, the new loss function EAPIOU introduces width and height error penalty terms to optimize the matching penalty mechanism; the expression of the EAPIOU loss function is:

[0029] (5);

[0030] In the above formula, B p and B g Represent the areas of the predicted box and the true box respectively; d Represents the Euclidean distance between the center coordinates of the real box and the predicted box; cIndicates the diagonal distance between the minimum enclosing rectangle of the real box and the predicted box; w g and h g are the width and height of the real frame respectively; w p and h p They are the width and height of the prediction box respectively; λ is a hyperparameter used to control the impact of the aspect ratio penalty; γ is a global balance parameter; w c and h c Represents the width and height of the minimum enclosing rectangle of the predicted box and the real box respectively, Represents the width difference penalty term between the real box and the predicted box, Represents the penalty term for the height difference between the real box and the predicted box, λv represents the angle penalty term, ρ Indicates the Euclidean distance between calculated elements, L represents the loss function Loss, L IOU represents the IOU loss function, L CEIOU Represents the CEIOU loss function.

[0031] The new loss function EAPIO described in the present invention introduces width and height error penalty terms, optimizes the matching penalty mechanism, and can effectively improve detection accuracy and efficiency.

[0032] A circuit board appearance defect visual recognition system based on an improved YOLO algorithm, comprising:

[0033] Image acquisition module: used to acquire the appearance image of the circuit board;

[0034] A data preprocessing module, used for marking and preprocessing the appearance defects in the collected circuit board image, wherein the preprocessing includes data enhancement and normalization operations;

[0035] The model building module is used to improve the YOLOv8n algorithm and build an improved YOLO algorithm model. The specific improvements include: building a DSXConv module based on the dynamic snake convolution Dsconv, using the DSXConv module to replace the ordinary convolution in C2f to form a C2f-DSM module; proposing a new loss function EAPIOU, which enhances the position, aspect ratio and angle matching penalty mechanism;

[0036] Model training module, using the preprocessed data set to train the improved YOLO algorithm model;

[0037] The defect detection module is used to input the circuit board image to be detected into the trained improved YOLO algorithm model and output the location and category information of the defects.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. Improve detection accuracy: This invention enhances the model's ability to identify tiny defects and complex shape defects by constructing the DSXConv module and introducing a new loss function EAPIOU, thereby improving detection accuracy. After experimental verification, the improved algorithm's mAP0.5 reached 99%, an increase of 2.3 percentage points compared to the original model.

[0040] 2. Improve detection efficiency: The improved algorithm of the present invention can quickly process the circuit board image while ensuring the detection accuracy, thereby improving the detection speed and meeting the efficient detection needs in actual production.

[0041] 3. Enhance the adaptability of the model: The present invention introduces a multi-dimensional collaborative attention module MCA, so that the detection model can better utilize multi-scale features and background information, thereby enhancing the adaptability of the model to defects of different scales and types. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of the visual recognition method for circuit board appearance defects based on the improved YOLO algorithm;

[0043] Figure 2 This is a schematic diagram of the overall structure of the improved YOLO algorithm model;

[0044] Figure 3 It is the C2f-DSM module diagram;

[0045] Figure 4 It is a structural diagram of the DSXConv module;

[0046] Figure 5 This is the flow effect diagram of circuit board appearance defect detection;

[0047] Figure 6 This is a comparison chart of the experimental results, which compares the differences in indicators such as detection accuracy and recall rate before and after the improvement. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] like Figure 1 As shown, a visual recognition method for circuit board appearance defects based on an improved YOLO algorithm includes the following steps:

[0050] Step S1: Data preparation: Collect an image dataset containing circuit board appearance defects, annotate and preprocess the dataset, and the preprocessing includes image enhancement and normalization operations.

[0051] Step S2: Model construction: Improve the YOLOv8n algorithm and build an improved YOLO algorithm model, such as Figure 2 As shown in the figure, the connection relationship and data flow of each module are shown. Specific improvements include:

[0052] Step S2.1: Construct a DSXConv convolution module based on the dynamic snake convolution Dsconv, and use the DSXConv convolution module to replace the ordinary convolution module in the backbone network C2f to form a C2f-DSM module, as shown in Figure 3 shown.

[0053] like Figure 4 As shown in the figure, the DSXConv convolution module is constructed. Specifically, first, snake convolution with kernel sizes of 3×3, 5×5, 7×7, and 9×9 is used to extract tiny defects in the circuit board. That is, the input x After convolution with 4 different convolution kernels, feature extraction is performed to obtain the feature map x m1 、x m2 、x m3 and x m4 , the specific formula is:

[0054] (1);

[0055] Secondly, the features extracted by snake convolution are introduced into the MCA module for processing to generate attention maps. xc 1. xc 2. xc 3 and xc 4. The specific formula is:

[0056] (2);

[0057] Then, the Concat operation is used in the connection layer to merge the features of all branches and obtain the input quantity Y. The specific formula is:

[0058] (3);

[0059] Finally, the processed input Y is compared with the original input information x Add them together and use 1×1 convolution to fuse the information to get the output feature of the DSXConv convolution module.

[0060] (4);

[0061] in, x Indicates the feature quantity at the time of input, DsConv i Indicates that the convolution kernel size is i × i Convolution operation, i is 3, 5, 7 and 9; Concat represents the channel fusion operation, and Conv1 represents the ordinary convolution operation with a convolution kernel of 1×1.

[0062] Step S2.2: Propose a new loss function EAPIOU, which enhances the position, aspect ratio and angle matching penalty mechanism. The expression of the EAPIOU loss function is:

[0063] (5);

[0064] In the above formula, B p and B g Represent the areas of the predicted box and the true box respectively; d Represents the Euclidean distance between the center coordinates of the real box and the predicted box; c Indicates the diagonal distance between the minimum enclosing rectangle of the real box and the predicted box; w g and h g are the width and height of the real frame respectively; w p and h p They are the width and height of the prediction box respectively; λ is a hyperparameter used to control the impact of the aspect ratio penalty; γ is a global balance parameter; w c and h c Represents the width and height of the minimum enclosing rectangle of the predicted box and the real box respectively, Represents the width difference penalty term between the real box and the predicted box, Represents the penalty term for the height difference between the real box and the predicted box, λv represents the angle penalty term, ρ Indicates the Euclidean distance between calculated elements, L represents the loss function Loss, L IOU represents the IOU loss function, L CEIOU Represents the CEIOU loss function.

[0065] Step S3: Model training: Use the preprocessed dataset to train the improved YOLO algorithm model.

[0066] Step S4: Defect detection: Input the circuit board image to be detected into the trained improved YOLO algorithm model for detection, and output the location and category information of the defects.

[0067] 1) Build the experimental environment

[0068] The operating system is Ubuntu 20.04.4, the CPU model is Inter(R) i9-13900K, the GPU model is NVIDIA RTX A6000, the deep learning framework is Pytorch 1.11.0, the editing language is Python 3.8, the CUDA version is 11.3.0, and the optimizer is Adam. The experimental setting is 200 epochs and 8 training batches.

[0069] 2) Dataset preparation

[0070] A dataset of 1,386 synthetic circuit board images containing six types of defects released by the Human-Computer Interaction Open Laboratory of Peking University was used. The specific defects are missing holes, rat-tooth holes, short circuits, open circuits, branches and miscellaneous copper. After data expansion, a total of 10,668 images were obtained.

[0071] The dataset is divided into training set, validation set and test set in a ratio of 7:1:2. The images in the training set are annotated, and the annotation information includes the location and category of the defects. At the same time, the training set images are preprocessed, including image enhancement operations such as random cropping, flipping, rotation, brightness adjustment, and normalization, to improve the generalization ability of the model.

[0072] 3) Model training

[0073] Initialize the model: Take YOLOv8n as the basic model, optimize the model according to the above improvement plan, and initialize the model parameters.

[0074] Set training parameters: Set training parameters such as learning rate, batch size, number of training rounds, etc. The learning rate adopts a dynamic adjustment strategy, such as gradually reducing the learning rate as the number of training rounds increases.

[0075] Training process: Input the training set data into the model for training. In each training round, calculate the model loss value and use the optimization algorithm (such as Adam optimizer) to update the model parameters. At the same time, use the validation set to verify the model and monitor the model's performance indicators, such as precision, recall, mean average precision (mAP), etc., to prevent the model from overfitting.

[0076] 4) Model Evaluation

[0077] Use the test set to evaluate the trained model, calculate the model's precision, recall, mean average precision (mAP) and other evaluation indicators, and evaluate the model's detection performance. Compare the evaluation results with the original model to verify the superiority of the improved algorithm. Specifically:

[0078] Evaluation indicators: To evaluate the detection effect of the improved algorithm in circuit board appearance defects, precision (Precision, P), recall (Recall, R) and meanAveragePrecision (meanAveragePrecision, mAP) are selected as evaluation indicators. Among them, precision refers to the proportion of correct predictions in the predicted positive samples; recall refers to the proportion of correct predictions of positive samples in the samples; the higher the meanAveragePrecision mAP, the better the prediction effect of the algorithm model. mAP is obtained by integrating the PR curve in the average precision (AveragePrecision, AP), and is divided into mAP0.5 and mAP0.5-0.95. The former refers to the result when the IoU threshold is 0.5, and the latter refers to the average mAP value between the IoU threshold of 0.5 and 0.95.

[0079] Detection effect verification:

[0080] Experiments were conducted based on the image processing model of the improved YOLOv8 algorithm, and compared with the original YOLOv8n model. The detection results of circuit board appearance defects based on the improved YOLOv8 algorithm are as follows: Figure 5 As shown in the figure, the rectangular box in the figure represents the defect location, and the text above the rectangle represents the type of defect and the probability of determining it as the defect form. It can be seen from the figure that the improved algorithm shows higher detection accuracy.

[0081] The PR curve on the test set is as follows Figure 6 As shown, Figure 6 (a) is the PR curve corresponding to the algorithm before improvement, and (b) is the PR curve corresponding to the algorithm after improvement. The horizontal axis in the figure is the recall rate, the vertical axis is the precision rate, and the four indicators in the upper right corner are the average accuracy of the six defect detections of the circuit board and the mean average accuracy (mAP value) of the six defect detections.

[0082] In order to verify the separate and combined application effects of the C2f-DSM and EAPIOU modules in circuit board defect identification, an ablation test was conducted, and the results are shown in Table 1. "√" indicates that the method was used in the experiment, and "-" indicates that the method was not used in the experiment. Through ablation test comparison, compared with the YOLOv8 basic algorithm, the mAP0.5 of the two modules added in the present invention has a certain improvement, the C2f-DSM module has increased by 1.4 percentage points, and the EAPIOU module has increased by 0.9 percentage points. When the two are used together, the mAP0.5 reaches 99%, and the mAP0.5 before the improvement is 96.7%, which is an increase of 2.3 percentage points, proving the superiority of the improved algorithm.

[0083] Table 1 Ablation experiment results

[0084]

[0085] Based on the improved YOLO algorithm, the detection data of the six defects including missing holes, rat-tooth holes, broken circuits, open circuits, branch lines and miscellaneous copper are shown in Table 2.

[0086] Table 2 Detection data of each defect

[0087]

[0088] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A visual recognition method for circuit board appearance defects based on an improved YOLO algorithm, characterized in that: The following steps are involved: Step S1: Data preparation: collecting an image dataset containing circuit board appearance defects, annotating and preprocessing the dataset, wherein the preprocessing includes image enhancement and normalization operations; Step S2: Model construction: Improve the YOLOv8n algorithm as the basic algorithm to build an improved YOLO algorithm model. The specific improvements include: Step S2.1: construct a DSXConv convolution module based on the dynamic snake convolution Dsconv, and use the DSXConv convolution module to replace the ordinary convolution module in the backbone network C2f to form a C2f-DSM module; Step S2.2: Propose a new loss function EAPIOU, wherein the loss function EAPIOU enhances the position, aspect ratio and angle matching penalty mechanism; The expression of the EAPIOU loss function is: （1）； In the above formula, B p and B g Represent the areas of the predicted box and the true box respectively; d Represents the Euclidean distance between the center coordinates of the real box and the predicted box; c Indicates the diagonal distance between the minimum enclosing rectangle of the real box and the predicted box; w g and h g are the width and height of the real frame respectively; w p and h p They are the width and height of the prediction box respectively; λ is a hyperparameter used to control the impact of the aspect ratio penalty; γ is a global balance parameter; w c and h c Represents the width and height of the minimum enclosing rectangle of the predicted box and the real box respectively, Represents the width difference penalty term between the real box and the predicted box, Represents the penalty term for the height difference between the real box and the predicted box, λ v represents the angle penalty term, ρ Indicates the Euclidean distance between calculated elements, L represents the loss function Loss, L IOU represents the IOU loss function, L CEIOU represents the CEIOU loss function; Step S3: Model training: Use the preprocessed data set to train the improved YOLO algorithm model; Step S4: Defect detection: Input the circuit board image to be detected into the trained improved YOLO algorithm model for detection, and output the location and category information of the defects in the circuit board image.

2. The circuit board appearance defect visual recognition method based on the improved YOLO algorithm according to claim 1 is characterized in that: In step S2.1, the process of constructing the DSXConv convolution module includes: Step S2.1.1: Using snake convolutions of different sizes to extract tiny defect features in the circuit board image; Step S2.1.2: Introduce a multi-dimensional collaborative attention module MCA; the multi-dimensional collaborative attention module MCA effectively utilizes multi-scale features and backgrounds in the backbone network through multi-scale feature extraction, feature fusion and cross calculation; Step S2.1.2: The extracted micro-defect features are fused with the MCA module to finally obtain the DSXConv convolution module.

3. The circuit board appearance defect visual recognition method based on the improved YOLO algorithm according to claim 2 is characterized in that: In step S2.1, first, snake convolution with convolution kernel size of 3×3, 5×5, 7×7, and 9×9 is used to extract tiny defects in the circuit board, that is, the input feature quantity x After convolution with 4 different convolution kernels, feature extraction is performed to obtain the feature map x m1 、x m2 、x m3 and x m4 , the specific formula is: （2）； Secondly, the features extracted by snake convolution are introduced into the MCA module for processing to generate attention maps. xc 1. xc 2. xc 3 and xc 4. The specific formula is: （3）； Then, the Concat operation is used in the connection layer to merge the features of all branches and obtain the input quantity Y. The specific formula is: （4）； Finally, the processed input quantity Y is compared with the original input feature quantity x Add and use 1×1 convolution to fuse information and get the output feature quantity: （5）； in, x Indicates the feature quantity at the time of input; DsConv i Indicates that the convolution kernel size is i × i Convolution operation, i is 3, 5, 7 and 9; Concat represents the channel fusion operation; Conv1 represents the ordinary convolution operation with a convolution kernel of 1×1.

4. A circuit board appearance defect visual recognition system based on an improved YOLO algorithm, characterized in that: include: Image acquisition module: used to acquire the appearance image of the circuit board; A data preprocessing module, used to mark and preprocess the appearance defects in the collected circuit board image, wherein the preprocessing includes data enhancement and normalization operations; The model building module is used to improve the YOLOv8n algorithm and build an improved YOLO algorithm model. The specific improvements include: building a DSXConv module based on the dynamic snake convolution Dsconv, using the DSXConv module to replace the ordinary convolution in C2f to form a C2f-DSM module; proposing a new loss function EAPIOU, which enhances the position, aspect ratio and angle matching penalty mechanism; the expression of the EAPIOU loss function is: （6）； In the above formula, B p and B g Represent the areas of the predicted box and the true box respectively; d Represents the Euclidean distance between the center coordinates of the real box and the predicted box; c Indicates the diagonal distance between the minimum enclosing rectangle of the real box and the predicted box; w g and h g are the width and height of the real frame respectively; w p and h p They are the width and height of the prediction box respectively; λ is a hyperparameter used to control the impact of the aspect ratio penalty; γ is a global balance parameter; w c and h c Represents the width and height of the minimum enclosing rectangle of the predicted box and the real box respectively, Represents the width difference penalty term between the real box and the predicted box, Represents the penalty term for the height difference between the real box and the predicted box, λ v represents the angle penalty term, ρ Indicates the Euclidean distance between calculated elements, L represents the loss function Loss, L IOU represents the IOU loss function, L CEIOU represents the CEIOU loss function; Model training module, using the preprocessed data set to train the improved YOLO algorithm model; The defect detection module is used to input the circuit board image to be detected into the trained improved YOLO algorithm model and output the location and category information of the defects.

Images