PCB flaw detection system and method based on deep learning

Abstract

The invention discloses a PCB flaw detection system and method based on deep learning, and relates to the technical field of monitoring analysis, and the system comprises an image collection module which is used for carrying out the image collection of a PCB, and determining the image data corresponding to the PCB; the preprocessing module is used for preprocessing the image data corresponding to the PCB; the feature extraction module is used for inputting the preprocessed image data into the YOLOv8 backbone network and further extracting feature data corresponding to the image data; the feature fusion module is used for fusing the extracted feature data through a cross-scale feature fusion pyramid; the classification module is used for classifying the fused feature data by using a lightweight classification head, and judging whether flaws exist or not and the types of the flaws; and the detection module is used for carrying out real-time flaw detection on the input PCB image data based on the classification result and outputting a detection result. The method has the effect of improving the detection efficiency.

Description

Technical Field

[0001] This application relates to the field of monitoring and analysis technologies, and particularly to a PCB board defect detection system and method based on deep learning. Background Art

[0002] The PCB board is an important component in electronic devices, and the detection of its surface defects is crucial for ensuring the quality and reliability of electronic devices. In fact, most of the defects in the PCB board only account for a very small part of it, which poses a great challenge to the PCB board defect detection work. By detecting and identifying the defects on the PCB board, defects can be discovered and repaired in a timely manner, ensuring that the product quality meets the standards and reducing the defective rate.

[0003] In the related art, traditional PCB board defect detection methods have problems of low detection accuracy and high false detection rate when facing complex backgrounds and small target defects, thereby reducing the efficiency of PCB board defect detection and there is room for improvement. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, this application provides a PCB board defect detection system and method based on deep learning.

[0005] In the first aspect, this application provides a PCB board defect detection system based on deep learning, including:

[0006] An image acquisition module, configured to acquire an image of the PCB board and confirm the corresponding image data of the PCB board;

[0007] A preprocessing module, configured to preprocess the image data corresponding to the PCB board;

[0008] A feature extraction module, configured to input the preprocessed image data into the YOLOv8 backbone network, and then extract the corresponding feature data of the image data;

[0009] A feature fusion module, configured to fuse the extracted feature data through a cross-scale feature fusion pyramid;

[0010] A classification module, configured to classify the fused feature data using a lightweight classification head and determine whether there are defects and the defect categories;

[0011] A detection module, configured to perform real-time defect detection on the input PCB board image data based on the classification result and output the detection result.

[0012] Preferably, the preprocessing module includes a normalization processing unit and an HSV space enhancement unit;

[0013] The normalization processing unit is used to perform a normalization operation on the image data corresponding to the PCB board to map the pixel values into the range of [0, 1];

[0014] The HSV space enhancement unit is used to perform grayscale processing on the image data, and perform histogram equalization on the brightness corresponding to the image data to enhance the saturation.

[0015] Preferably, the YOLOv8 backbone network is embedded with an adaptive channel-spatial dual attention module.

[0016] Preferably, the feature data corresponding to the image data includes shallow features, middle features, and deep features;

[0017] The shallow features are used to extract the edge texture features of the PCB board pads;

[0018] The middle features are used to capture the morphological features of the copper foil traces on the PCB board;

[0019] The deep features are used to characterize the three-dimensional morphological features of the solder joints on the PCB board.

[0020] Preferably, the adaptive channel-spatial dual attention module includes a channel attention unit and a spatial attention unit;

[0021] The channel attention unit adopts an improved SE structure, and generates a channel weight map through dual-path feature compression of global average pooling and max pooling;

[0022] The spatial attention unit introduces a deformable convolutional layer to dynamically perceive the spatial distribution of the defect areas.

[0023] Preferably, the cross-scale feature fusion pyramid adopts a bidirectional feature transfer structure, and the bidirectional feature transfer structure includes a bottom-up path and a top-down path; the bottom-up path is used to fuse the feature data; the top-down path is used to adopt bilinear interpolation sampling;

[0024] And after feature fusion, a learnable weight is introduced to control the feature fusion ratio, and then a professional detection head is added to strengthen the texture features of the micro defects.

[0025] Preferably, the lightweight classification head is composed of depthwise separable convolution and Shuffle channels.

[0026] Preferably, it further includes a training optimization module including a loss function innovation unit and a training strategy optimization unit;

[0027] The training strategy optimization unit includes a curriculum learning mechanism unit and a dynamic enhancement unit;

[0028] The curriculum learning mechanism unit is used to train in stages to improve the model convergence speed;

[0029] The dynamic enhancement unit is used for intelligent data enhancement based on defect density, thereby improving the defect detection rate.

[0030] In a second aspect, the present application provides a method for detecting PCB board defects based on deep learning, including the following steps:

[0031] Collect images of the PCB board and confirm the corresponding image data of the PCB board;

[0032] Preprocess the image data corresponding to the PCB board;

[0033] Input the preprocessed image data into the YOLOv8 backbone network to extract the corresponding feature data of the image data;

[0034] Fuse the extracted feature data through a cross-scale feature fusion pyramid;

[0035] Use a lightweight classification head to classify the fused feature data and determine whether there are defects and the defect categories;

[0036] Perform real-time defect detection on the input PCB board image data based on the classification result and output the detection result.

[0037] In a third aspect, the present application provides a computer-readable storage medium storing instructions, which when run on a computer, cause the computer to execute a system for detecting PCB board defects based on deep learning as described in any one of the above.

[0038] In summary, the present application includes the following beneficial technical effects:

[0039] The present application provides a system for detecting PCB board defects based on deep learning. By accurately obtaining the PCB board image data and optimizing the image data based on the preprocessing module, noise and interference are eliminated, the data quality is improved, and the key image features are extracted using the YOLOv8 backbone network, efficiently and accurately capturing defect-related information. The cross-scale feature fusion pyramid is used to integrate feature data at different levels, enhancing the model's ability to detect subtle defects. A lightweight classification head is used for rapid classification, reducing the computational complexity, while accurately determining the defect categories. Combining the classification result, real-time defect detection is achieved and the result is output, meeting the high requirements for speed and accuracy in industrial production. The image processing, feature extraction, classification, and detection are organically combined, improving the efficiency and reliability of PCB board defect detection, while reducing the model complexity, providing an efficient solution for intelligent detection. Description of the Drawings

[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0041] Figure 1 It is a schematic diagram of a system for PCB board defect detection based on deep learning in an embodiment of this application.

[0042] Figure 2 It is a flowchart of a method for PCB board defect detection based on deep learning in an embodiment of this application. Detailed implementation manners

[0043] The following will further elaborate on this application in conjunction with the attached Figure 1-2 for a more detailed description.

[0044] Embodiment 1

[0045] An embodiment of this application discloses a PCB board defect detection system based on deep learning.

[0046] Referring to Figure 1 , a PCB board defect detection system based on deep learning includes:

[0047] An image acquisition module, which is used to acquire images of the PCB board and confirm the corresponding image data of the PCB board;

[0048] A preprocessing module, which is used to preprocess the image data corresponding to the PCB board;

[0049] A feature extraction module, which is used to input the preprocessed image data into the YOLOv8 backbone network to extract the corresponding feature data of the image data;

[0050] A feature fusion module, which is used to fuse the extracted feature data through a cross-scale feature fusion pyramid;

[0051] A classification module, which is used to classify the fused feature data using a lightweight classification head and determine whether there are defects and the defect categories;

[0052] A detection module, which is used to perform real-time defect detection on the input PCB board image data based on the classification result and output the detection result.

[0053] By adopting the above technical solution, the image acquisition module ensures the accurate acquisition of PCB board image data, laying a foundation for subsequent processing. The preprocessing module optimizes the image data, eliminates noise and interference, and improves the data quality. The feature extraction module uses the YOLOv8 backbone network to extract key image features, efficiently and accurately capturing information related to defects. The feature fusion module integrates feature data at different levels through a cross-scale feature fusion pyramid, enhancing the model's ability to detect subtle defects. The classification module uses a lightweight classification head for rapid classification, reducing computational complexity while accurately determining the defect category. Finally, the detection module combines the classification results to achieve real-time defect detection and output the results, meeting the high requirements for speed and accuracy in industrial production. It organically combines image processing, feature extraction, classification, and detection, improving the efficiency and reliability of PCB board defect detection, while reducing the model complexity and providing an efficient solution for intelligent detection.

[0054] Further, the preprocessing module includes a normalization processing unit and an HSV space enhancement unit;

[0055] The normalization processing unit is used to perform a normalization operation on the image data corresponding to the PCB board to map the pixel values into the range of [0, 1];

[0056] The HSV space enhancement unit is used to perform grayscale processing on the image data and perform histogram equalization on the brightness corresponding to the image data to enhance the saturation.

[0057] Specifically, the preprocessing module optimizes the PCB board image data through the normalization processing unit and the HSV space enhancement unit, significantly improving the efficiency and accuracy of subsequent image analysis. The normalization processing unit maps the pixel values into the range of [0, 1], making the data distribution more uniform, reducing the amplitude difference between different images, avoiding the impact of excessive or too small values on model training and inference, and at the same time accelerating the convergence speed of the network. The HSV space enhancement unit simplifies the image information through grayscale processing, retains key features, and at the same time performs histogram equalization on the brightness, optimizing the contrast of the image and making the defect area more prominent. In addition, the saturation enhancement further improves the color expression ability of the image, making subtle defects more easily detectable in complex backgrounds. The overall preprocessing step effectively improves the image quality, enhances the sensitivity of the model to defect features, reduces noise interference, provides high-quality input data for the subsequent feature extraction and detection modules, and significantly improves the robustness and accuracy of PCB board defect detection.

[0058] Further, the YOLOv8 backbone network is embedded with an adaptive channel-spatial dual attention module.

[0059] It should be noted that the feature data corresponding to the image data includes shallow features, middle features, and deep features;

[0060] The shallow features are used to extract the texture features of the edges of the PCB pads;

[0061] The middle features are used to capture the morphological features of the copper foil traces on the PCB;

[0062] The deep features are used to characterize the three-dimensional morphological features of the solder joints on the PCB.

[0063] Specifically, in the PCB defect detection, the processing method of decomposing the image data into shallow features, middle features, and deep features greatly improves the accuracy and effectiveness of the detection. Among them, the shallow features focus on the texture features of the pad edges, capturing subtle edge changes, which helps to identify edge defects and irregularities. The middle features focus on the morphological features of the copper foil traces and can effectively detect problems such as trace breaks or short circuits. The deep features characterize the three-dimensional morphology of the solder joints. By capturing complex deep information, they can identify the three-dimensional structural defects of the solder joints, such as insufficient solder joints or deformations. Such a hierarchical feature extraction method enables the model to capture various types of defect information at different levels, improving the comprehensiveness and accuracy of the detection. The comprehensive utilization of the features of each layer ensures the detailed analysis of the PCB by the model, enhances the ability to identify complex defects, and provides strong technical support for high-precision automated detection.

[0064] It should be noted that the adaptive channel-spatial dual attention module includes a channel attention unit and a spatial attention unit;

[0065] The channel attention unit adopts an improved SE structure and generates a channel weight map through dual-path feature compression of global average pooling and max pooling;

[0066] The spatial attention unit introduces a deformable convolutional layer to dynamically perceive the spatial distribution of the defect areas.

[0067] Specifically, the adaptive channel-spatial dual attention module significantly improves the accuracy and robustness of PCB board defect detection by combining a channel attention unit and a spatial attention unit. The channel attention unit adopts an improved SE structure, generates a channel weight map through dual-path feature compression of global average pooling and max pooling, can effectively identify and emphasize key channel features, enhance the model's attention to important features, and suppress irrelevant or redundant information. The spatial attention unit introduces a deformable convolutional layer, which can dynamically perceive and capture the spatial distribution of defect areas and adapt to defect changes of different shapes and scales. By comprehensively utilizing channel and spatial information, the model is more flexible and accurate in dealing with complex backgrounds and diverse defects. By adaptively adjusting the feature focus points, the model can more accurately locate and identify defects during the detection process, improving the overall detection performance and providing strong support for efficient automated quality control.

[0068] It should be noted that the cross-scale feature fusion pyramid adopts a bidirectional feature transfer structure, and the bidirectional feature transfer structure includes a bottom-up path and a top-down path; the bottom-up path is used to fuse feature data; the top-down path is used for bilinear interpolation sampling;

[0069] And after feature fusion, a learnable weight is introduced to control the feature fusion ratio, and then a professional detection head is added to enhance the texture features of small defects.

[0070] It should be noted that the lightweight classification head is composed of depthwise separable convolution and Shuffle channels.

[0071] It should be noted that it also includes a training optimization module including a loss function innovation unit and a training strategy optimization unit;

[0072] The training strategy optimization unit includes a curriculum learning mechanism unit and a dynamic enhancement unit;

[0073] The curriculum learning mechanism unit is used to train in stages to improve the model convergence speed;

[0074] The dynamic enhancement unit is used for intelligent data augmentation based on defect density, thereby improving the defect detection rate.

[0075] Specifically, through the collaborative action of the loss function innovation unit and the training strategy optimization unit, the training optimization module significantly improves the training efficiency and detection performance of the PCB board defect detection model. The curriculum learning mechanism unit adopts a phased training strategy, starting from simple tasks and gradually transitioning to complex tasks, which can effectively reduce the difficulty in the initial stage of training, improve the convergence speed of the model, and reduce the risk of overfitting, enabling the model to learn features more stably at different training stages. The dynamic enhancement unit intelligently adjusts the data augmentation strategy according to the defect density, focusing on processing the defect areas, such as increasing sample diversity or generating more challenging defect scenarios, thereby improving the detection rate of rare or complex defects by the model. The innovative loss function design further optimizes the target guiding ability of the model, ensuring that the model pays more attention to the detection of defect areas during training. The overall training optimization module comprehensively improves the robustness, generalization ability, and detection accuracy of the model through phased learning and intelligent data augmentation, combined with an efficient loss function design, providing an efficient and reliable solution for PCB board defect detection.

[0076] Embodiment 2

[0077] This application embodiment also discloses a PCB board defect detection method based on deep learning.

[0078] Referring to Figure 2 , a PCB board defect detection method based on deep learning includes the following steps:

[0079] Collect images of the PCB board and confirm the corresponding image data of the PCB board;

[0080] Preprocess the image data corresponding to the PCB board;

[0081] Input the preprocessed image data into the YOLOv8 backbone network to extract the corresponding feature data of the image data;

[0082] Fuse the extracted feature data through a cross-scale feature fusion pyramid;

[0083] Use a lightweight classification head to classify the fused feature data and determine whether there are defects and the defect categories;

[0084] Perform real-time defect detection on the input PCB board image data based on the classification result and output the detection result.

[0085] The above content is only an example and illustration of the concept of the present invention. Those skilled in the art of this technology can make various modifications or supplements to the described specific embodiments or use similar methods for substitution, as long as they do not deviate from the concept of the invention, they should all fall within the protection scope of the present invention.

[0086] In the description of this specification, the descriptions referring to terms such as "one embodiment", "example", "specific example", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples.

[0087] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the present invention to only the specific embodiments. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described in this specification in order to better explain the principles and practical applications of the present invention, so that those skilled in the relevant technical fields can understand and utilize the present invention well.

Claims

1. A PCB defect detection system based on deep learning, characterized in that: include: An image acquisition module is used to acquire images of the PCB board and confirm the image data corresponding to the PCB board; A preprocessing module, used for preprocessing image data corresponding to the PCB board; The feature extraction module is used to input the preprocessed image data into the YOLOv8 backbone network to extract the feature data corresponding to the image data; A feature fusion module is used to fuse the extracted feature data through a cross-scale feature fusion pyramid; The classification module is used to classify the fused feature data using a lightweight classification head and determine whether there are defects and the defect category; The detection module is used to perform real-time defect detection on the input PCB board image data based on the classification results and output the detection results.

2. According to a deep learning-based PCB defect detection system according to claim 1, it is characterized in that: The preprocessing module includes a normalization processing unit and an HSV space enhancement unit; The normalization processing unit is used to perform a normalization operation on the image data corresponding to the PCB board so that the pixel value is mapped to the interval [0, 1]; The HSV space enhancement unit is used to perform grayscale processing on the image data, and to perform histogram equalization on the brightness corresponding to the image data to enhance the saturation.

3. According to a deep learning-based PCB defect detection system according to claim 1, it is characterized in that: The YOLOv8 backbone network is embedded with an adaptive channel-spatial dual attention module.

4. A PCB board defect detection system based on deep learning according to claim 1, characterized in that: The feature data corresponding to the image data includes shallow features, middle features and deep features; The shallow features are used to extract the edge texture features of the PCB pad; The middle layer features are used to capture the morphological features of the copper foil traces on the PCB board; The deep features are used to characterize the three-dimensional morphology of the solder joints on the PCB.

5. A PCB board defect detection system based on deep learning according to claim 3, characterized in that: The adaptive channel-space dual attention module includes a channel attention unit and a space attention unit; The channel attention unit adopts an improved SE structure and generates a channel weight map through dual-path feature compression of global average pooling and maximum pooling; The spatial attention unit introduces a deformable convolutional layer to dynamically perceive the spatial distribution of the defect area.

6. A PCB board defect detection system based on deep learning according to claim 5, characterized in that: The cross-scale feature fusion pyramid adopts a bidirectional feature transfer structure, which includes a bottom-up path and a top-down path; the bottom-up path is used to fuse feature data; the top-down path is used to adopt bilinear interpolation sampling; After feature fusion, learnable weights are introduced to control the feature fusion ratio, and then a professional detection head is added to enhance the texture features of tiny defects.

7. A PCB board defect detection system based on deep learning according to claim 1, characterized in that: The lightweight classification head consists of depthwise separable convolution and shuffle channel.

8. A PCB board defect detection system based on deep learning according to claim 1, characterized in that: It also includes a training optimization module including a loss function innovation unit and a training strategy optimization unit; The training strategy optimization unit includes a course learning mechanism unit and a dynamic enhancement unit; The course learning mechanism unit is used for staged training to improve the model convergence speed; The dynamic enhancement unit is used for intelligent data enhancement based on defect density, thereby improving the defect detection rate.

9. A PCB board defect detection method based on deep learning, applied to a PCB board defect detection system based on deep learning as described in any one of claims 1 to 8, characterized in that: The following steps are involved: Capture images of the PCB board and confirm the image data corresponding to the PCB board; Preprocess the image data corresponding to the PCB board; Input the preprocessed image data into the YOLOv8 backbone network to extract the feature data corresponding to the image data; The extracted feature data is fused through the cross-scale feature fusion pyramid; Use a lightweight classification head to classify the fused feature data and determine whether there are defects and the defect category; Based on the classification results, real-time defect detection is performed on the input PCB board image data, and the detection results are output.

10. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are executed on a computer, the computer executes a PCB board defect detection system based on deep learning as described in any one of claims 1 to 8.

Images