PCB defect detection method, system, device and storage medium

Abstract

The present application discloses a PCB defect detection method, system, device and storage medium, which belongs to the field of defect detection technology. The method comprises: obtaining a PCB data set to preprocess the image of the obtained PCB data set; constructing a YOLO-v10n network model, and modifying the SPPF module in the YOLO-v10n network model to an EASPF module; constructing a feature fusion network BiFPN, and passing the extracted features into the feature fusion network BiFPN; adding an adaptive MBAM attention mechanism module between the backbone network and the feature fusion network BiFPN; inputting the PCB data set into the improved YOLO-v10n network model for pre-training to obtain the optimal weight; detecting the image to be detected according to the optimal weight, evaluating the detection result, and realizing the detection of PCB defects; by constructing and improving the YOLO-v10n network model, the accuracy of PCB defect detection is improved and the missed detection rate is reduced.

Description

Technical Field

[0001] The present application belongs to the field of defect detection, and specifically relates to a PCB defect detection method, system, device and storage medium. Background Art

[0002] As the foundation of the electronic information industry, the printed circuit board (PCB) industry is widely used in industrial manufacturing, communications, medical and aerospace fields. However, the manufacturing process of PCB is complicated and requires more than a dozen complex processes. Each process has an extremely important impact on the yield rate of the later products. Therefore, testing them before leaving the factory is the key to ensure the normal sales of products. Traditional manual testing methods are time-consuming and labor-intensive, with high labor costs. The accuracy of testing is easily affected by personal subjective consciousness and working hours, and may damage the surface of the product. Moreover, as the structure of PCB becomes more and more complex, it is no longer practical to use human eyes to judge.

[0003] At present, traditional image processing methods are greatly affected by light sources and have a large number of false detections and missed detections of small target defects. The deep learning-based PCB defect detection method has been widely used due to its excellent detection performance, end-to-end detection process and low system cost. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a PCB defect detection method, system, device and storage medium, which can solve the technical problems of low PCB defect detection accuracy and high missed detection rate in the prior art.

[0005] In order to solve the above technical problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a PCB defect detection method, the method comprising:

[0007] Acquire a PCB data set to preprocess the acquired image of the PCB data set;

[0008] Constructing a YOLO-v10n network model, and modifying the SPPF module of the backbone network in the YOLO-v10n network model to an improved EASPF module;

[0009] Constructing a feature fusion network BiFPN, and transferring the features extracted by the backbone network into the feature fusion network BiFPN;

[0010] An adaptive MBAM attention mechanism module is added between the backbone network and the feature fusion network BiFPN to adaptively process channel and spatial features simultaneously;

[0011] Input the PCB data set into the improved YOLO-v10n network model for pre-training, take the pre-training weight with the smallest training loss value for transfer learning, combine the PCB data set and the pre-training weight for training, and obtain the optimal weight;

[0012] The image to be inspected is inspected according to the optimal weight, and the inspection result is evaluated to realize the inspection of PCB defects.

[0013] As an optional implementation of the first aspect of the present application, the EASPF module is a feature extraction module, and the EASPF module includes a convolution block, an attention mechanism module, a fully connected layer and a dilated convolution module.

[0014] According to the above optional implementation, the convolution block is used to preliminarily extract local features in the image, and through multi-layer convolution operations, more abstract and complex feature information can be gradually captured; by introducing the attention mechanism, the model can automatically learn and focus on the more critical parts of the image for defect detection, thereby suppressing unimportant background information and improving the accuracy and efficiency of feature extraction; the fully connected layer further integrates and maps the extracted features to provide richer and more effective feature representations for subsequent decision-making and classification; the dilated convolution module can capture a wider range of spatial context information without increasing the amount of calculation by increasing the receptive field of the convolution kernel, which is particularly important for identifying complex and tiny PCB defects; the combination of the attention mechanism module and the dilated convolution module enables the model to better cope with PCB defect detection tasks under complex environmental conditions such as illumination changes and noise interference, thereby improving the robustness and adaptability of the model.

[0015] As an optional implementation of the first aspect of the present application, the feature fusion network BiFPN is a bidirectional weighted feature pyramid fusion architecture, consisting of multiple BiFPN blocks and multiple convolution blocks; the BiFPN block can effectively fuse multiple layers of feature information to improve detection accuracy.

[0016] According to the above optional implementation, the BiFPN block can effectively fuse feature information from different levels through bidirectional cross-scale connections; this fusion not only considers the top-down up-sampling features, but also combines the bottom-up down-sampling features, thereby realizing efficient integration of multi-scale features; by stacking multiple BiFPN blocks, the degree of feature fusion can be further deepened, so that the final feature map contains rich contextual information and fine detail features, which is crucial to improving the accuracy of PCB defect detection; because BiFPN can make full use of multi-scale feature information, the model has higher sensitivity in detecting defects of different sizes and shapes. sensitivity and accuracy; through fine feature fusion, the model can better distinguish defects and backgrounds, reduce false detections and missed detections, and thus improve the overall detection accuracy; due to the design of the BiFPN network structure, the model can better adapt to PCB defect detection tasks under different data sets and scenarios; by learning the association and weight distribution between features of different scales during the training process, the model can automatically adjust its attention to different features, thereby enhancing the generalization ability of new data and unknown defect types; in addition, since BiFPN adopts a weighted fusion method, it can make adaptive adjustments according to the importance of the features, thereby avoiding redundant calculations and improving computational efficiency.

[0017] As an optional implementation of the first aspect of the present application, the adaptive MBAM attention mechanism includes three branches: (C, H, W), (H, C, W) and (W, H, C), which are respectively denoted as , and ; The operation process of the adaptive MBAM attention mechanism is specifically as follows:

[0018] Regarding the , and The three branches perform attention production processing respectively and obtain corresponding attention weights respectively. , and ;

[0019] By the attention weight , and For the corresponding , and Perform weight multiplication to obtain the corresponding three branches with attention weights , and ;

[0020] Regarding the , and Perform adaptive weighted summation to obtain the adaptive MBAM attention mechanism:

[0021] ;

[0022] Among them, H, W, and C represent the height, width, and number of channels of the input feature map respectively. , and They represent the three branches of the adaptive MBAM attention mechanism. , and express , and The corresponding attention weights are, , and Respectively , and The corresponding three branches contain attention weights. Output represents the output value of the adaptive MBAM attention mechanism. , and express , and The corresponding feature weight value, the larger the weight value, the more obvious the corresponding feature.

[0023] According to the above optional implementation mode, by introducing branches of three different dimensions, the adaptive MBAM attention mechanism can more comprehensively capture the key information in the input feature map, including the correlation between channels, the importance of spatial positions and their combination; the introduction of attention weights enables the model to adaptively enhance important features while suppressing non-critical features, thereby improving the robustness and accuracy of feature representation; integrating the adaptive MBAM attention mechanism into the neural network can significantly improve the performance of the model in complex scenarios, such as image classification, target detection, semantic segmentation and other tasks; the design of the adaptive MBAM attention mechanism is flexible and scalable, and the number of branches, the way of attention production processing and the method of calculating weight values ​​can be adjusted according to the needs of specific tasks.

[0024] As an optional implementation of the first aspect of the present application, The specific formula for attention production processing and weight multiplication processing is as follows:

[0025] ;

[0026] Regarding the The specific formula for attention production processing and weight multiplication processing is as follows:

[0027] ;

[0028] Regarding the The specific formula for attention production processing and weight multiplication processing is as follows:

[0029] ;

[0030] Among them, GAP represents global average pooling, GMP represents global maximum pooling, MLP represents shared multi-layer perceptron, σ represents activation function, cat represents concatenation of two features, and Conv represents convolutional layer.

[0031] According to the above optional implementation, GAP (global average pooling) and GMP (global maximum pooling) are used to extract global context information of the input feature map; these two pooling operations capture the average activation and maximum activation in the feature map, respectively, thereby providing a global perspective on the importance of features; by using the outputs of GAP and GMP as the input of the subsequent MLP, the model can learn how to use the global context information to enhance the expressiveness of the features; the combination of MLP and activation function σ is used to perform nonlinear transformation on the global context information, which helps the model learn more complex and abstract feature representations; the cat operation fuses features from different pooling strategies (GAP and GMP), thereby combining the complementary information of average activation and maximum activation, further enriching the features. The concatenated features are further processed by another convolutional layer Conv, and the model can learn how to automatically adjust the weights according to the importance of the features. This adaptive weight adjustment mechanism enables the model to dynamically focus on the key parts of the input feature map, thereby improving the effectiveness and pertinence of the features. Incorporating the adaptive MBAM attention mechanism into the YOLO-v10n network model can significantly enhance the model's ability to identify PCB defects. By capturing global context information, performing nonlinear transformation and feature fusion, and adaptively adjusting feature weights, the model can more accurately locate and identify defective areas in PCB images. This not only improves the detection accuracy, but also enhances the model's robustness to complex environmental conditions such as illumination changes and noise interference.

[0032] As an optional implementation manner of the first aspect of the present application, the evaluation indicators in the evaluation of the detection results include average accuracy mean, precision, recall rate and frame rate;

[0033] The average precision mean is calculated by the following formula:

[0034] ;

[0035] The accuracy is calculated by the following formula:

[0036] ;

[0037] The recall rate is calculated by the following formula:

[0038] ;

[0039] The frame rate is calculated by the following formula:

[0040] ;

[0041] Among them, mAP represents the mean average precision, P represents the precision rate, R represents the recall rate, FPS represents the frame rate, i represents the i-th category, AP i is the average precision of the i-th category, N is the total number of categories, TP is the number of positive samples predicted as positive, FP is the number of negative samples predicted as positive, FN is the number of positive samples predicted as negative, Fn is the number of detected images, and T is the total time used to detect images.

[0042] According to the above optional implementation, mAP is one of the key indicators for measuring the performance of the target detection algorithm, which takes into account the average detection accuracy of all categories; by calculating the accuracy of each category and taking the average, mAP can comprehensively reflect the detection performance of the algorithm in different categories; a high mAP value means that the algorithm can more accurately identify the target object and perform stably in different categories; the precision rate reflects the proportion of instances predicted by the algorithm as positive samples that are truly positive samples; a high precision rate means that the results predicted by the algorithm are more reliable and reduce false positives; in PCB defect detection, a high precision rate can ensure that the detected defects are more accurate and reduce unnecessary rework and cost waste; the recall rate measures the proportion of positive samples that the algorithm can correctly identify to the total positive samples; a high recall rate means that the algorithm can detect more actual defects and reduce missed reports; in practical applications, a high recall rate is crucial to ensuring product quality and safety; FPS is an important indicator for measuring the processing speed of the algorithm, which indicates the number of images that the algorithm can process per second; a high FPS value means that the algorithm has a faster processing speed and can process input images in real time or near real time; in PCB defect detection, a high FPS value can ensure the efficient operation of the production line and reduce waiting time and cost.

[0043] As an optional implementation of the first aspect of the present application, the preprocessing of the acquired image of the PCB data set is specifically:

[0044] The produced PCB boards are imaged to obtain a PCB data set containing positive and negative samples, and defects in the PCB data set are annotated; the defects include short circuits, open circuits, gaps, burrs, leaks and excess copper. The PCB data set is expanded using data enhancement.

[0045] According to the above optional implementation methods, through detailed defect labeling, the algorithm can learn more accurate and specific defect features, thereby improving detection accuracy; data enhancement technology can increase the diversity of the data set, so that the algorithm can learn more feature changes during the training process, and further improve detection accuracy; data enhancement technology simulates different lighting, angles, noise and other conditions, so that the model can better adapt to various changes in the actual production environment; this helps to reduce the false positives and false negatives of the model under unknown or complex conditions, and improve the generalization ability of the model; imaging and data enhancement technology in the preprocessing process can optimize the performance of the algorithm, making it more efficient and stable when processing large-scale data sets; accurate defect labeling and rich data sets can accelerate the convergence speed of the algorithm and improve training efficiency.

[0046] In a second aspect, an embodiment of the present application provides a PCB defect detection system, the system comprising:

[0047] A first acquisition module is used to establish a YOLO-v10n network model and obtain a PCB data set required by the network model, wherein the PCB data set includes a training set, a validation set, and a test set;

[0048] A first processing module, configured to preprocess the PCB data set acquired by the first acquisition module to obtain the preprocessed PCB data set;

[0049] The first improvement module is used to modify the backbone network in the YOLO-v10n network model, modify the SPPF module in the backbone network to the improved EASPF module; construct a feature fusion network BiFPN, and pass the features extracted by the backbone network into the feature fusion network BiFPN;

[0050] The second improvement module is used to add an adaptive MBAM attention mechanism module between the backbone network and the feature fusion network BiFPN to adaptively process channel and spatial features at the same time;

[0051] A first training module pre-trains the YOLO-v10n network model improved by the first modification module and the second modification module through the PCB data set pre-processed by the first processing module;

[0052] The second training module performs transfer learning on the pre-trained weights with the smallest training loss value in the first training module; combines the pre-processed PCB data set with the pre-trained weights for training to obtain the optimal weights;

[0053] The first evaluation module is used to detect the image to be detected according to the optimal weight and evaluate the detection result.

[0054] According to a PCB defect detection system of the second aspect of the present invention, a first acquisition module establishes a YOLO-v10n network model and acquires a PCB data set (including a training set, a validation set and a test set), providing a basic network architecture and data resources for the system, and ensuring the feasibility of subsequent processing and analysis; the first processing module preprocesses the PCB data set, improves data quality, reduces noise and redundant information, and helps to improve the training effect and detection accuracy of the network model; the first improvement module modifies the backbone network of the YOLO-v10n network model, and replaces the SPPF module with the improved EASPF module; constructs a feature fusion network BiFPN, strengthens the feature extraction capability of the network model, and improves the sensitivity and recognition accuracy of PCB defects; the feature fusion network BiFPN further fuses feature information at different levels and improves the generalization ability of the model. The second improvement module adds an adaptive MBAM attention mechanism module between the backbone network and the feature fusion network BiFPN. By adaptively processing the feature information of channels and spaces, the network model's ability to capture PCB defect features is improved, and the detection accuracy and robustness are further improved. The first training module uses the preprocessed PCB data set to pre-train the improved YOLO-v10n network model, so that the network model can initially learn the characteristics and laws of PCB defects, providing a good foundation for subsequent transfer learning. The second training module performs transfer learning on the pre-trained weights, combines the preprocessed PCB data set for training, and obtains the optimal weights. By taking advantage of the transfer learning, the training process of the network model is accelerated and the training efficiency is improved. At the same time, through continuous iteration and optimization, the optimal model parameters are obtained, which further improves the detection accuracy and stability. The first evaluation module detects the image to be detected according to the optimal weights and evaluates the detection results. It can objectively and accurately evaluate the performance of the network model, providing a basis for subsequent model optimization and improvement. At the same time, it also provides a reliable guarantee for PCB defect detection in practical applications.

[0055] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0056] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0057] In the embodiment of the present application, by modifying the backbone network SPPF module in the YOLO-v10n network model to an improved EASPF module, the context information of a larger area can be captured, the feature extraction capability is enhanced, and the model can more accurately capture the detail features in the PCB image, thereby improving the accuracy of defect detection; compared with the traditional PAN architecture, the introduction of the feature fusion network BiFPN can effectively fuse feature information at different levels, improve the detection accuracy, especially in the detection of small targets, and has a lower computational complexity, which can speed up the detection speed while maintaining high accuracy; the addition of the adaptive MBAM attention mechanism module effectively focuses on more useful position information in the shallow network, and can better focus on semantic information in the deep network; the model can simultaneously adaptively process channel and spatial features, enhance the adaptability of the model to different PCB defect types, and improve the generalization ability of detection; the pre-training and transfer learning methods are adopted, the improved YOLO-v10n network model is first pre-trained using the PCB data set, and then the pre-training weight with the smallest training loss value is taken for transfer learning. This method can accelerate the training process of the model, reduce the risk of overfitting, and improve the stability and performance of the model; YOLO-v10n itself is a real-time target detection algorithm with a fast detection speed. Through improvement and optimization, this method further improves the detection efficiency while ensuring the detection accuracy, making PCB defect detection faster and more efficient; this method realizes the automatic detection of PCB defects by constructing and improving the YOLO-v10n network model; this can not only reduce the cost and error of manual detection, but also improve the consistency and reliability of detection, providing strong support for quality control in the PCB manufacturing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is a flow chart of a PCB defect detection method provided by some embodiments of the present application;

[0059] Figure 2 is a schematic diagram of the structure of an improved EASPF module in a PCB defect detection method provided by some embodiments of the present application;

[0060] Figure 3 This is a schematic diagram of the structure of an adaptive MBAM attention mechanism module in a PCB defect detection method provided by some embodiments of the present application

[0061] Figure 4It is a schematic diagram of the structure of an improved YOLO-v10n model in a PCB defect detection method provided in some embodiments of the present application. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0063] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0064] The following, in conjunction with the accompanying drawings, describes in detail the PCB defect detection method, system, device and storage medium provided in the embodiments of the present application through specific embodiments and their application scenarios.

[0065] Example

[0066] A PCB defect detection method comprises the following steps:

[0067] S100: Acquire a PCB data set to preprocess an image of the acquired PCB data set;

[0068] Furthermore, the images of the acquired PCB dataset are preprocessed, specifically:

[0069] The produced PCB boards are imaged to obtain a PCB dataset containing positive and negative samples, and the defects in the PCB dataset are annotated; defects include short circuits, open circuits, gaps, burrs, leaks, and excess copper. The PCB dataset is expanded using data enhancement.

[0070] It should be understood that, based on the above-mentioned feature definitions, first, the produced PCB boards are imaged with high quality, and high-resolution cameras or scanners are often used to capture images of the PCB boards; after imaging, the images are classified according to positive and negative samples (i.e., PCB images containing defects and those not containing defects) to construct an initial PCB data set; each image in the data set is annotated with detailed defects, and the annotation work is usually completed by professionals or with the help of semi-automatic annotation tools; the annotated defect types include common PCB defects such as short circuits, open circuits, gaps, burrs, leaks and excess copper; the annotation process needs to ensure that each defect is accurately marked, and record information such as the location, size and category of the defect; in order to improve the generalization and robustness of the model, the data set is expanded using data enhancement; data enhancement techniques include rotation, scaling, translation, flipping, color change, noise addition, etc. These technologies can simulate various changes that PCB images may encounter in the real world; through data enhancement, more diverse training samples can be generated to help the model learn richer feature representations; after completing annotation and data enhancement, the dataset is divided into training set, validation set and test set; the training set is used for the model training process, the validation set is used to adjust the model's hyperparameters and monitor the training process, and the test set is used to evaluate the final performance of the model; the quality of the preprocessed dataset is evaluated to ensure the accuracy of annotation, the effectiveness of data enhancement and the rationality of dataset division; the preprocessing effect can be evaluated through visualization tools or statistical indicators, such as annotation consistency, image differences before and after data enhancement, etc.

[0071] It should be noted that through detailed defect labeling, the algorithm can learn more accurate and specific defect features, thereby improving detection accuracy; data enhancement technology can increase the diversity of the data set, so that the algorithm can learn more feature changes during the training process, and further improve detection accuracy; data enhancement technology simulates different lighting, angles, noise and other conditions, so that the model can better adapt to various changes in the actual production environment; this helps to reduce false positives and false negatives of the model under unknown or complex conditions and improve the generalization ability of the model; imaging and data enhancement technology in the preprocessing process can optimize the performance of the algorithm, making it more efficient and stable when processing large-scale data sets; accurate defect labeling and rich data sets can accelerate the convergence of the algorithm and improve training efficiency.

[0072] S200: Build a YOLO-v10n network model, and modify the SPPF module of the backbone network in the YOLO-v10n network model to an improved EASPF module;

[0073] Furthermore, the EASPF module is a feature extraction module, and the EASPF module includes a convolution block, an attention mechanism module, a fully connected layer, and a dilated convolution module;

[0074] It should be understood that, based on the above-mentioned feature definitions, the convolution block is used to preliminarily extract local features in the image. Through multi-layer convolution operations, more abstract and complex feature information can be gradually captured. By introducing the attention mechanism, the model can automatically learn and focus on the more critical parts of the image for defect detection, thereby suppressing unimportant background information and improving the accuracy and efficiency of feature extraction. The fully connected layer further integrates and maps the extracted features to provide richer and more effective feature representations for subsequent decision-making and classification. The dilated convolution module increases the receptive field of the convolution kernel and can capture a wider range of spatial context information without increasing the amount of computation, which is particularly important for identifying complex and tiny PCB defects. The combination of the attention mechanism module and the dilated convolution module enables the model to better cope with PCB defect detection tasks under complex environmental conditions such as illumination changes and noise interference, thereby improving the robustness and adaptability of the model.

[0075] S300: construct a feature fusion network BiFPN, and pass the features extracted by the backbone network into the feature fusion network BiFPN;

[0076] Furthermore, the feature fusion network BiFPN is a bidirectional weighted feature pyramid fusion architecture, which consists of multiple BiFPN blocks and multiple convolution blocks; the BiFPN block can effectively fuse multiple layers of feature information to improve detection accuracy;

[0077] It should be understood that, based on the above feature definitions, BiFPN is composed of multiple BiFPN blocks, each of which can effectively fuse feature information from different levels; this fusion is bidirectional, that is, the upper-level features can transmit information to the lower level, and the lower-level features can also transmit information to the upper level, thereby realizing cross-level feature interaction; between BiFPN blocks, convolution blocks are usually inserted to further extract and integrate features; these convolution blocks can help the model better capture and represent the detailed information in the image; BiFPN uses a weighted fusion method to fuse features at different levels; this weighted method can automatically adjust the weights according to the importance of the features, thereby ensuring that key features receive more attention during the fusion process; the features extracted by the backbone network (including the improved EASPF module) are passed to the feature fusion network BiFPN, and are processed by multiple BiFPN blocks and convolution blocks to form the final feature representation.

[0078] It should be noted that the BiFPN block can effectively fuse feature information from different levels through bidirectional cross-scale connections; this fusion not only considers the top-down upsampling features, but also combines the bottom-up downsampling features, thereby achieving efficient integration of multi-scale features; by stacking multiple BiFPN blocks, the degree of feature fusion can be further deepened, so that the final feature map contains rich contextual information and fine detail features, which is crucial to improving the accuracy of PCB defect detection; because BiFPN can make full use of multi-scale feature information, the model has higher sensitivity in detecting defects of different sizes and shapes and accuracy; through fine feature fusion, the model can better distinguish defects and backgrounds, reduce false detections and missed detections, and thus improve the overall detection accuracy; due to the design of the BiFPN network structure, the model can better adapt to PCB defect detection tasks under different data sets and scenarios; by learning the association and weight distribution between features of different scales during the training process, the model can automatically adjust its attention to different features, thereby enhancing the generalization ability of new data and unknown defect types; in addition, since BiFPN adopts a weighted fusion method, it can make adaptive adjustments according to the importance of the features, thereby avoiding redundant calculations and improving computational efficiency.

[0079] S400: An adaptive MBAM attention mechanism module is added between the backbone network and the feature fusion network BiFPN to adaptively process channel and spatial features at the same time;

[0080] Furthermore, the adaptive MBAM attention mechanism includes three branches: (C, H, W), (H, C, W) and (W, H, C), which are respectively denoted as , and ; The operation process of the adaptive MBAM attention mechanism is as follows:

[0081] right , and The three branches perform attention production processing respectively and obtain corresponding attention weights respectively. , and ;

[0082] By attention weight , and For the corresponding , and Perform weight multiplication to obtain the corresponding three branches with attention weights , and ;

[0083] right , and Perform adaptive weighted summation to obtain the adaptive MBAM attention mechanism:

[0084] ;

[0085] Among them, H, W, and C represent the height, width, and number of channels of the input feature map respectively. , and They represent the three branches of the adaptive MBAM attention mechanism. , and express , and The corresponding attention weights are, , and Respectively , and The corresponding three branches contain attention weights. Output represents the output value of the adaptive MBAM attention mechanism. , and express , and The corresponding feature weight value, the larger the weight value, the more obvious the corresponding feature.

[0086] It should be understood that, based on the above-mentioned feature definition, the branch (C, H, W): Processing channel dimension features and generating processing functions through attention , extract the correlation information between channels; branch (H, C, W): Processing high-dimensional features, also through the attention generation processing function , extract feature information in the height direction; branch (W,H,C): Processing width dimension features, using attention generation processing function To extract the feature information in the width direction; for each branch ( , , ), firstly generate the corresponding attention weights through a specific attention production process , and , these weights reflect the importance of different positions or channels in each branch; then, the generated attention weights are multiplied by the original branch features to obtain three branch features containing attention weights , and , this step aims to enhance important features while suppressing non-critical features; finally, the three branch features containing attention weights are adaptively weighted and summed; the weight value , and They are obtained through learning, and they reflect the importance of different branch features in the final output; the sum result is the output value Output of the adaptive MBAM attention mechanism.

[0087] It should be noted that by introducing branches in three different dimensions, the adaptive MBAM attention mechanism can more comprehensively capture the key information in the input feature map, including the correlation between channels, the importance of spatial positions and their combination; the introduction of attention weights enables the model to adaptively enhance important features while suppressing non-critical features, thereby improving the robustness and accuracy of feature representation; integrating the adaptive MBAM attention mechanism into the neural network can significantly improve the performance of the model in complex scenarios, such as image classification, target detection, semantic segmentation and other tasks; the design of the adaptive MBAM attention mechanism is flexible and scalable, and the number of branches, the way of attention production and processing, and the calculation method of weight values ​​can be adjusted according to the needs of specific tasks.

[0088] Furthermore, The specific formula for attention production processing and weight multiplication processing is as follows:

[0089] ;

[0090] right The specific formula for attention production processing and weight multiplication processing is as follows:

[0091] ;

[0092] right The specific formula for attention production processing and weight multiplication processing is as follows:

[0093] ;

[0094] Among them, GAP represents global average pooling, GMP represents global maximum pooling, MLP represents shared multi-layer perceptron, σ represents activation function, cat represents concatenation of two features, and Conv represents convolutional layer.

[0095] It should be understood that, based on the above-mentioned feature definition, first of all, the branch Apply global average pooling (GAP) and global maximum pooling (GMP) to obtain two pooled feature maps; then, the two pooled feature maps are processed by a shared multi-layer perceptron (MLP, simplified as a convolutional layer) to obtain two processed feature vectors; then, the two feature vectors are added and activated by an activation function (σ, such as ReLU) to obtain the attention weight ; The attention weight With the original branch feature Multiply them to get the branch features containing attention weights ; For branches and , the process and branches of attention production processing and weight multiplication processing Similar, but with some subtle differences; for branches and Apply global average pooling (GAP) and global maximum pooling (GMP) to obtain two pooled feature maps; then, concatenate (cat) the two pooled feature maps in the channel dimension to obtain a concatenated feature map; then, process the concatenated feature map through a convolutional layer (Conv) and activate it through an activation function (σ, such as ReLU) to obtain the attention weight and ; The attention weight and With the original branch feature and Multiply them to get the branch features containing attention weights and ; Through the above formula, we can clearly understand the specific operation process of attention production and weight multiplication of each branch in the adaptive MBAM attention mechanism. These operations work together on the input feature map, enabling it to adaptively enhance important features while suppressing non-critical features, thereby improving the performance of the model.

[0096] It should be noted that GAP (global average pooling) and GMP (global maximum pooling) are used to extract global context information of the input feature map; these two pooling operations capture the average activation and maximum activation in the feature map, respectively, thereby providing a global perspective on the importance of features; by using the output of GAP and GMP as the input of the subsequent MLP, the model can learn how to use global context information to enhance the expressiveness of features; the combination of MLP and activation function σ is used to perform nonlinear transformation on global context information, which helps the model learn more complex and abstract feature representations; the cat operation fuses features from different pooling strategies (GAP and GMP), thereby combining the complementary information of average activation and maximum activation, further enriching the diversity of features. The concatenated features are further processed by another convolutional layer Conv, and the model can learn how to automatically adjust the weights according to the importance of the features. This adaptive weight adjustment mechanism enables the model to dynamically focus on the key parts of the input feature map, thereby improving the effectiveness and pertinence of the features. Incorporating the adaptive MBAM attention mechanism into the YOLO-v10n network model can significantly enhance the model's ability to identify PCB defects. By capturing global context information, performing nonlinear transformation and feature fusion, and adaptively adjusting feature weights, the model can more accurately locate and identify defective areas in PCB images. This not only improves the detection accuracy, but also enhances the model's robustness to complex environmental conditions such as illumination changes and noise interference.

[0097] S500: Input the PCB data set into the improved YOLO-v10n network model for pre-training, take the pre-training weight with the smallest training loss value for transfer learning, combine the PCB data set and the pre-training weight for training, and obtain the optimal weight;

[0098] S600: Detect the image to be detected according to the optimal weight, evaluate the detection result, and realize the detection of PCB defects;

[0099] Furthermore, the evaluation indicators in evaluating the detection results include average precision mean, precision, recall rate and frame rate;

[0100] The mean average precision is calculated by the following formula:

[0101] ;

[0102] The accuracy is calculated using the following formula:

[0103] ;

[0104] The recall rate is calculated by the following formula:

[0105] ;

[0106] The frame rate is calculated using the following formula:

[0107] ;

[0108] Among them, mAP represents the mean average precision, P represents the precision rate, R represents the recall rate, FPS represents the frame rate, i represents the i-th category, AP i is the average precision of the i-th category, N is the total number of categories, TP is the number of positive samples predicted to be positive, FP is the number of negative samples predicted to be positive, FN is the number of positive samples predicted to be negative, Fn is the number of detected images, and T is the total time used to detect images.

[0109] It should be understood that, based on the above-mentioned feature definitions, mAP is one of the key indicators for measuring the performance of target detection algorithms, which takes into account the average detection accuracy of all categories; by calculating the accuracy of each category and taking the average, mAP can fully reflect the detection performance of the algorithm in different categories; a high mAP value means that the algorithm can identify the target object more accurately and perform stably in different categories; the precision rate reflects the proportion of instances predicted by the algorithm as positive samples that are truly positive samples; a high precision rate means that the results predicted by the algorithm are more reliable and reduce false positives; in PCB defect detection, a high precision rate can ensure that the detected defects are more accurate, reducing unnecessary rework and cost waste; the recall rate measures the proportion of positive samples that the algorithm can correctly identify to the total positive samples; a high recall rate means that the algorithm can detect more actual defects and reduce missed reports; in practical applications, a high recall rate is crucial to ensuring product quality and safety; FPS is an important indicator for measuring the processing speed of the algorithm, which indicates the number of images that the algorithm can process per second; a high FPS value means that the algorithm has a faster processing speed and can process input images in real time or near real time; in PCB defect detection, a high FPS value can ensure the efficient operation of the production line and reduce waiting time and cost.

[0110] According to a PCB defect detection method provided by this embodiment, first, a PCB data set is obtained, which contains a large number of PCB images, and these images are used for model training and testing; the obtained PCB data set images are preprocessed; a YOLO-v10n network model is constructed, which is a target detection model for detecting defects in PCB images; the backbone network in the YOLO-v10n network model is modified, and the SPPF module is replaced with an improved EASPF module, and this improvement is intended to improve the efficiency and accuracy of feature extraction; a feature fusion network BiFPN is constructed, and the network is used to fuse different levels of features from the backbone network to enhance the model's adaptability to complex backgrounds and target diversity; an adaptive MBAM attention mechanism module is added between the backbone network and the feature fusion network BiFPN, and this module can adaptively process both the main network and the main network. The features of the track and space are used to improve the model's attention to key features and detection accuracy; secondly, the PCB dataset is input into the improved YOLO-v10n network model for pre-training. This step is to preliminarily adjust the model parameters so that it can adapt to the PCB defect detection task; during the pre-training process, the pre-trained weights with the smallest training loss value are selected for transfer learning; transfer learning uses the knowledge of the pre-trained model and combines it with the new PCB dataset for training to accelerate the training process and improve model performance; by combining the PCB dataset and the pre-trained weights for training, the optimal weights are obtained, and these weights enable the model to achieve the best performance in the PCB defect detection task; the trained model (i.e., the model with the optimal weights) is used to detect the image to be detected, and the detection results are evaluated, including defect location, classification, and confidence assessment, so as to realize the detection of PCB defects.

[0111] By modifying the backbone network SPPF module in the YOLO-v10n network model to the improved EASPF module, the context information of a larger area can be captured, the feature extraction capability is enhanced, and the model can more accurately capture the detailed features in the PCB image, thereby improving the accuracy of defect detection; compared with the traditional PAN architecture, the introduction of the feature fusion network BiFPN can effectively fuse feature information at different levels and improve the detection accuracy, especially in the detection of small targets, and has lower computational complexity, which can speed up the detection speed while maintaining high accuracy; the addition of the adaptive MBAM attention mechanism module effectively The more useful position information in the shallow network can be better focused on semantic information in the deep network; the model can adaptively process channel and spatial features at the same time, enhance the adaptability of the model to different PCB defect types, and improve the generalization ability of detection; the pre-training and transfer learning methods are adopted, and the improved YOLO-v10n network model is first pre-trained using the PCB data set, and then the pre-training weight with the smallest training loss value is taken for transfer learning; this method can accelerate the training process of the model, while reducing the risk of overfitting and improving the stability and performance of the model; YOLO-v10n itself, as a real-time target detection algorithm, has a faster detection speed. Through improvement and optimization, this method further improves the detection efficiency on the basis of ensuring detection accuracy, making PCB defect detection faster and more efficient; this method realizes the automatic detection of PCB defects by constructing and improving the YOLO-v10n network model; this can not only reduce the cost and error of manual detection, but also improve the consistency and reliability of detection, and provide strong support for quality control in the PCB manufacturing process.

[0112] It should be noted that the PCB defect detection method provided in the embodiment of the present application can be executed by a PCB defect detection system, or a control module in the PCB defect detection system for executing and loading a PCB defect detection method. In the embodiment of the present application, a PCB defect detection method provided in the embodiment of the present application is described by taking a PCB defect detection system executing and loading a PCB defect detection method as an example.

[0113] A PCB defect detection system, comprising:

[0114] The first acquisition module is used to establish a YOLO-v10n network model and obtain the PCB data set required by the network model. The PCB data set includes a training set, a validation set, and a test set;

[0115] A first processing module, used for preprocessing the PCB data set acquired by the first acquisition module to obtain a preprocessed PCB data set;

[0116] The first improvement module is used to modify the backbone network in the YOLO-v10n network model, modify the SPPF module in the backbone network to the improved EASPF module; construct a feature fusion network BiFPN, and pass the features extracted by the backbone network into the feature fusion network BiFPN;

[0117] The second improved module is used to add an adaptive MBAM attention mechanism module between the backbone network and the feature fusion network BiFPN to adaptively process channel and spatial features at the same time;

[0118] The first training module pre-trains the YOLO-v10n network model improved by the first modification module and the second modification module through the PCB data set pre-processed by the first processing module;

[0119] The second training module performs transfer learning on the pre-trained weights with the smallest training loss value in the first training module; the pre-processed PCB data set and the pre-trained weights are combined for training to obtain the optimal weights;

[0120] The first evaluation module is used to detect the image to be detected according to the optimal weight and evaluate the detection result.

[0121] A PCB defect detection system in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), etc., which is not specifically limited in the embodiment of the present application.

[0122] A PCB defect detection system in an embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0123] The PCB defect detection system provided in the embodiment of the present application can realize Figure 1 to Figure 2 The various processes of implementing a PCB defect detection method in the method embodiment will not be described again here to avoid repetition.

[0124] According to a PCB defect detection system of this embodiment, the first acquisition module establishes a YOLO-v10n network model and acquires a PCB data set (including a training set, a validation set and a test set), providing a basic network architecture and data resources for the system, ensuring the feasibility of subsequent processing and analysis; the first processing module preprocesses the PCB data set, improves data quality, reduces noise and redundant information, and helps to improve the training effect and detection accuracy of the network model; the first improvement module modifies the backbone network of the YOLO-v10n network model, and replaces the SPPF module with the improved EASPF module; constructs a feature fusion network BiFPN, strengthens the feature extraction capability of the network model, and improves the sensitivity and recognition accuracy of PCB defects; the feature fusion network BiFPN further integrates feature information at different levels and improves the generalization ability of the model. The second improvement module adds an adaptive MBAM attention mechanism module between the backbone network and the feature fusion network BiFPN. By adaptively processing the feature information of channels and spaces, the network model's ability to capture PCB defect features is improved, and the detection accuracy and robustness are further improved. The first training module uses the preprocessed PCB data set to pre-train the improved YOLO-v10n network model, so that the network model can initially learn the characteristics and laws of PCB defects, providing a good foundation for subsequent transfer learning. The second training module performs transfer learning on the pre-trained weights, combines the preprocessed PCB data set for training, and obtains the optimal weights. By taking advantage of the transfer learning, the training process of the network model is accelerated and the training efficiency is improved. At the same time, through continuous iteration and optimization, the optimal model parameters are obtained, which further improves the detection accuracy and stability. The first evaluation module detects the image to be detected according to the optimal weights and evaluates the detection results. It can objectively and accurately evaluate the performance of the network model, providing a basis for subsequent model optimization and improvement. At the same time, it also provides a reliable guarantee for PCB defect detection in practical applications.

[0125] Optionally, an embodiment of the present application also provides an electronic device, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, each process of the above-mentioned PCB defect detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0126] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned PCB defect detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0127] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0128] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0129] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0130] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A PCB defect detection method, characterized in that: The method comprises: Acquire a PCB data set to preprocess the acquired image of the PCB data set; Constructing a YOLO-v10n network model, and modifying the SPPF module of the backbone network in the YOLO-v10n network model to an improved EASPF module; Constructing a feature fusion network BiFPN, and transferring the features extracted by the backbone network into the feature fusion network BiFPN; An adaptive MBAM attention mechanism module is added between the backbone network and the feature fusion network BiFPN to adaptively process channel and spatial features simultaneously; Input the PCB data set into the improved YOLO-v10n network model for pre-training, take the pre-training weight with the smallest training loss value for transfer learning, combine the PCB data set and the pre-training weight for training, and obtain the optimal weight; The image to be inspected is inspected according to the optimal weight, and the inspection result is evaluated to realize the inspection of PCB defects; The EASPF module includes a convolution block, an attention mechanism module, a fully connected layer and a dilated convolution module; The adaptive MBAM attention mechanism includes three branches: (C, H, W), (H, C, W) and (W, H, C), which are respectively denoted as , and ; The operation process of the adaptive MBAM attention mechanism is specifically as follows: Regarding the , and The three branches perform attention production processing respectively and obtain corresponding attention weights respectively. , and ; By the attention weight , and For the corresponding , and Perform weight multiplication to obtain the corresponding three branches with attention weights , and ; Regarding the , and Perform adaptive weighted summation to obtain the adaptive MBAM attention mechanism: ； Among them, H, W, and C represent the height, width, and number of channels of the input feature map respectively. , and They represent the three branches of the adaptive MBAM attention mechanism. , and express , and The corresponding attention weights are, , and Respectively , and The corresponding three branches contain attention weights. Output represents the output value of the adaptive MBAM attention mechanism. , and express , and The corresponding feature weight value, the larger the weight value, the more obvious the corresponding feature.

2. A PCB defect detection method according to claim 1, characterized in that: The feature fusion network BiFPN is a bidirectional weighted feature pyramid fusion architecture, which is composed of multiple BiFPN blocks and multiple convolution blocks; the BiFPN block can effectively fuse multiple layers of feature information to improve detection accuracy.

3. A PCB defect detection method according to claim 1, characterized in that: Regarding the The specific formula for attention production processing and weight multiplication processing is as follows: ； Regarding the The specific formula for attention production processing and weight multiplication processing is as follows: ； Regarding the The specific formula for attention production processing and weight multiplication processing is as follows: ； Among them, GAP represents global average pooling, GMP represents global maximum pooling, MLP represents shared multi-layer perceptron, σ represents activation function, cat represents concatenation of two features, and Conv represents convolutional layer.

4. A PCB defect detection method according to claim 1, characterized in that: The evaluation indicators for evaluating the detection results include average accuracy, precision, recall rate and frame rate; The average precision mean is calculated by the following formula: ； The accuracy is calculated by the following formula: ； The recall rate is calculated by the following formula: ； The frame rate is calculated by the following formula: ； Among them, mAP represents the mean average precision, P represents the precision rate, R represents the recall rate, FPS represents the frame rate, i represents the i-th category, AP i is the average precision of the i-th category, N is the total number of categories, TP is the number of positive samples predicted as positive, FP is the number of negative samples predicted as positive, FN is the number of positive samples predicted as negative, Fn is the number of detected images, and T is the total time used to detect images.

5. A PCB defect detection method according to claim 1, characterized in that: The preprocessing of the acquired image of the PCB data set is specifically as follows: The produced PCB boards are imaged to obtain a PCB data set containing positive and negative samples, and defects in the PCB data set are annotated; the defects include short circuits, open circuits, gaps, burrs, leaks and excess copper. The PCB data set is expanded using data enhancement.

6. A PCB defect detection system capable of implementing a PCB defect detection method according to any one of claims 1 to 5, characterized in that: The system comprises: A first acquisition module is used to establish a YOLO-v10n network model and obtain a PCB data set required by the network model, wherein the PCB data set includes a training set, a validation set, and a test set; A first processing module, configured to preprocess the PCB data set acquired by the first acquisition module to obtain the preprocessed PCB data set; The first improvement module is used to modify the backbone network in the YOLO-v10n network model, modify the SPPF module in the backbone network to the improved EASPF module; construct a feature fusion network BiFPN, and pass the features extracted by the backbone network into the feature fusion network BiFPN; The second improved module is used to add an adaptive MBAM attention mechanism module between the backbone network and the feature fusion network BiFPN to adaptively process channel and spatial features at the same time; A first training module pre-trains the YOLO-v10n network model improved by the first improvement module and the second improvement module through the PCB data set pre-processed by the first processing module; The second training module performs transfer learning on the pre-trained weights with the smallest training loss value in the first training module; combines the pre-processed PCB data set with the pre-trained weights for training to obtain the optimal weights; The first evaluation module is used to detect the image to be detected according to the optimal weight and evaluate the detection result.

7. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of a PCB defect detection method as described in claims 1-5 are implemented.

8. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of a PCB defect detection method as described in claims 1-5 are implemented.

Images