Printed circuit board defect detection method, device, equipment and medium

Abstract

The invention relates to a printed circuit board defect detection method, device and equipment and a medium, and the method comprises the steps: obtaining a to-be-detected printed circuit board image, inputting the to-be-detected printed circuit board image into a YOLO neural network model, and obtaining a printed circuit board defect detection result, the model comprising a backbone network, a neck part and a head part; the backbone network comprises a cross-stage partial network module and a cross-stage binary network module which are alternately arranged in sequence, a detected printed circuit board image is sequentially subjected to feature extraction and down-sampling operation of different scales for multiple times, and a high-dimensional feature map obtained by the cross-stage partial network module is output to the neck; sequentially performing up-sampling on the neck from the high-dimensional feature map with the highest dimension and fusing the neck with the high-dimensional feature map with the dimension lower than that of the neck, and outputting a plurality of multi-scale fused feature maps to the head; and the head performs detection and identification on the plurality of multi-scale fused feature maps, and outputs printed circuit board defect detection results of boundary box positions and types of defects.

Description

Technical Field

[0001] The present disclosure relates to the field of detection technology, and in particular to a printed circuit board defect detection method, device, equipment and medium. Background Art

[0002] Printed Circuit Board (PCB) is a key component in electronic devices and is widely used because of its intelligence, lightweight, high density and integration. Defects in PCBs may seriously damage the function and safety of electronic equipment, not only resulting in reduced production, but also posing safety risks. In related scenarios, PCB defect detection mainly relies on manual visual inspection and automated optical inspection (AOI). However, manual inspection is affected by human errors, and the performance of AOI is highly dependent on optical conditions, resulting in a high false detection rate. These limitations have led to reduced PCB inspection accuracy, increased costs, and reduced efficiency in complex inspection environments. Summary of the invention

[0003] The purpose of the present invention is to provide a printed circuit board defect detection method, device, equipment and medium, aiming to solve the technical problems of reduced PCB detection accuracy, increased cost and reduced efficiency in complex detection environments in related scenarios.

[0004] In order to achieve the above-mentioned purpose, a first aspect of an embodiment of the present disclosure provides a printed circuit board defect detection method, the method comprising: Acquire a printed circuit board image to be detected, input the printed circuit board image to be detected into a YOLO neural network model, and obtain a printed circuit board defect detection result, wherein the YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck; Wherein, the backbone network comprises a cross-stage partial network module and a cross-stage binary network module which are alternately arranged in sequence, and the backbone network sequentially performs multiple feature extraction and downsampling operations of different scales on the detected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each of the cross-stage partial network modules to the neck; The neck starts from the high-dimensional feature map with the highest dimension, sequentially performs upsampling and fusion with the high-dimensional feature map with lower dimension, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimension, and outputs a plurality of multi-scale fused feature maps to the head; The head detects and identifies the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the boundary box position and category of the defect marked.

[0005] In a possible implementation, the backbone network is further provided with a convolution module and a plurality of cross-stage binary network modules in sequence before the cross-stage partial network modules and the cross-stage binary network modules that are alternately provided in sequence; The backbone network performs convolution processing on the detected printed circuit board image through the convolution module, and performs convolution and normalization processing on the input respectively through multiple cross-stage binary network modules, and inputs the image after feature extraction into the first of the cross-stage binary network modules that are alternately arranged in sequence.

[0006] In a possible implementation, the backbone network is further provided with a spatial pyramid pooling module after the cross-stage partial network modules and the cross-stage binary network modules that are alternately provided in sequence; The backbone network performs pooling operations of different scales on the highest-dimensional high-dimensional feature map output by the last cross-stage partial network module through the spatial pyramid pooling module, and outputs the high-dimensional feature map after the pooling operation to the neck.

[0007] In a possible implementation, each of the cross-stage partial network modules of the backbone network performs feature enhancement on the input feature map once, performs multiple maximum pooling convolutions on the feature map after one feature enhancement, and then splices it with the feature map after another feature enhancement to obtain high-dimensional feature maps corresponding to different scales, wherein the number of residual blocks in each of the cross-stage partial network modules is different, so that the feature map after one feature enhancement is sequentially subjected to multiple maximum pooling convolutions for different numbers of times.

[0008] In a possible implementation, the neck starts from the high-dimensional feature map with the highest dimension, sequentially up-samples and fuses with the high-dimensional feature map with a lower dimension, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with a higher dimension, and outputs multiple multi-scale fused feature maps to the head, including: The neck starts from the high-dimensional feature map with the highest dimension and sequentially upsamples the high-dimensional feature map; Use the connection operation to connect the upsampled high-dimensional feature maps to feature maps of different scales to achieve multi-scale feature fusion; Residual connection is performed through the cross-stage partial network module set in the neck to enhance the multi-scale feature extraction capability, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with a higher dimension, and multiple multi-scale fused feature maps are output to the head.

[0009] In a possible implementation, the head detects and identifies the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the bounding box position and category of the defect marked, including: The head performs convolution and residual processing on the multiple multi-scale fused feature maps respectively to obtain multiple processed multi-scale fused feature maps; The detection and recognition layer of the head performs defect marking of different sizes on the multiple processed multi-scale fused feature maps respectively to obtain the boundary box position and category of the defect; The head outputs the printed circuit board defect detection result with a bounding box position and category marked with the defect.

[0010] In a possible implementation, the loss function of the YOLO neural network model includes: positioning error, confidence error of containing an object, confidence error of not containing an object, and classification error; Among them, the loss function L YOLOv5 The discriminant formula is: ; Among them, λ coord is the positioning loss, which is used to measure the spatial position difference between the predicted bounding box and the actual bounding box; i and i The predicted center point coordinates that constitute the i-th bounding box; x i and i The coordinates of the real center point that constitutes the i-th bounding box; w i and h i are the predicted width and predicted height of the i-th bounding box, respectively. and are the true width and true height of the i-th bounding box, B is the number of bounding boxes, S is the number of feature grids, and λ noobj is the confidence loss, which is used to measure the confidence of whether there is a defect in the predicted bounding box; p i (c) is the prediction confidence of the i-th bounding box, which is used to indicate the probability that the i-th bounding box contains a defect; is the true confidence of the i-th bounding box, which is used to indicate whether the i-th bounding box contains defects; C i is the predicted category probability of the i-th bounding box, is the true category label of the i-th bounding box.

[0011] According to a second aspect of the embodiments of the present disclosure, there is provided a printed circuit board defect detection device, the device comprising: An acquisition module is configured to acquire an image of a printed circuit board to be inspected; An input module is configured to input the printed circuit board image to be detected into a YOLO neural network model to obtain a printed circuit board defect detection result, wherein the YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck; Wherein, the backbone network comprises a cross-stage partial network module and a cross-stage binary network module which are alternately arranged in sequence, and the backbone network sequentially performs multiple feature extraction and downsampling operations of different scales on the detected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each of the cross-stage partial network modules to the neck; The neck starts from the high-dimensional feature map with the highest dimension, sequentially performs upsampling and fusion with the high-dimensional feature map with lower dimension, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimension, and outputs a plurality of multi-scale fused feature maps to the head; The head detects and identifies the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the boundary box position and category of the defect marked.

[0012] According to a third aspect of the present disclosure, an electronic device is provided, including: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of any one of the methods in the first aspect.

[0013] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0014] The present invention provides a printed circuit board defect detection method, device, equipment and medium. Compared with the prior art, it has the following beneficial effects: The image of a printed circuit board to be inspected is obtained, and the image of the printed circuit board to be inspected is input into a YOLO neural network model to obtain a printed circuit board defect detection result. The YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck; wherein the backbone network includes a cross-stage partial network module and a cross-stage binary network module which are alternately arranged in sequence, and the backbone network sequentially performs multiple feature extraction and downsampling operations of different scales on the detected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each cross-stage partial network module to the neck; the learning ability of the model is improved through cross-stage partial connection (CSP) and residual connection, and more feature information is retained, thereby improving the accuracy of PCB defect detection.

[0015] Then, starting from the high-dimensional feature map with the highest dimension, the neck is upsampled and fused with high-dimensional feature maps with lower dimensions in sequence, until the high-dimensional feature map with the lowest dimension is fused with high-dimensional feature maps with higher dimensions, and multiple multi-scale fused feature maps are output to the head; through residual connection, the model's ability to extract multi-scale features is enhanced.

[0016] The head detects and identifies multiple multi-scale fused feature maps separately, and outputs the printed circuit board defect detection results with the bounding box position and category of the defect marked. Up- and down-sampling and feature fusion of feature maps at multiple scales ensure that the model can effectively cope with defect detection of different sizes and complexities, and the model can effectively improve efficiency and reduce labor costs, thereby solving the technical problems of reduced PCB detection accuracy, increased costs, and reduced efficiency in complex detection environments in related scenarios.

[0017] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1 The present invention is a flowchart of a printed circuit board defect detection method according to an embodiment of the specification.

[0019] Figure 2 It is a structural schematic diagram of a YOLO neural network model shown in an embodiment of the specification.

[0020] Figure 3 It is a block diagram of a printed circuit board defect detection device shown according to an embodiment of the specification.

[0021] Figure 4 It is a block diagram of another printed circuit board defect detection device shown according to an embodiment of the specification. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0024] The present invention provides a method for detecting defects in a printed circuit board. Figure 1 The following is a flow chart of a printed circuit board defect detection method according to an embodiment. The method comprises: In step S11, an image of a printed circuit board to be inspected is acquired.

[0025] In the present disclosure, images of the printed circuit board (PCB) to be inspected are acquired through an image acquisition device (such as a high-resolution camera or scanner). These images should be able to clearly show all the details on the PCB, including wires, components, solder joints, etc. After image acquisition, pre-processing operations such as denoising, grayscale, binarization, etc. are usually performed to improve image quality and reduce the complexity of subsequent processing.

[0026] As an example, suppose an industrial-grade high-definition camera is used, which is fixed at the end of the PCB production line to take non-contact photos of the PCB boards that have just come off the line. The pictures taken by the camera are transmitted to the computer via the USB or Ethernet interface. The software on the computer automatically receives these pictures and immediately performs grayscale and simple denoising processing to prepare for subsequent defect detection.

[0027] In step S12, the printed circuit board image to be detected is input into a YOLO neural network model to obtain a printed circuit board defect detection result, wherein the YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck; Among them, YOLO (You Only Look Once) is trained to identify defects in images, such as short circuits, open circuits, missing or misplaced components, etc. in PCB defect detection. The YOLO neural network model integrates multiple levels of residual connections in a single unit, which can extract feature information of different scales, thereby realizing PCB defect detection of various sizes and complexities. YOLO consists of three main parts: Backbone, Neck, and Head.

[0028] Wherein, the backbone network comprises a cross-stage partial network module and a cross-stage binary network module which are alternately arranged in sequence, and the backbone network sequentially performs multiple feature extraction and downsampling operations of different scales on the detected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each of the cross-stage partial network modules to the neck; See also Figure 2 As shown in the figure, Backbone is responsible for feature extraction. The input PCB image is processed by multiple convolution blocks and then outputs a high-level feature map. This part uses multi-level CSP (Cross Stage Partial) blocks and residual structures (ResX).

[0029] Among them, multi-scale features are extracted from the input image. By alternately using the cross-stage partial network module (CSP module) and the cross-stage binary network module (CBS module), the model can gradually reduce the spatial resolution of the image (downsampling) while increasing the number of channels of the feature map (i.e., feature depth), thereby capturing image information at different levels.

[0030] For example, the input image size is 640×640, and the features are extracted through the initial convolution block and CSP1_X block; then, the size of the feature map is gradually reduced and high-dimensional features are extracted through deeper convolution blocks (such as CBL, CSP1_X) and downsampling operations (P1 / 2, P2 / 4, etc.); Backbone passes feature maps of different scales to the Neck part of the next stage. CSP1_X and CSP2_X improve the learning ability of the model through cross-stage partial connections (CSP) and residual connections, and retain more feature information.

[0031] The neck starts from the high-dimensional feature map with the highest dimension, sequentially performs upsampling and fusion with the high-dimensional feature map with lower dimension, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimension, and outputs a plurality of multi-scale fused feature maps to the head; Among them, Neck combines feature maps of different scales through upsampling and feature fusion operations to generate a series of multi-scale fused feature maps. These feature maps contain both high-level semantic information (from deep feature maps) and detail information (from shallow feature maps), which are very suitable for multi-scale target detection and feature fusion and enhancement. This part uses the structure of the Feature Pyramid Network (FPN) to improve the effect of detecting small targets through upsampling and feature fusion. The key steps include: Upsample the feature maps of different layers in Backbone (such as 512×20×20, 256×40×40); use the Concat operation to connect the feature maps of different scales to achieve multi-scale feature fusion; the CSP2_X module enhances the multi-scale feature extraction capability of the network through residual connection; finally, pass the multi-scale fused feature map to the Head part for detection.

[0032] The head detects and identifies the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the boundary box position and category of the defect marked.

[0033] The Head performs independent detection and recognition of feature maps at each scale. Through the preset anchor boxes and classifiers, the model can predict the bounding box location and category of the defect for final detection and classification of the defect. This part receives the fused multi-scale feature map from the Neck and performs further processing to locate and classify the target. The specific operations include: The feature map obtained from Neck is further processed by the CSP1 module and then passed to the Detect layer; the Detect layer is responsible for target detection on feature maps of different scales and processing PCB defect targets of different sizes; finally, the model outputs the bounding box location and category information of the defect.

[0034] Among them, the residual block (ResX) is used to introduce short-circuit connections (shortcut=true) in the network to help alleviate the gradient vanishing problem and improve the training stability of the model. Multi-scale feature extraction performs up- and down-sampling and feature fusion through feature maps of multiple scales to ensure that the model can effectively cope with defect detection of different sizes and complexities.

[0035] For example, a preprocessed PCB image is fed into a trained YOLO model. The model first extracts multi-scale features from the image through the backbone network (a combination of multiple CSP and CBS modules) to generate a series of high-dimensional feature maps. Then, in the neck stage, these feature maps are integrated into multiple multi-scale fused feature maps through upsampling and feature fusion operations. Finally, in the head stage, each feature map is used to predict potential defects. For example, the model may detect a short circuit bounding box on a feature map and give a prediction result of its location and type as "short circuit". These prediction results are aggregated to form the final PCB defect detection report.

[0036] The above technical solution obtains the image of the printed circuit board to be detected, inputs the image of the printed circuit board to be detected into the YOLO neural network model, and obtains the printed circuit board defect detection result. The YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck; wherein the backbone network includes a cross-stage partial network module and a cross-stage binary network module that are alternately arranged in sequence, and the backbone network sequentially performs multiple feature extraction and downsampling operations of different scales on the detected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature map corresponding to different scales obtained by each cross-stage partial network module to the neck; the learning ability of the model is improved through cross-stage partial connection and residual connection, and more feature information is retained, thereby improving the accuracy of PCB defect detection.

[0037] Then, starting from the high-dimensional feature map with the highest dimension, the neck is upsampled and fused with high-dimensional feature maps with lower dimensions in sequence, until the high-dimensional feature map with the lowest dimension is fused with high-dimensional feature maps with higher dimensions, and multiple multi-scale fused feature maps are output to the head; through residual connection, the model's ability to extract multi-scale features is enhanced.

[0038] The head detects and identifies multiple multi-scale fused feature maps separately, and outputs the printed circuit board defect detection results with the bounding box position and category of the defect marked. Up- and down-sampling and feature fusion of feature maps at multiple scales ensure that the model can effectively cope with defect detection of different sizes and complexities, and the model can effectively improve efficiency and reduce labor costs, thereby solving the technical problems of reduced PCB detection accuracy, increased costs, and reduced efficiency in complex detection environments in related scenarios.

[0039] In one possible implementation, see Figure 2 As shown, the backbone network is also provided with a convolution module and a plurality of cross-stage binary network modules in sequence before the cross-stage partial network modules and the cross-stage binary network modules that are alternately provided in sequence; The backbone network performs convolution processing on the detected printed circuit board image through the convolution module, and performs convolution and normalization processing on the input respectively through multiple cross-stage binary network modules, and inputs the image after feature extraction into the first of the cross-stage binary network modules that are alternately arranged in sequence.

[0040] Among them, the convolution module: This module is usually located at the front end of the backbone network and is used to perform preliminary feature extraction on the input PCB image. The convolution layer locally perceives the image through a sliding window and introduces nonlinearity through an activation function (such as ReLU) to extract basic features such as edges and textures of the image.

[0041] Multiple cross-stage binary network modules (pre-): After the convolution module, multiple consecutive cross-stage binary network modules are set (which may be the same or slightly different from the CBS in the subsequent alternating modules), which further convolve and normalize the image. The normalization here may refer to batch normalization or other forms of normalization, which aims to speed up the training process and improve model stability and performance. Through multiple such modules, the features of the image are gradually abstracted and enhanced, laying a good foundation for subsequent feature extraction and downsampling operations.

[0042] After the above preprocessing, the backbone network begins to use the alternately set CSP module and CBS module to perform multi-scale feature extraction and downsampling. The operation of this stage has been explained in detail in the previous answer and will not be repeated here.

[0043] Example description: Assuming there is a PCB image, it is first fed into the convolution module. The convolution module extracts the basic features of the image, such as edges, corners, etc., through several convolution layers and ReLU activation functions. These feature maps are then fed into a series of preceding cross-stage binary network modules.

[0044] In each CSP module, the image first passes through a convolutional layer for feature extraction, and then passes through a normalization layer (such as batch normalization) for feature normalization to reduce internal covariate shift and improve training stability. After passing through multiple such modules, the features of the image are further abstracted and enhanced.

[0045] Subsequently, these enhanced feature maps are fed into the alternating CSP and CBS modules. The CSP module enhances feature extraction capabilities through its unique structure (such as residual connection and group convolution) while reducing the amount of calculation and parameters. The CBS module may focus on the processing or enhancement of binary features to meet the specific needs of PCB defect detection.

[0046] Ultimately, through this series of complex operations, the backbone network is able to generate information-rich feature maps at multiple scales, which will be sent to the neck and head for further processing and detection.

[0047] In one possible implementation, see Figure 2 As shown, the backbone network is further provided with a spatial pyramid pooling module after the cross-stage partial network modules and the cross-stage binary network modules that are alternately arranged in sequence; The backbone network performs pooling operations of different scales on the highest-dimensional high-dimensional feature map output by the last cross-stage partial network module through the spatial pyramid pooling module, and outputs the high-dimensional feature map after the pooling operation to the neck.

[0048] Among them, the spatial pyramid pooling module: The main function of this module is to perform pooling operations of different scales on the highest-dimensional high-dimensional feature map output by the backbone network. Different from the traditional single-scale pooling, the SSPF module adopts a multi-scale pooling strategy, which divides the feature map into multiple grids of different sizes (such as 1×1, 2×2, 4×4, etc.), and pools the features in each grid (usually maximum pooling or average pooling). This multi-scale pooling method can capture contextual information of different scales and enhance the robustness and generalization ability of the model.

[0049] Fusion and output of feature maps: After being processed by the SSPF module, the pooled feature maps of multiple scales will be further integrated or kept independent (depending on the specific design of the model) and then used as the input of the neck network. These multi-scale feature maps not only contain rich semantic information, but also have a certain scale invariance, which is helpful for subsequent detection tasks.

[0050] Example description: Assume that after a series of alternating CSP and CBS modules, the backbone network outputs a high-dimensional feature map with dimensions of H×W×C (where H, W, and C represent the height, width, and number of channels of the feature map, respectively). This feature map is then fed into the spatial pyramid pooling module.

[0051] In the SSPF module, the feature map is first divided into multiple grids of different sizes. For example, it can be divided into grids of three scales: 1×1, 2×2, and 4×4. For each scale of the grid, a maximum pooling operation is performed, that is, the maximum value of all feature values ​​in each grid is taken as the output of the grid. In this way, for the original high-dimensional feature map, the SSPF module outputs three pooled feature maps (or feature vectors if it is flattened) of different scales.

[0052] These pooled feature maps (or feature vectors) are then fed into the neck network. In the neck network, these feature maps may undergo upsampling, feature fusion, and other operations to generate multi-scale fused feature maps for final detection. These feature maps not only contain rich information of the original feature maps, but also introduce multi-scale contextual information through the SSPF module, thereby improving the model's ability to detect PCB defects.

[0053] It should be noted that although maximum pooling is used as an example here, in fact the SSPF module can also use other types of pooling operations (such as average pooling) or combine multiple pooling strategies to further improve the performance of the model.

[0054] In a possible implementation, each of the cross-stage partial network modules of the backbone network performs feature enhancement on the input feature map once, performs multiple maximum pooling convolutions on the feature map after one feature enhancement, and then splices it with the feature map after another feature enhancement to obtain high-dimensional feature maps corresponding to different scales, wherein the number of residual blocks in each of the cross-stage partial network modules is different, so that the feature map after one feature enhancement is sequentially subjected to multiple maximum pooling convolutions for different numbers of times.

[0055] Among them, feature enhancement: In each CSP module, the input feature map will first undergo one or more feature enhancement operations. These feature enhancement operations may include convolutional layers, activation functions (such as ReLU), normalization layers (such as batch normalization), etc., aiming to extract and enhance the expressiveness of feature maps.

[0056] Multi-scale feature extraction: After feature enhancement, the CSP module uses a special design to generate multi-scale feature maps. Specifically, it performs multiple max pooling operations on one of the feature enhancements (called the "main path" feature map) and concatenates a convolutional layer (or a series of convolutional layers) after each pooling to further extract and transform features. These feature maps that have been concatenated with different times of max pooling are then concatenated with another feature enhancement (called the "residual path" feature map).

[0057] Residual blocks and multi-scale splicing: It is worth noting that the number of residual blocks in each CSP module may be different. This design allows the model to learn features of different scales at different depths. During the splicing process, due to the difference in the number of residual blocks, the main path feature map will undergo different numbers of maximum pooling splicing operations to generate feature maps of multiple scales. These feature maps are then spliced ​​with the residual path feature map to form the final high-dimensional feature maps corresponding to different scales.

[0058] Example description: Assume there is a CSP module, which contains two residual blocks. In this module, the input feature map first undergoes a feature enhancement operation (such as a convolution layer + ReLU activation function + batch normalization layer). Then, the enhanced feature map is divided into two parts: one part enters the main path, and the other part enters the residual path.

[0059] In the main path, the feature map first undergoes a max pooling operation, and then a convolutional layer is concatenated for feature transformation. Next, the pooled and convolutional feature map undergoes a max pooling operation again, and a convolutional layer is concatenated again. Since this module contains two residual blocks, the main path feature map actually only undergoes two max pooling concatenation operations (but each concatenation may contain multiple convolutional layers).

[0060] At the same time, the feature map in the residual path also undergoes one or more feature enhancement operations (depending on the module design), but no pooling operation is performed to maintain its spatial resolution. Finally, the feature maps of the main path and the residual path are concatenated in the channel dimension to form a high-dimensional feature map corresponding to different scales output by the CSP module. This feature map is then fed into the next CSP module or CBS module for further feature extraction and conversion.

[0061] It should be noted that the above example is only used to illustrate one possible implementation method within the CSP module. In actual applications, the specific design of the CSP module may be adjusted according to factors such as task requirements, model complexity, and computing resources.

[0062] In a possible implementation, the neck starts from the high-dimensional feature map with the highest dimension, sequentially up-samples and fuses with the high-dimensional feature map with a lower dimension, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with a higher dimension, and outputs multiple multi-scale fused feature maps to the head, including: The neck starts from the high-dimensional feature map with the highest dimension and sequentially upsamples the high-dimensional feature map; The neck network starts with the highest-dimensional feature map and upsamples the feature maps in turn. The purpose of upsampling is to restore the lower-resolution feature map to a higher resolution so that it can be fused with the subsequent higher-dimensional feature map. The upsampling operation can be achieved through interpolation algorithms (such as bilinear interpolation, nearest neighbor interpolation) or transposed convolution (deconvolution).

[0063] Use the connection operation to connect the upsampled high-dimensional feature maps to feature maps of different scales to achieve multi-scale feature fusion; Among them, after upsampling, the neck network uses a connection operation (such as Concatenate) to connect the upsampled high-dimensional feature map with a high-dimensional feature map with a lower dimension than it (but properly upsampled to match the resolution). This connection method allows feature maps of different scales to be fused in the channel dimension, thereby making full use of contextual information of different scales.

[0064] In order to further enhance the ability to extract multi-scale features, a cross-stage partial network module (or a similar residual structure) can also be set in the neck network. These modules add the input feature map to the output feature map inside the module through residual connections, thereby retaining more original information and alleviating the gradient vanishing problem. The use of residual connections helps improve the overall performance of the model.

[0065] Residual connection is performed through the cross-stage partial network module set in the neck to enhance the multi-scale feature extraction capability, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with a higher dimension, and multiple multi-scale fused feature maps are output to the head.

[0066] After a series of upsampling, feature map connections, and possible residual connections, the neck network finally outputs multiple multi-scale fused feature maps to the head. These feature maps not only contain rich semantic information, but also enhance the feature expression ability through multi-scale fusion and residual connections.

[0067] Example description: Assume that the neck network receives three high-dimensional feature maps of different scales, which are recorded as Feature Map 1 (highest dimension), Feature Map 2, and Feature Map 3 (lowest dimension).

[0068] Upsampling operation: First, Feature Map 1 is upsampled so that its resolution matches Feature Map 2.

[0069] Feature map connection: Connect the upsampled Feature Map 1 with Feature Map 2 to form a fused feature map.

[0070] Residual connection (optional): If a cross-stage partial network module is set in the neck network, the fused feature map can be used as input to further enhance its feature expression ability through residual connection.

[0071] Repeat the operation: Next, the fused feature map obtained in the previous step is upsampled (if necessary) to match the resolution of Feature Map 3 and connected with Feature Map 3. Similarly, residual connections can also be applied in this step (if any in the design).

[0072] Output multi-scale fusion feature map: After the above operations, the neck network outputs multiple multi-scale fusion feature maps to the head. These feature maps will be used as input to the head network for the final detection or classification task.

[0073] It should be noted that the number of upsampling times, feature map connection methods, and use of residual connections in the above examples are determined according to the specific model design and task requirements. In practical applications, these parameters and structures can be adjusted as needed to achieve optimal performance.

[0074] In a possible implementation, the head detects and identifies the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the bounding box position and category of the defect marked, including: The head performs convolution and residual processing on the multiple multi-scale fused feature maps respectively to obtain multiple processed multi-scale fused feature maps; The head network first performs convolution on each multi-scale fused feature map to further extract and transform features. These convolutional layers are usually followed by activation functions (such as ReLU) and normalization layers (such as batch normalization) to enhance the expressiveness of features. In addition, in order to maintain the richness of features and avoid the gradient vanishing problem, the head network can also introduce residual connections between these convolutional layers. The residual connection adds the input feature map to the output feature map of the convolutional layer, ensuring that more original information can be retained during training.

[0075] The detection and recognition layer of the head performs defect marking of different sizes on the multiple processed multi-scale fused feature maps respectively to obtain the boundary box position and category of the defect; After convolution and residual processing, the head network uses the detection and recognition layer to mark defects in each processed multi-scale fusion feature map. The detection and recognition layer usually includes a series of operations for predicting the position and category of the bounding box. These operations may include the generation of anchor boxes, regression of bounding boxes (used to predict the precise position of bounding boxes), and classification of categories (used to determine the category of defects within the bounding box). In specific implementation, the architecture of a single-stage detector (such as YOLO, SSD) or a multi-stage detector (such as Faster R-CNN) can be used.

[0076] The head outputs the printed circuit board defect detection result with a bounding box position and category marked with the defect.

[0077] The head network integrates the defect bounding box position and category information obtained by the detection and recognition layer to form the final printed circuit board defect detection results. These results are usually output in the form of a list or set, each element of which contains the bounding box coordinates of the defect (such as the coordinates of the upper left corner and the lower right corner) and the corresponding category label.

[0078] Example description: Assume that the head network receives three multi-scale fusion feature maps processed by the neck network, denoted as Feature Map A, Feature Map B, and Feature Map C.

[0079] Convolution and residual processing: For each feature map, the head network first applies a series of convolutional layers for feature extraction and transformation. Between convolutional layers, residual connections can be introduced as needed to enhance feature propagation. For example, for Feature Map A, the head network may process features through two convolutional layers (each followed by a ReLU activation function and a batch normalization layer), and add the input feature map to the output feature map through a residual connection after the second convolutional layer.

[0080] Detection and recognition layer: After convolution and residual processing, the head network uses the detection and recognition layer to mark defects on each feature map. Assuming that the head network uses a simplified version of the YOLO architecture, a certain number of anchor boxes and the bounding box regression parameters and category probabilities corresponding to these anchor boxes are predicted for each grid cell on each feature map. Specifically, for a grid cell on Feature Map A, the head network may predict three anchor boxes (these anchor boxes have different sizes and scales) and output four bounding box regression parameters for each anchor box (used to adjust the anchor box to match the bounding box of the real defect) and a category probability vector (indicating the probability that the defect in the anchor box belongs to each category).

[0081] Output defect detection results: Finally, the head network integrates the bounding box locations and category information of the defects detected on all feature maps. It may apply post-processing techniques such as non-maximum suppression (NMS) to remove duplicate bounding boxes and select the bounding box with the highest score as the final detection result. These results are then output in the form of a list or set, containing the bounding box locations and category information of all detected defects on the printed circuit board.

[0082] In a possible implementation, the loss function of the YOLO neural network model includes: positioning error, confidence error of containing an object, confidence error of not containing an object, and classification error; During the training process of the YOLOV5 model, the back propagation (BP) algorithm is used to generate an error signal by comparing the actual output (y) with the expected output (r). The error signal is back propagated layer by layer, and the error signal is calculated through each layer. Subsequently, the performance of the neural network is optimized by adjusting the connection weights of each layer to minimize the error.

[0083] Among them, the loss function L YOLOv5 The discriminant formula is: ; Among them, λ coord is the positioning loss, which is used to measure the spatial position difference between the predicted bounding box and the actual bounding box; i and i The predicted center point coordinates that constitute the i-th bounding box; x i and i The coordinates of the real center point that constitutes the i-th bounding box; w i and h i are the predicted width and predicted height of the i-th bounding box, respectively. and are the true width and true height of the i-th bounding box, B is the number of bounding boxes, S is the number of feature grids, and λ noobj is the confidence loss, which is used to measure the confidence of whether there is a defect in the predicted bounding box; p i (c) is the prediction confidence of the i-th bounding box, which is used to indicate the probability that the i-th bounding box contains a defect; is the true confidence of the i-th bounding box, which is used to indicate whether the i-th bounding box contains defects; C i is the predicted category probability of the i-th bounding box, is the true category label of the i-th bounding box.

[0084] The present disclosure also provides a printed circuit board defect detection device. Figure 3 As shown, the device comprises: The acquisition module 310 is configured to acquire an image of a printed circuit board to be inspected; An input module 320 is configured to input the printed circuit board image to be detected into a YOLO neural network model to obtain a printed circuit board defect detection result, wherein the YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck; Wherein, the backbone network comprises a cross-stage partial network module and a cross-stage binary network module which are alternately arranged in sequence, and the backbone network sequentially performs multiple feature extraction and downsampling operations of different scales on the detected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each of the cross-stage partial network modules to the neck; The neck starts from the high-dimensional feature map with the highest dimension, sequentially performs upsampling and fusion with the high-dimensional feature map with lower dimension, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimension, and outputs a plurality of multi-scale fused feature maps to the head; The head detects and identifies the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the boundary box position and category of the defect marked.

[0085] The present disclosure also provides an electronic device, including: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of the method in any one of the aforementioned embodiments.

[0086] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method described in any one of the aforementioned embodiments are implemented.

[0087] Figure 3 The printed circuit board defect detection device 100 shown includes: a processor 1001 and a memory 1003. The processor 1001 and the memory 1003 are connected, such as through a bus 1002. Optionally, the printed circuit board defect detection device 100 may also include a communication component, which can be used for data interaction between the device 100 and other devices, such as data transmission and / or data reception. It should be noted that in actual scheduling, the communication component is not limited to one, and the structure of the printed circuit board defect detection device 100 does not constitute a limitation on the embodiments of the present application.

[0088] Processor 1001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 1001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0089] The bus 1002 may include a path to transmit information between the above components. The bus 1002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 1002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0090] The memory 1003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium, other magnetic storage devices, or any other medium that can be used to carry or store program code and can be read by a computer, without limitation herein.

[0091] The memory 1003 is used to store program codes for executing the embodiments of the present disclosure, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the program codes stored in the memory 1003 to implement the steps shown in the above-mentioned printed circuit board defect detection method embodiment.

[0092] The embodiment of the present disclosure further provides a computer-readable storage medium having program codes stored thereon. When the program codes are executed by a processor, the steps and corresponding contents of the above-mentioned printed circuit board defect detection method embodiment can be implemented.

[0093] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments; within the technical concept of the present disclosure, various changes, modifications, substitutions and variations may be made to these embodiments, and these changes, modifications, substitutions and variations all fall within the protection scope of the present disclosure.

[0094] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction, and they should also be regarded as the contents disclosed in this disclosure. In order to avoid unnecessary repetition, this disclosure will not further describe various possible combinations. The technical scope of this application is not limited to the contents in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A printed circuit board defect detection method, characterized in that: The method comprises: Acquire a printed circuit board image to be detected, input the printed circuit board image to be detected into a YOLO neural network model, and obtain a printed circuit board defect detection result, wherein the YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck; Wherein, the backbone network comprises a cross-stage partial network module and a cross-stage binary network module which are alternately arranged in sequence, and the backbone network sequentially performs multiple feature extraction and downsampling operations of different scales on the detected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each of the cross-stage partial network modules to the neck; The neck starts from the high-dimensional feature map with the highest dimension, sequentially performs upsampling and fusion with the high-dimensional feature map with lower dimension, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimension, and outputs a plurality of multi-scale fused feature maps to the head; The head detects and identifies the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the boundary box position and category of the defect marked.

2. The printed circuit board defect detection method according to claim 1, characterized in that: The backbone network is also provided with a convolution module and a plurality of cross-stage binary network modules in sequence before the cross-stage partial network modules and the cross-stage binary network modules that are alternately provided in sequence; The backbone network performs convolution processing on the detected printed circuit board image through the convolution module, and performs convolution and normalization processing on the input respectively through multiple cross-stage binary network modules, and inputs the image after feature extraction into the first of the cross-stage binary network modules that are alternately arranged in sequence.

3. The printed circuit board defect detection method according to claim 2, characterized in that: The backbone network is further provided with a spatial pyramid pooling module after the cross-stage partial network modules and the cross-stage binary network modules which are alternately arranged in sequence; The backbone network performs pooling operations of different scales on the highest-dimensional high-dimensional feature map output by the last cross-stage partial network module through the spatial pyramid pooling module, and outputs the high-dimensional feature map after the pooling operation to the neck.

4. The printed circuit board defect detection method according to claim 1, characterized in that: After each of the cross-stage partial network modules of the backbone network performs feature enhancement on the input feature map once, the feature map after one feature enhancement is sequentially subjected to multiple maximum pooling splicing convolutions, and then is spliced ​​with the feature map after another feature enhancement to obtain high-dimensional feature maps corresponding to different scales, wherein the number of residual blocks in each of the cross-stage partial network modules is different, so that the feature map after one feature enhancement is sequentially subjected to multiple maximum pooling splicings of different numbers of times.

5. The printed circuit board defect detection method according to claim 1, characterized in that: The neck starts from the high-dimensional feature map with the highest dimension, sequentially performs upsampling and fusion with the high-dimensional feature map with lower dimension, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimension, and outputs multiple multi-scale fused feature maps to the head, including: The neck starts from the high-dimensional feature map with the highest dimension and sequentially upsamples the high-dimensional feature map; Use the connection operation to connect the upsampled high-dimensional feature maps with feature maps of different scales to achieve multi-scale feature fusion; Residual connection is performed through the cross-stage partial network module set in the neck to enhance the multi-scale feature extraction capability, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with a higher dimension, and multiple multi-scale fused feature maps are output to the head.

6. The printed circuit board defect detection method according to claim 1, wherein: The head detects and identifies the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the position and category of the boundary box of the defect marked, including: The head performs convolution and residual processing on the multiple multi-scale fused feature maps respectively to obtain multiple processed multi-scale fused feature maps; The detection and recognition layer of the head performs defect marking of different sizes on the multiple processed multi-scale fused feature maps respectively to obtain the boundary box position and category of the defect; The head outputs the printed circuit board defect detection result with a bounding box position and a category of the defect marked.

7. The printed circuit board defect detection method according to any one of claims 1 to 6, characterized in that: The loss function of the YOLO neural network model includes: positioning error, confidence error of containing objects, confidence error of not containing objects, and classification error; Among them, the loss function L YOLOv5 The discriminant formula is: ； Among them, λ coord is the positioning loss, which is used to measure the spatial position difference between the predicted bounding box and the actual bounding box; i and i The predicted center point coordinates that constitute the i-th bounding box; x i and i The coordinates of the real center point that constitutes the i-th bounding box; w i and h i are the predicted width and predicted height of the i-th bounding box, respectively. and are the true width and true height of the i-th bounding box, B is the number of bounding boxes, S is the number of feature grids, and λ noobj is the confidence loss, which is used to measure the confidence of whether there is a defect in the predicted bounding box; p i (c) is the prediction confidence of the i-th bounding box, which is used to indicate the probability that the i-th bounding box contains a defect; is the true confidence of the i-th bounding box, which is used to indicate whether the i-th bounding box contains defects; C i is the predicted category probability of the i-th bounding box, is the true category label of the i-th bounding box.

8. A printed circuit board defect detection device, characterized in that: The device comprises: An acquisition module is configured to acquire an image of a printed circuit board to be inspected; An input module is configured to input the printed circuit board image to be detected into a YOLO neural network model to obtain a printed circuit board defect detection result, wherein the YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck; Wherein, the backbone network comprises a cross-stage partial network module and a cross-stage binary network module which are alternately arranged in sequence, and the backbone network sequentially performs multiple feature extraction and downsampling operations of different scales on the detected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each of the cross-stage partial network modules to the neck; The neck starts from the high-dimensional feature map with the highest dimension, sequentially performs upsampling and fusion with the high-dimensional feature map with lower dimension, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimension, and outputs a plurality of multi-scale fused feature maps to the head; The head detects and identifies the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the boundary box position and category of the defect marked.

9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images