Printed circuit board defect detection method, device, equipment and medium

Abstract

The present invention relates to a printed circuit board defect detection method, device, equipment and medium, comprising obtaining an image of a printed circuit board to be detected, inputting the image of the printed circuit board to be detected into a YOLO neural network model, and obtaining a printed circuit board defect detection result, wherein the model comprises a backbone network, a neck and a head; the backbone network comprises a cross-stage partial network module and a cross-stage binary network module which are alternately arranged in sequence, and sequentially performs feature extraction and downsampling operations of multiple different scales on the detected printed circuit board image, and outputs a high-dimensional feature map obtained by the cross-stage partial network module to the neck; the neck starts from the high-dimensional feature map with the highest dimension, sequentially performs upsampling and fusion with high-dimensional feature maps with lower dimensions, and outputs multiple multi-scale fused feature maps to the head; the head respectively detects and identifies the multiple multi-scale fused feature maps, and outputs a printed circuit board defect detection result of the defect boundary box position and category.

Description

Technical Field

[0001] The present disclosure relates to the field of detection technology, and in particular to a printed circuit board defect detection method, device, equipment and medium. Background Art

[0002] Printed circuit boards (PCBs) are key components in electronic devices and are widely used due to their intelligence, lightweight, high density, and integration. Defects in PCBs can seriously compromise the functionality and safety of electronic devices, leading to reduced production and safety risks. PCB defect detection primarily relies on manual visual inspection and automated optical inspection (AOI). However, manual inspection is subject to human error, while AOI performance is highly dependent on optical conditions, resulting in a high false detection rate. These limitations lead to reduced PCB inspection accuracy, increased costs, and decreased efficiency in complex inspection environments. Summary of the Invention

[0003] The purpose of the present invention is to provide a printed circuit board defect detection method, device, equipment and medium, aiming to solve the technical problems of reduced PCB detection accuracy, increased cost and decreased efficiency in complex detection environments in related scenarios.

[0004] In order to achieve the above-mentioned object, a first aspect of an embodiment of the present disclosure provides a method for detecting defects in a printed circuit board, the method comprising:

[0005] Obtaining an image of a printed circuit board to be inspected, inputting the image of the printed circuit board to be inspected into a YOLO neural network model to obtain a printed circuit board defect detection result, wherein the YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck;

[0006] The backbone network includes a cross-stage partial network module and a cross-stage binary network module that are alternately arranged in sequence. The backbone network sequentially performs multiple feature extraction and downsampling operations at different scales on the detected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each cross-stage partial network module to the neck;

[0007] The neck starts from the high-dimensional feature map with the highest dimension, sequentially upsampling and fusing with the high-dimensional feature maps with lower dimensions, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimensions, and outputs multiple multi-scale fused feature maps to the head;

[0008] The head performs detection and recognition on the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the bounding box position and category of the defect marked.

[0009] In a possible implementation, the backbone network is further provided with a convolution module and a plurality of cross-stage binary network modules in sequence before the cross-stage partial network modules and the cross-stage binary network modules that are alternately provided in sequence;

[0010] The backbone network performs convolution processing on the detected printed circuit board image through the convolution module, and performs convolution and normalization processing on the input respectively through multiple cross-stage binary network modules, and inputs the image after feature extraction into the first of the cross-stage binary network modules that are alternately arranged in sequence.

[0011] In a possible implementation, the backbone network is further provided with a spatial pyramid pooling module after the cross-stage partial network module and the cross-stage binary network module that are alternately provided in sequence;

[0012] The backbone network performs pooling operations of different scales on the highest-dimensional high-dimensional feature map output by the last cross-stage partial network module through the spatial pyramid pooling module, and outputs the high-dimensional feature map after the pooling operation to the neck.

[0013] In a possible implementation, each of the cross-stage partial network modules of the backbone network performs feature enhancement on the input feature map once, performs multiple maximum pooling convolutions on the feature map after one feature enhancement, and then splices it with the feature map after another feature enhancement to obtain high-dimensional feature maps corresponding to different scales, wherein the number of residual blocks in each of the cross-stage partial network modules is different, so that the feature map after one feature enhancement is subjected to multiple maximum pooling splicings for different numbers of times.

[0014] In a possible implementation, the neck starts from the high-dimensional feature map with the highest dimension, sequentially upsampling and fusing with the high-dimensional feature maps with lower dimensions, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimensions, and outputting multiple multi-scale fused feature maps to the head, including:

[0015] The neck starts from the high-dimensional feature map with the highest dimension and sequentially upsamples the high-dimensional feature map;

[0016] Use the connection operation to connect the upsampled high-dimensional feature maps with feature maps of different scales to achieve multi-scale feature fusion;

[0017] Residual connection is performed through the cross-stage partial network module set in the neck to enhance the multi-scale feature extraction capability, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimension, and multiple multi-scale fused feature maps are output to the head.

[0018] In a possible implementation, the head performs detection and recognition on multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the bounding box position and category of the defect marked, including:

[0019] The head performs convolution and residual processing on the multiple multi-scale fused feature maps respectively to obtain multiple processed multi-scale fused feature maps;

[0020] The detection and recognition layer of the head performs defect marking of different sizes on the multiple processed multi-scale fused feature maps to obtain the bounding box position and category of the defect;

[0021] The head outputs the printed circuit board defect detection result with a bounding box position and category of the defect marked.

[0022] In one possible implementation, the loss function of the YOLO neural network model includes: positioning error, confidence error of containing an object, confidence error of not containing an object, and classification error;

[0023] Among them, the loss function L YOLOv5 The discriminant formula is:

[0024] ;

[0025] Among them, λ coord is the positioning loss, which is used to measure the spatial position difference between the predicted bounding box and the true bounding box; i and y i The predicted center coordinates of the i-th bounding box; x i and y i The coordinates of the true center point of the i-th bounding box; w i and h i are the predicted width and predicted height of the i-th bounding box, respectively, and are the true width and true height of the i-th bounding box, B is the number of bounding boxes, S is the number of feature grids, and λ noobj is the confidence loss, which is used to measure the confidence of whether there is a defect in the predicted bounding box; p i (c) is the prediction confidence of the i-th bounding box, which is used to indicate the probability that the i-th bounding box contains a defect; is the true confidence of the i-th bounding box, which is used to indicate whether the i-th bounding box contains defects; C i is the predicted category probability of the i-th bounding box, is the true category label of the i-th bounding box.

[0026] According to a second aspect of the present disclosure, there is provided a printed circuit board defect detection device, the device comprising:

[0027] an acquisition module, configured to acquire an image of a printed circuit board to be inspected;

[0028] An input module is configured to input the printed circuit board image to be inspected into a YOLO neural network model to obtain a printed circuit board defect detection result, wherein the YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck;

[0029] The backbone network includes a cross-stage partial network module and a cross-stage binary network module that are alternately arranged in sequence. The backbone network sequentially performs multiple feature extraction and downsampling operations at different scales on the detected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each cross-stage partial network module to the neck;

[0030] The neck starts from the high-dimensional feature map with the highest dimension, sequentially upsampling and fusing with the high-dimensional feature maps with lower dimensions, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimensions, and outputs multiple multi-scale fused feature maps to the head;

[0031] The head performs detection and recognition on the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the bounding box position and category of the defect marked.

[0032] According to a third aspect of the present disclosure, an electronic device is provided, including:

[0033] a memory having a computer program stored thereon;

[0034] A processor is used to execute the computer program in the memory to implement the steps of any one of the methods in the first aspect.

[0035] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0036] The present invention provides a printed circuit board defect detection method, device, equipment, and medium. Compared with the prior art, it has the following advantages:

[0037] An image of a printed circuit board to be inspected is obtained and input into a YOLO neural network model to obtain a printed circuit board defect detection result. The YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck. The backbone network includes a cross-stage partial network module and a cross-stage binary network module that are alternately arranged in sequence. The backbone network performs multiple feature extraction and downsampling operations at different scales on the inspected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs high-dimensional feature maps corresponding to different scales obtained by each cross-stage partial network module to the neck. The learning ability of the model is improved through cross-stage partial connections (CSP) and residual connections, and more feature information is retained, thereby improving the accuracy of PCB defect detection.

[0038] Then, starting from the high-dimensional feature map with the highest dimension, the neck is upsampled and fused with high-dimensional feature maps with lower dimensions in sequence, until the high-dimensional feature map with the lowest dimension is fused with high-dimensional feature maps with higher dimensions, and multiple multi-scale fused feature maps are output to the head; through residual connection, the model's ability to extract multi-scale features is enhanced.

[0039] The head performs detection and recognition on multiple, multi-scale fused feature maps, outputting PCB defect detection results with bounding box locations and categories. Upsampling and downsampling, as well as feature fusion, across multiple scales ensure the model's ability to effectively detect defects of varying sizes and complexities. This model also significantly improves efficiency and reduces labor costs, addressing the technical challenges of reduced PCB inspection accuracy, increased costs, and decreased efficiency in complex inspection environments.

[0040] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0042] Figure 1 The present invention is a flowchart of a printed circuit board defect detection method according to an embodiment of the specification.

[0043] Figure 2 It is a structural diagram of a YOLO neural network model according to an embodiment of the specification.

[0044] Figure 3 The present invention is a block diagram of a printed circuit board defect detection device according to an embodiment of the present invention.

[0045] Figure 4 It is a block diagram of another printed circuit board defect detection device according to an embodiment of the specification. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.

[0048] The present disclosure provides a method for detecting defects in a printed circuit board. Figure 1 This is a flow chart showing a method for detecting defects in a printed circuit board according to an embodiment. The method includes:

[0049] In step S11, an image of a printed circuit board to be inspected is acquired.

[0050] In this disclosure, images of the printed circuit board (PCB) to be inspected are acquired using an image acquisition device (such as a high-resolution camera or scanner). These images should clearly show all details on the PCB, including wires, components, and solder joints. After image acquisition, preprocessing operations such as denoising, grayscale conversion, and binarization are typically performed to improve image quality and reduce the complexity of subsequent processing.

[0051] As an example, consider an industrial-grade HD camera mounted at the end of a PCB production line, non-contactingly photographing the PCBs just off the line. The camera's images are transferred to a computer via USB or Ethernet. Software on the computer automatically receives these images and immediately performs grayscale conversion and simple noise reduction to prepare for subsequent defect detection.

[0052] In step S12, the printed circuit board image to be inspected is input into a YOLO neural network model to obtain a printed circuit board defect detection result, wherein the YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck;

[0053] YOLO (You Only Look Once) is trained in PCB defect detection to identify defects in images, such as short circuits, open circuits, and missing or misplaced components. The YOLO neural network model integrates multiple levels of residual connections within a single unit, extracting feature information at different scales, enabling detection of PCB defects of various sizes and complexities. YOLO consists of three main components: Backbone, Neck, and Head.

[0054] The backbone network includes a cross-stage partial network module and a cross-stage binary network module that are alternately arranged in sequence. The backbone network sequentially performs multiple feature extraction and downsampling operations at different scales on the detected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each cross-stage partial network module to the neck;

[0055] See also Figure 2 As shown in the figure, the Backbone network is responsible for feature extraction. The input PCB image is processed by multiple convolutional blocks to output high-level feature maps. This part uses multi-level CSP (Cross Stage Partial) blocks and residual structures (ResX).

[0056] The model extracts multi-scale features from the input image. By alternating between a Cross-Stage Partial Network (CSP) module and a Cross-Stage Binary Network (CBS) module, it can gradually reduce the spatial resolution of the image (downsampling) while increasing the number of channels in the feature map (i.e., feature depth), thereby capturing image information at different levels.

[0057] For example, a 640×640 input image undergoes feature extraction through the initial convolutional blocks and CSP1_X blocks. Subsequently, deeper convolutional blocks (such as CBL and CSP1_X) and downsampling operations (P1 / 2, P2 / 4, etc.) gradually reduce the size of the feature map and extract high-dimensional features. The Backbone passes feature maps of varying scales to the Neck portion of the next stage. CSP1_X and CSP2_X utilize cross-stage partial connections (CSP) and residual connections to enhance the model's learning capabilities and preserve more feature information.

[0058] The neck starts from the high-dimensional feature map with the highest dimension, sequentially upsampling and fusing with the high-dimensional feature maps with lower dimensions, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimensions, and outputs multiple multi-scale fused feature maps to the head;

[0059] The neck layer combines feature maps of different scales through upsampling and feature fusion, generating a series of multi-scale fused feature maps. These feature maps contain both high-level semantic information (from deep feature maps) and detailed information (from shallow feature maps), making them ideal for multi-scale object detection and feature fusion and enhancement. This part uses the Feature Pyramid Network (FPN) architecture to improve the detection of small objects through upsampling and feature fusion. The key steps include:

[0060] Upsampling is performed on feature maps from different layers of the Backbone (e.g., 512×20×20, 256×40×40). Concat operations are used to connect feature maps of different scales to achieve multi-scale feature fusion. The CSP2_X module enhances the network's multi-scale feature extraction capability through residual connections. Finally, the multi-scale fused feature maps are passed to the Head for detection.

[0061] The head performs detection and recognition on the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the bounding box position and category of the defect marked.

[0062] The Head performs independent detection and recognition of feature maps at each scale. Using preset anchor boxes and classifiers, the model predicts the bounding box location and category of defects for final detection and classification. This component receives the fused multi-scale feature maps from the Neck and further processes them to locate and classify the target. Specific operations include:

[0063] The feature map obtained from Neck is further processed by the CSP1 module and then passed to the Detect layer. The Detect layer is responsible for object detection on feature maps of different scales and handles PCB defect targets of different sizes. Finally, the model outputs the bounding box location and category information of the defect.

[0064] The residual block (ResX) is used to introduce short-circuit connections (shortcut=true) into the network, helping to alleviate the vanishing gradient problem and improve model training stability. Multi-scale feature extraction uses upsampling and downsampling and feature fusion of feature maps at multiple scales to ensure that the model can effectively detect defects of varying sizes and complexities.

[0065] For example, a preprocessed PCB image is fed into a trained YOLO model. The model first performs multi-scale feature extraction on the image through the backbone network (comprising a combination of multiple CSP and CBS modules), generating a series of high-dimensional feature maps. Next, in the neck stage, these feature maps are combined into multiple multi-scale fused feature maps through upsampling and feature fusion. Finally, in the head stage, each feature map is used to predict potential defects. For example, the model may detect a short circuit bounding box in a feature map and predict its location and type as "short circuit." These predictions are aggregated to form the final PCB defect inspection report.

[0066] The above technical solution obtains the image of the printed circuit board to be inspected and inputs the image of the printed circuit board to be inspected into the YOLO neural network model to obtain the printed circuit board defect detection result. The YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck; wherein the backbone network includes a cross-stage partial network module and a cross-stage binary network module that are alternately arranged in sequence. The backbone network performs multiple feature extraction and downsampling operations of different scales on the inspected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each cross-stage partial network module to the neck; the learning ability of the model is improved through cross-stage partial connections and residual connections, and more feature information is retained, thereby improving the accuracy of PCB defect detection.

[0067] Then, starting from the high-dimensional feature map with the highest dimension, the neck is upsampled and fused with high-dimensional feature maps with lower dimensions in sequence, until the high-dimensional feature map with the lowest dimension is fused with high-dimensional feature maps with higher dimensions, and multiple multi-scale fused feature maps are output to the head; through residual connection, the model's ability to extract multi-scale features is enhanced.

[0068] The head performs detection and recognition on multiple, multi-scale fused feature maps, outputting PCB defect detection results with bounding box locations and categories. Upsampling and downsampling, as well as feature fusion, across multiple scales ensure the model's ability to effectively detect defects of varying sizes and complexities. This model also significantly improves efficiency and reduces labor costs, addressing the technical challenges of reduced PCB inspection accuracy, increased costs, and decreased efficiency in complex inspection environments.

[0069] In one possible implementation, see Figure 2 As shown, the backbone network is further provided with a convolution module and a plurality of cross-stage binary network modules in sequence before the cross-stage partial network modules and the cross-stage binary network modules that are alternately provided in sequence;

[0070] The backbone network performs convolution processing on the detected printed circuit board image through the convolution module, and performs convolution and normalization processing on the input respectively through multiple cross-stage binary network modules, and inputs the image after feature extraction into the first of the cross-stage binary network modules that are alternately arranged in sequence.

[0071] The convolutional layer is typically located at the front end of the backbone network and performs preliminary feature extraction on the input PCB image. The convolutional layer uses a sliding window to perform local perception of the image and introduces nonlinearity through activation functions (such as ReLU) to extract basic image features such as edges and textures.

[0072] Multiple Cross-Stage Binary Network Modules (prepend): Following the convolutional module, multiple consecutive Cross-Stage Binary Network modules (which may be identical to or slightly different from the CBS in the subsequent alternating modules) are placed. These modules further convolve and normalize the image. Normalization here may refer to batch normalization or other forms of normalization, which aims to accelerate training and improve model stability and performance. Through multiple such modules, image features are gradually abstracted and enhanced, laying a solid foundation for subsequent feature extraction and downsampling operations.

[0073] After the above preprocessing, the backbone network begins to use the alternating CSP and CBS modules to perform multi-scale feature extraction and downsampling. The operation of this stage has been explained in detail in the previous answer and will not be repeated here.

[0074] Example: Consider a PCB image. It is first fed into a convolutional module. The convolutional module extracts basic image features, such as edges and corners, through several convolutional layers and ReLU activation functions. These feature maps are then fed into a series of preceding cross-stage binary network modules.

[0075] In each CSP module, the image first passes through a convolutional layer for feature extraction and then passes through a normalization layer (such as batch normalization) to normalize the features to reduce internal covariate shift and improve training stability. After passing through multiple such modules, the image features are further abstracted and enhanced.

[0076] These enhanced feature maps are then fed into alternating CSP and CBS modules. The CSP module enhances feature extraction capabilities through its unique architecture (such as residual connections and grouped convolutions) while reducing computational effort and parameter requirements. The CBS module, on the other hand, may focus on processing or enhancing binary features to meet the specific needs of PCB defect detection.

[0077] Ultimately, through this series of complex operations, the backbone network is able to generate information-rich feature maps at multiple scales, which will be fed into the neck and head for further processing and detection.

[0078] In one possible implementation, see Figure 2 As shown, the backbone network is further provided with a spatial pyramid pooling module after the cross-stage partial network module and the cross-stage binary network module are alternately provided in sequence;

[0079] The backbone network performs pooling operations of different scales on the highest-dimensional high-dimensional feature map output by the last cross-stage partial network module through the spatial pyramid pooling module, and outputs the high-dimensional feature map after the pooling operation to the neck.

[0080] The Spatial Pyramid Pooling (SSPF) module performs pooling operations at different scales on the highest-dimensional feature map output by the backbone network. Unlike traditional single-scale pooling, the SSPF module adopts a multi-scale pooling strategy, dividing the feature map into multiple grids of different sizes (such as 1×1, 2×2, 4×4, etc.) and pooling the features within each grid (typically using max pooling or average pooling). This multi-scale pooling approach captures contextual information at different scales, enhancing the model's robustness and generalization capabilities.

[0081] Feature Map Fusion and Output: After processing by the SSPF module, the pooled feature maps at multiple scales are further integrated or kept independent (depending on the specific model design) and then used as input to the neck network. These multi-scale feature maps not only contain rich semantic information but also exhibit a certain degree of scale invariance, which is helpful for subsequent detection tasks.

[0082] Example: Assume that after a series of alternating CSP and CBS modules, the backbone network outputs a high-dimensional feature map of dimensions H×W×C (where H, W, and C represent the height, width, and number of channels of the feature map, respectively). This feature map is then fed into the spatial pyramid pooling module.

[0083] In the SSPF module, the feature map is first divided into multiple grids of different sizes. For example, it can be divided into grids of three sizes: 1×1, 2×2, and 4×4. For each grid size, a max pooling operation is performed, taking the maximum value of all feature values within each grid as the output for that grid. Thus, for the original high-dimensional feature map, the SSPF module outputs three pooled feature maps (or feature vectors, if flattened) of different scales.

[0084] These pooled feature maps (or feature vectors) are then fed into the neck network. There, they may undergo upsampling, feature fusion, and other operations to generate multi-scale fused feature maps for detection. These feature maps not only contain the rich information of the original feature maps but also incorporate multi-scale contextual information through the SSPF module, thereby improving the model's ability to detect PCB defects.

[0085] It should be noted that although maximum pooling is used as an example here, in fact the SSPF module can also use other types of pooling operations (such as average pooling) or combine multiple pooling strategies to further improve the performance of the model.

[0086] In a possible implementation, each of the cross-stage partial network modules of the backbone network performs feature enhancement on the input feature map once, performs multiple maximum pooling convolutions on the feature map after one feature enhancement, and then splices it with the feature map after another feature enhancement to obtain high-dimensional feature maps corresponding to different scales, wherein the number of residual blocks in each of the cross-stage partial network modules is different, so that the feature map after one feature enhancement is subjected to multiple maximum pooling splicings for different numbers of times.

[0087] Feature enhancement: In each CSP module, the input feature map first undergoes one or more feature enhancement operations. These feature enhancement operations may include convolutional layers, activation functions (such as ReLU), and normalization layers (such as batch normalization), aiming to extract and enhance the expressive power of the feature map.

[0088] Multi-scale Feature Extraction: After feature enhancement, the CSP module employs a unique design to generate multi-scale feature maps. Specifically, it sequentially applies multiple max-pooling operations to one of the feature maps after feature enhancement (referred to as the "main path" feature map). Each pooling operation is followed by a convolutional layer (or series of convolutional layers) to further extract and transform features. These feature maps, which have undergone different times of max-pooling, are then concatenated with the feature map after another feature enhancement (referred to as the "residual path" feature map).

[0089] Residual Blocks and Multi-Scale Splicing: It's worth noting that the number of residual blocks in each CSP module can vary. This design allows the model to learn features of different scales at different depths. During the splicing process, due to the varying number of residual blocks, the main path feature map undergoes varying numbers of max-pooling splicing operations, generating feature maps at multiple scales. These feature maps are then spliced with the residual path feature map to form the final high-dimensional feature maps corresponding to the different scales.

[0090] Example: Consider a CSP module containing two residual blocks. In this module, the input feature map first undergoes a feature enhancement operation (e.g., a convolutional layer followed by a ReLU activation function and a batch normalization layer). This enhanced feature map is then split into two parts: one that enters the main path and the other that enters the residual path.

[0091] In the main path, the feature map first undergoes a max-pooling operation, followed by a convolutional layer for feature transformation. Next, this pooled and convolutional feature map undergoes another max-pooling operation and another convolutional layer. Because this module contains two residual blocks, the main path feature map actually only undergoes two max-pooling concatenation operations (although each concatenation may include multiple convolutional layers).

[0092] Meanwhile, the feature maps in the residual path undergo one or more feature enhancement operations (depending on the module design), but are not pooled to maintain their spatial resolution. Finally, the feature maps from the main and residual paths are concatenated along the channel dimension to form the high-dimensional feature maps corresponding to the different scales output by the CSP module. This feature map is then fed into the next CSP or CBS module for further feature extraction and transformation.

[0093] It should be noted that the above example only illustrates one possible implementation within the CSP module. In actual applications, the specific design of the CSP module may be adjusted based on factors such as task requirements, model complexity, and computing resources.

[0094] In a possible implementation, the neck starts from the high-dimensional feature map with the highest dimension, sequentially upsampling and fusing with the high-dimensional feature maps with lower dimensions, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimensions, and outputting multiple multi-scale fused feature maps to the head, including:

[0095] The neck starts from the high-dimensional feature map with the highest dimension and sequentially upsamples the high-dimensional feature map;

[0096] The neck network starts with the highest-dimensional feature map and sequentially upsamples these feature maps. The goal of upsampling is to restore lower-resolution feature maps to a higher resolution for fusion with subsequent higher-dimensional feature maps. Upsampling can be achieved using interpolation algorithms (such as bilinear interpolation and nearest neighbor interpolation) or transposed convolution (deconvolution).

[0097] Use the connection operation to connect the upsampled high-dimensional feature maps with feature maps of different scales to achieve multi-scale feature fusion;

[0098] Among them, after upsampling, the neck network uses a connection operation (such as Concatenate) to connect the upsampled high-dimensional feature map with a high-dimensional feature map of lower dimension (but appropriately upsampled to match the resolution). This connection method allows feature maps of different scales to be fused in the channel dimension, thereby making full use of contextual information at different scales.

[0099] To further enhance the ability to extract multi-scale features, the neck network can also include cross-stage partial network modules (or similar residual structures). These modules use residual connections to add the input feature map to the output feature map within the module, thereby preserving more original information and alleviating the vanishing gradient problem. The use of residual connections helps improve the overall performance of the model.

[0100] Residual connection is performed through the cross-stage partial network module set in the neck to enhance the multi-scale feature extraction capability, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimension, and multiple multi-scale fused feature maps are output to the head.

[0101] After a series of upsampling, feature map concatenation, and possible residual connections, the neck network ultimately outputs multiple multi-scale fused feature maps to the head. These feature maps not only contain rich semantic information, but also enhance the expressive power of features through multi-scale fusion and residual connections.

[0102] Example description: Assume that the neck network receives three high-dimensional feature maps of different scales, which are recorded as Feature Map 1 (highest dimension), Feature Map 2 and Feature Map 3 (lowest dimension).

[0103] Upsampling operation: First, Feature Map 1 is upsampled so that its resolution matches Feature Map 2.

[0104] Feature map connection: Connect the upsampled Feature Map 1 and Feature Map 2 to form a fused feature map.

[0105] Residual connection (optional): If a cross-stage partial network module is set in the neck network, the fused feature map can be used as input to further enhance its feature expression capability through residual connection.

[0106] Repeat the operation: Next, upsample the fused feature map obtained in the previous step (if necessary) to match the resolution of Feature Map 3 and connect it with Feature Map 3. Similarly, residual connections can also be applied in this step (if any in the design).

[0107] Outputting multi-scale fused feature maps: After the above operations, the neck network outputs multiple multi-scale fused feature maps to the head. These feature maps will serve as input to the head network for the final detection or classification task.

[0108] It is important to note that the number of upsampling times, feature map concatenation methods, and use of residual connections in the above examples are determined by the specific model design and task requirements. In practical applications, these parameters and structures can be adjusted as needed to achieve optimal performance.

[0109] In a possible implementation, the head performs detection and recognition on multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the bounding box position and category of the defect marked, including:

[0110] The head performs convolution and residual processing on the multiple multi-scale fused feature maps respectively to obtain multiple processed multi-scale fused feature maps;

[0111] The head network first performs convolution on each multi-scale fused feature map to further extract and transform features. These convolutional layers are typically followed by activation functions (such as ReLU) and normalization layers (such as batch normalization) to enhance feature representation. Furthermore, to maintain feature richness and avoid the vanishing gradient problem, the head network can also introduce residual connections between these convolutional layers. Residual connections add the input feature map to the output feature map of the convolutional layer, ensuring that more original information is retained during training.

[0112] The detection and recognition layer of the head performs defect marking of different sizes on the multiple processed multi-scale fused feature maps to obtain the bounding box position and category of the defect;

[0113] After convolution and residual processing, the head network uses the detection and recognition layer to label defects in each processed multi-scale fused feature map. The detection and recognition layer typically includes a series of operations to predict the location and category of bounding boxes. These operations may include anchor box generation, bounding box regression (for predicting the precise location of the bounding box), and category classification (for determining the category of the defect within the bounding box). In specific implementations, single-stage detectors (such as YOLO and SSD) or multi-stage detector architectures (such as Faster R-CNN) can be used.

[0114] The head outputs the printed circuit board defect detection result with a bounding box position and category of the defect marked.

[0115] The head network integrates the defect bounding box locations and category information obtained by the detection and recognition layer to form the final PCB defect detection results. These results are typically output as a list or set, with each element containing the bounding box coordinates of the defect (such as the coordinates of the upper left and lower right corners) and the corresponding category label.

[0116] Example description: Assume that the head network receives three multi-scale fusion feature maps processed by the neck network, denoted as Feature Map A, Feature Map B, and Feature Map C.

[0117] Convolution and Residual Processing: For each feature map, the head network first applies a series of convolutional layers for feature extraction and transformation. Residual connections can be introduced between convolutional layers as needed to enhance feature propagation. For example, for Feature Map A, the head network might process the feature using two convolutional layers (each followed by a ReLU activation function and a batch normalization layer), and after the second convolutional layer, add the input feature map to the output feature map using a residual connection.

[0118] Detection and recognition layer: After convolution and residual processing, the head network uses the detection and recognition layer to mark defects on each feature map. Assuming that the head network adopts a simplified version of the YOLO architecture, a certain number of anchor boxes and the bounding box regression parameters and category probabilities corresponding to these anchor boxes are predicted for each grid cell on each feature map. Specifically, for a grid cell on Feature Map A, the head network may predict three anchor boxes (these anchor boxes have different sizes and scales) and output four bounding box regression parameters for each anchor box (used to adjust the anchor box to match the bounding box of the real defect) and a category probability vector (indicating the probability that the defect in the anchor box belongs to each category).

[0119] Outputting defect detection results: Finally, the head network integrates the bounding box locations and category information of the defects detected across all feature maps. It may apply post-processing techniques such as non-maximum suppression (NMS) to remove duplicate bounding boxes and select the bounding box with the highest score as the final detection result. These results are then output as a list or set containing the bounding box locations and category information for all detected defects on the PCB.

[0120] In one possible implementation, the loss function of the YOLO neural network model includes: positioning error, confidence error of containing an object, confidence error of not containing an object, and classification error;

[0121] During the YOLOV5 model training process, the backpropagation (BP) algorithm is used to generate an error signal by comparing the actual output (y) with the expected output (r). This error signal is backpropagated layer by layer, calculating the error signal at each layer. The performance of the neural network is then optimized by adjusting the connection weights of each layer to minimize the error.

[0122] Among them, the loss function L YOLOv5 The discriminant formula is:

[0123] ;

[0124] Among them, λ coord is the positioning loss, which is used to measure the spatial position difference between the predicted bounding box and the true bounding box; i and y i The predicted center coordinates of the i-th bounding box; x i and y i The coordinates of the true center point of the i-th bounding box; w i and h i are the predicted width and predicted height of the i-th bounding box, respectively, and are the true width and true height of the i-th bounding box, B is the number of bounding boxes, S is the number of feature grids, and λ noobj is the confidence loss, which is used to measure the confidence of whether there is a defect in the predicted bounding box; p i (c) is the prediction confidence of the i-th bounding box, which is used to indicate the probability that the i-th bounding box contains a defect; is the true confidence of the i-th bounding box, which is used to indicate whether the i-th bounding box contains defects; C i is the predicted category probability of the i-th bounding box, is the true category label of the i-th bounding box.

[0125] The present disclosure also provides a printed circuit board defect detection device. Figure 3 As shown, the device includes:

[0126] An acquisition module 310 is configured to acquire an image of a printed circuit board to be inspected;

[0127] An input module 320 is configured to input the printed circuit board image to be inspected into a YOLO neural network model to obtain a printed circuit board defect detection result, wherein the YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck;

[0128] The backbone network includes a cross-stage partial network module and a cross-stage binary network module that are alternately arranged in sequence. The backbone network sequentially performs multiple feature extraction and downsampling operations at different scales on the detected printed circuit board image through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each cross-stage partial network module to the neck;

[0129] The neck starts from the high-dimensional feature map with the highest dimension, sequentially upsampling and fusing with the high-dimensional feature maps with lower dimensions, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimensions, and outputs multiple multi-scale fused feature maps to the head;

[0130] The head performs detection and recognition on the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the bounding box position and category of the defect marked.

[0131] The present disclosure also provides an electronic device, including:

[0132] a memory having a computer program stored thereon;

[0133] A processor is used to execute the computer program in the memory to implement the steps of the method in any one of the aforementioned embodiments.

[0134] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in any one of the aforementioned embodiments when the program is executed by a processor.

[0135] Figure 3 The printed circuit board defect detection device 100 shown includes: a processor 1001 and a memory 1003. The processor 1001 and the memory 1003 are connected, for example, via a bus 1002. Optionally, the printed circuit board defect detection device 100 may also include a communication component, which can be used for data exchange between the device 100 and other devices, such as data transmission and / or data reception. It should be noted that in actual deployment, the number of communication components is not limited to one, and the structure of the printed circuit board defect detection device 100 does not constitute a limitation on the embodiments of the present application.

[0136] Processor 1001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 1001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, or a combination of a DSP and a microprocessor.

[0137] Bus 1002 may include a path for transmitting information between the above components. Bus 1002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 1002 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0138] The memory 1003 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store program code and can be read by a computer, without limitation herein.

[0139] The memory 1003 is used to store program codes for executing the embodiments of the present disclosure, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the program codes stored in the memory 1003 to implement the steps shown in the above-mentioned printed circuit board defect detection method embodiment.

[0140] The embodiment of the present disclosure further provides a computer-readable storage medium having program code stored thereon. When the program code is executed by a processor, the steps and corresponding contents of the aforementioned printed circuit board defect detection method embodiment can be implemented.

[0141] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, various changes, modifications, replacements and variations can be made to these embodiments, and these changes, modifications, replacements and variations all fall within the scope of protection of the present disclosure.

[0142] It should also be noted that the various specific technical features described in the above specific embodiments may be combined in any suitable manner, unless there is any contradiction, and these combinations shall also be considered as the contents disclosed in this disclosure. To avoid unnecessary repetition, this disclosure will not further describe various possible combinations. The technical scope of this application is not limited to the contents of the specification and must be determined based on the scope of the claims.

Claims

1. A method for detecting defects in a printed circuit board, characterized in that: The method comprises: Obtaining an image of a printed circuit board to be inspected, inputting the image of the printed circuit board to be inspected into a YOLO neural network model to obtain a printed circuit board defect detection result, wherein the YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck; Wherein, the backbone network includes a cross-stage partial network module and a cross-stage binary network module which are alternately arranged in sequence. The backbone network performs multiple feature extraction and downsampling operations of different scales on the detected printed circuit board image in sequence through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each cross-stage partial network module to the neck. After each cross-stage partial network module of the backbone network performs feature enhancement on the input feature map once, the feature map after one feature enhancement is sequentially subjected to multiple maximum pooling convolutions, and then is spliced with the feature map after another feature enhancement to obtain high-dimensional feature maps corresponding to different scales. The number of residual blocks in each cross-stage partial network module is different, so that the feature map after one feature enhancement is sequentially subjected to multiple maximum pooling convolutions of different numbers of times. The neck starts from the high-dimensional feature map with the highest dimension, sequentially upsampling and fusing with the high-dimensional feature maps with lower dimensions, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimensions, and outputs multiple multi-scale fused feature maps to the head; The head performs detection and recognition on the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the bounding box position and category of the defect marked.

2. The printed circuit board defect detection method according to claim 1, wherein: The backbone network is further provided with a convolution module and a plurality of cross-stage binary network modules in sequence before the cross-stage partial network modules and the cross-stage binary network modules that are alternately provided in sequence; The backbone network performs convolution processing on the detected printed circuit board image through the convolution module, and performs convolution and normalization processing on the input respectively through multiple cross-stage binary network modules, and inputs the image after feature extraction into the first of the cross-stage binary network modules that are alternately arranged in sequence.

3. The printed circuit board defect detection method according to claim 2, wherein: The backbone network is further provided with a spatial pyramid pooling module after the cross-stage partial network module and the cross-stage binary network module which are alternately arranged in sequence; The backbone network performs pooling operations of different scales on the highest-dimensional high-dimensional feature map output by the last cross-stage partial network module through the spatial pyramid pooling module, and outputs the high-dimensional feature map after the pooling operation to the neck.

4. The printed circuit board defect detection method according to claim 1, wherein: The neck starts from the high-dimensional feature map with the highest dimension, sequentially upsampling and fusing with the high-dimensional feature map with lower dimension, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimension, and outputs multiple multi-scale fused feature maps to the head, including: The neck starts from the high-dimensional feature map with the highest dimension and sequentially upsamples the high-dimensional feature map; Use the connection operation to connect the upsampled high-dimensional feature maps with feature maps of different scales to achieve multi-scale feature fusion; Residual connection is performed through the cross-stage partial network module set in the neck to enhance the multi-scale feature extraction capability, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimension, and multiple multi-scale fused feature maps are output to the head.

5. The printed circuit board defect detection method according to claim 1, wherein: The head performs detection and recognition on the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the position and category of the defect bounding box marked, including: The head performs convolution and residual processing on the multiple multi-scale fused feature maps respectively to obtain multiple processed multi-scale fused feature maps; The detection and recognition layer of the head performs defect marking of different sizes on the multiple processed multi-scale fused feature maps to obtain the bounding box position and category of the defect; The head outputs the printed circuit board defect detection result with a bounding box position and category of the defect marked.

6. The printed circuit board defect detection method according to any one of claims 1 to 5, wherein: The loss function of the YOLO neural network model includes: positioning error, confidence error of containing objects, confidence error of not containing objects, and classification error; Among them, the loss function L YOLOv5 The discriminant formula is: ； Among them, λ coord is the positioning loss, which is used to measure the spatial position difference between the predicted bounding box and the true bounding box; i and y i The predicted center point coordinates that constitute the i-th bounding box; and The coordinates of the true center point that constitutes the i-th bounding box; and are the predicted width and predicted height of the i-th bounding box, respectively, and are the true width and true height of the i-th bounding box, B is the number of bounding boxes, S is the number of feature grids, and λ noobj is the confidence loss, which is used to measure the confidence of whether there is a defect in the predicted bounding box; p i (c) is the prediction confidence of the i-th bounding box, which is used to indicate the probability that the i-th bounding box contains a defect; is the true confidence of the i-th bounding box, which is used to indicate whether the i-th bounding box contains defects; C i is the predicted category probability of the i-th bounding box, is the true category label of the i-th bounding box.

7. A printed circuit board defect detection device, characterized in that: The device comprises: an acquisition module, configured to acquire an image of a printed circuit board to be inspected; An input module is configured to input the printed circuit board image to be inspected into a YOLO neural network model to obtain a printed circuit board defect detection result, wherein the YOLO neural network model includes a backbone network, a neck connected to the backbone network, and a head connected to the neck; Wherein, the backbone network includes a cross-stage partial network module and a cross-stage binary network module which are alternately arranged in sequence. The backbone network performs multiple feature extraction and downsampling operations of different scales on the detected printed circuit board image in sequence through the cross-stage partial network module and the cross-stage binary network module, and outputs the high-dimensional feature maps corresponding to different scales obtained by each cross-stage partial network module to the neck. After each cross-stage partial network module of the backbone network performs feature enhancement on the input feature map once, the feature map after one feature enhancement is sequentially subjected to multiple maximum pooling convolutions, and then is spliced with the feature map after another feature enhancement to obtain high-dimensional feature maps corresponding to different scales. The number of residual blocks in each cross-stage partial network module is different, so that the feature map after one feature enhancement is sequentially subjected to multiple maximum pooling convolutions of different numbers of times. The neck starts from the high-dimensional feature map with the highest dimension, sequentially upsampling and fusing with the high-dimensional feature maps with lower dimensions, until the high-dimensional feature map with the lowest dimension is fused with the high-dimensional feature map with higher dimensions, and outputs multiple multi-scale fused feature maps to the head; The head performs detection and recognition on the multiple multi-scale fused feature maps respectively, and outputs the printed circuit board defect detection result with the bounding box position and category of the defect marked.

8. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images