PCB defect detection method, device, equipment and storage medium

Abstract

The present invention discloses a PCB defect detection method, apparatus, device, and storage medium. The detection method includes performing reflective suppression processing on a PCB image; and performing defect detection on the reflective suppression-treated PCB image using a pretrained defect detection model. The defect detection model utilizes an improved YOLOv11 model. Specifically, based on the original YOLOv11 model, an RGB channel branch module is added to the backbone network, and the original C3K2 module is replaced with a C3K2-CA module. The RGB channel branch module effectively enhances the model's ability to perceive color details and minor defects. By adding auxiliary branches to the C3K2 module, the model's spatial and channel attention expression capabilities are enhanced, thereby improving PCB defect detection accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a PCB defect detection method, device, equipment and storage medium. Background Art

[0002] Current product defect detection research mainly focuses on the optimization of target detection model structure. Mainstream methods generally adopt the YOLO series of target detection algorithms, such as YOLOv5, YOLOv8, YOLOv11, etc., and introduce feature enhancement, attention mechanism, multi-scale fusion and other modules based on their model structure to optimize the network structure and improve detection performance.

[0003] The YOLOv8-GFPN model enhances small object detection capabilities by introducing multi-level feature fusion mechanisms, such as the Bidirectional Feature Pyramid Network (BiFPN) and the Guided Filter Pyramid Network (GFPN). However, its spatial perception capabilities are significantly limited, resulting in unclear extraction of PCB defect features and requiring further improvement in recognition accuracy. The YOLOv11-MAS model incorporates the Median Enhanced Attention (MECS) mechanism and the Adaptive Fusion Network (AHFIN), which incorporates a spatial channel attention perception module. While capable of detecting smaller object features, it lacks preprocessing of the original image and only improves the model structure, making it difficult to handle reflections in PCB images.

[0004] However, most of the improvements in these algorithms originate from general target detection tasks, and are not fully adaptable to specific application scenarios such as PCB images, which have high noise and fine-grained defect features. Existing model structures have difficulty fully extracting the semantics and edge information of tiny defect areas, and are more prone to false detection and missed detection, especially in cases of poor image quality. In addition, existing methods generally ignore the pre-influence of input image quality on detection performance, often directly inputting unprocessed raw images into the model, and lack preprocessing strategies for PCB image noise features and defect-sensitive areas. For example, in complex acquisition environments such as uneven lighting, image blur, and reflective interference, the decline in raw image quality severely weakens the model's ability to perceive defect features, becoming a major bottleneck restricting further improvements in detection accuracy.

[0005] Therefore, the lack of targeted image preprocessing mechanisms and network structure optimization for quality degradation problems has become a prominent shortcoming in current PCB defect detection research. Summary of the Invention

[0006] The purpose of the present invention is to provide a PCB defect detection method, device, equipment and storage medium to solve the problem of low PCB defect detection accuracy in existing methods due to poor image quality of the input model and the difficulty of the model structure in fully extracting information about tiny defect areas.

[0007] The present invention solves the above technical problems through the following technical solutions: a PCB defect detection method, comprising:

[0008] Perform reflection suppression processing on PCB images;

[0009] Use the pre-trained defect detection model to perform defect detection on the PCB image after reflection suppression processing;

[0010] The defect detection model adopts an improved YOLOv11 model. The improved YOLOv11 model is based on the original YOLOv11 model, adds an RGB channel branch module to the backbone network, and adopts a C3K2-CA module to replace the original C3K2 module;

[0011] The RGB channel branch module includes a first convolution layer, a first splicing layer, and a second convolution layer connected in sequence; the first convolution layer is used to separate the R, G, and B channels of the PCB image after the reflection suppression process; the first splicing layer is used to splice the G and B channel images output by the first convolution layer to obtain a GB spliced ​​image; the second convolution layer is used to extract features from the GB spliced ​​image to obtain a GB fusion feature map; wherein the GB fusion feature map is spliced ​​with the output features of the first convolution layer in the backbone network, and the GB fusion feature map is also spliced ​​with the features after secondary splicing in the neck network;

[0012] The C3K2-CA module is based on the original C3K2 module and adds an auxiliary branch. The auxiliary branch includes a CBAM module, a first RepConv module, a second RepConv module and a third RepConv module connected in sequence. The input end of the CBAM module is connected to the output end of the first convolutional layer in the C3K2 module, and the output ends of the CBAM module, the first RepConv module, the second RepConv module and the third RepConv module are respectively connected to the splicing layer in the C3K2 module.

[0013] Furthermore, the performing of reflection suppression processing on the PCB image includes:

[0014] According to the PCB image, the reflection components of the R, G, and B channels are calculated respectively, and the reflection components of the R, G, and B channels are merged to obtain an enhanced image. The specific calculation formulas of the reflection components of the R, G, and B channels are:

[0015] ;

[0016] ;

[0017] ;

[0018] in, 、 、 Respectively represent the R channel, G channel, and B channel in position The reflection component at ; 、 、 Respectively represent the R channel, G channel, and B channel in the PCB image at positions The pixel value at ; 、 、 Respectively represent the R channel, G channel, and B channel in position Gaussian filter kernel at ;

[0019] The fusion intensity mapping is performed on the primary enhanced image. The specific formula is:

[0020] ;

[0021] in, In the image after fusion intensity mapping, the c channel is at position The pixel value at ; represents the weight of the c channel; Indicates that in an enhanced image, the c channel is at position The pixel value at ; Indicates the offset;

[0022] The partition-aware histogram enhancement is used to perform partition masking on the image after fusion intensity mapping according to brightness, and differentiated enhancement strategies are adopted for different partitions. The enhanced partitions are fused to obtain a secondary enhanced image, that is, a PCB image after reflection suppression processing. The specific formula is:

[0023] ;

[0024] ;

[0025] in, Represents the position in the secondary enhanced image The pixel value at ; N represents the number of partitions; Indicates location Whether the pixel value at belongs to the mask of the kth partition; ZHE represents the partition-aware histogram enhancement strategy; Represents the position in the image after fusion intensity mapping The pixel value at ; represents the enhancement parameter of the kth partition; Represents the kth partition.

[0026] Furthermore, the specific training process of the defect detection model includes:

[0027] Constructing a sample data set and dividing the sample data set into a training set and a validation set in proportion; wherein each sample in the sample data set includes a PCB image after reflection suppression processing and its defect label;

[0028] Build an improved YOLOv11 model;

[0029] The improved YOLOv11 model is trained using the training set, and the trained improved YOLOv11 model is verified using the verification set to obtain a defect detection model.

[0030] Furthermore, the backbone network includes a third convolutional layer, a second splicing layer, a fourth convolutional layer, a first C3K2-CA module, a fifth convolutional layer, a second C3K2-CA module, a sixth convolutional layer, a third C3K2-CA module, a seventh convolutional layer, a fourth C3K2-CA module, an SPPF module, a C2PSA module and an Attention mechanism connected in sequence;

[0031] The neck network includes a first upsampling module, a third splicing layer, a fifth C3K2-CA module, a second upsampling module, a fourth splicing layer, a sixth C3K2-CA module, a third upsampling module, a fifth splicing layer, a seventh C3K2-CA module, an eighth C3K2-CA module, an eighth convolutional layer, a sixth splicing layer, a ninth C3K2-CA module, a ninth convolutional layer, a seventh splicing layer and a tenth C3K2-CA module, which are connected in sequence;

[0032] The second convolutional layer is connected to the second splicing layer and the fifth splicing layer; the second C3K2-CA module is connected to the fourth splicing layer, and the third C3K2-CA module is connected to the third splicing layer; the Attention mechanism is connected to the first upsampling module and the seventh splicing layer, and the fifth C3K2-CA module is connected to the sixth splicing layer.

[0033] Furthermore, the CBAM module includes a channel attention module and a spatial attention module;

[0034] The channel attention module is used to integrate the spatial information of the input feature map through average pooling and maximum pooling operations to obtain two third intermediate feature maps; then, the two third intermediate feature maps are subjected to feature extraction by a shared multi-layer perceptron and then element-wise addition, and a sigmoid activation function is applied to obtain a channel attention map;

[0035] The spatial attention module is used to perform channel-based global maximum pooling and global average pooling operations on the channel attention map to obtain two fourth intermediate feature maps; then the two fourth intermediate feature maps are spliced ​​in the channel dimension, and then a convolution operation is performed to obtain a fifth intermediate feature map; and a sigmoid operation is performed on the fifth intermediate feature map to obtain a spatial attention feature map.

[0036] Furthermore, a pre-trained defect detection model is used to perform defect detection on the PCB image after the reflection suppression process, including:

[0037] The RGB channel branch module is used to perform channel separation, splicing, and feature extraction on the PCB image after reflection suppression processing to obtain the GB fusion feature map;

[0038] The first convolutional layer in the backbone network is used to extract features from the PCB image after reflection suppression processing to obtain a first intermediate feature map; the first splicing layer in the backbone network is used to splice the first intermediate feature map and the GB fusion feature map to obtain a second intermediate feature map; the rest of the backbone network is used to perform multi-scale feature extraction on the second intermediate feature map to obtain a first scale feature map, a second scale feature map, and a third scale feature map;

[0039] The neck network is used to fuse the low-level semantic features and high-level semantic features in the first-scale feature map, the second-scale feature map, and the third-scale feature map to obtain a first-scale fused feature map, a second-scale fused feature map, and a third-scale fused feature map; and during the fusion process, the GB fused feature map is spliced ​​with the feature map output by the third upsampling module in the neck network, so that the subsequent fusion process pays more attention to the GB channel features;

[0040] The first-scale fusion feature map, the second-scale fusion feature map, and the third-scale fusion feature map are respectively detected using a detection network to obtain a first-scale detection frame, a second-scale detection frame, and a third-scale detection frame.

[0041] Based on the same concept, the present invention also provides a PCB defect detection device, comprising:

[0042] An image processing unit, used for performing reflection suppression processing on PCB images;

[0043] A defect detection unit, configured to perform defect detection on the PCB image after the reflection suppression process using a pre-trained defect detection model;

[0044] The defect detection model adopts an improved YOLOv11 model. The improved YOLOv11 model is based on the original YOLOv11 model, adds an RGB channel branch module to the backbone network, and adopts a C3K2-CA module to replace the original C3K2 module;

[0045] The RGB channel branch module includes a first convolution layer, a first splicing layer, and a second convolution layer connected in sequence; the first convolution layer is used to separate the R, G, and B channels of the PCB image after the reflection suppression process; the first splicing layer is used to splice the G and B channel images output by the first convolution layer to obtain a GB spliced ​​image; the second convolution layer is used to extract features from the GB spliced ​​image to obtain a GB fusion feature map; wherein the GB fusion feature map is spliced ​​with the output features of the first convolution layer in the backbone network, and the GB fusion feature map is also spliced ​​with the features after secondary splicing in the neck network;

[0046] The C3K2-CA module is based on the original C3K2 module and adds an auxiliary branch. The auxiliary branch includes a CBAM module, a first RepConv module, a second RepConv module and a third RepConv module connected in sequence. The input end of the CBAM module is connected to the output end of the first convolutional layer in the C3K2 module, and the output ends of the CBAM module, the first RepConv module, the second RepConv module and the third RepConv module are respectively connected to the splicing layer in the C3K2 module.

[0047] Based on the same concept, the present invention also provides an electronic device, including a memory, a processor, and a computer program / instruction stored in the memory, wherein the processor executes the computer program / instruction to implement the PCB defect detection method as described above.

[0048] Based on the same concept, the present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon, which implements the above-mentioned PCB defect detection method when executed by a processor.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The defect detection model of the present invention improves the original YOLOv11 model. The first improvement is to add an RGB channel branch module in the backbone network. The RGB channel branch module is used to independently separate the three color channels of the PCB image, R, G, and B, and then the G and B channels are spliced ​​together. Then, features are extracted to obtain a GB fusion feature map, which makes the model pay more attention to the GB channel features, reduces the interference of the R channel, which has little influence in the PCB image, and enhances the model's ability to express color information and detail contrast. The model can effectively focus on the effective information area, breaking the traditional one-time convolution processing of the entire image. The model can learn more discernible features in different color channels, which is particularly suitable for defect detection tasks that are sensitive to color changes. The GB fusion feature map is not only spliced ​​into the backbone network, achieving complementation and synergy while retaining the significant features of each color channel, but also spliced ​​into the neck network, so that the subsequent convolution layers in the neck network continue to learn on the fused feature map, which is conducive to capturing contextual dependencies between channels.

[0051] The second improvement is the addition of an auxiliary branch connected in parallel to the original C3K2 module, forming a main-auxiliary dual-path structure. The main path (i.e., the C3K2 module) retains the traditional C3K module stacking method to enhance semantic modeling capabilities. The auxiliary branch introduces the CBAM module and the RepConv module to strengthen channel and spatial feature expression, thereby enhancing detail perception while maintaining context extraction capabilities. Through the Split-Parallel Processing-Concat fusion method, the model's modeling capabilities for defects, small objects, and weakly textured areas are effectively improved.

[0052] In the auxiliary branch, the CBAM module implements spatial and channel attention learning, guiding the model to focus on key features of the defect area. Simultaneously, the RepConv module is introduced to increase modeling depth during the training phase and reparameterize the structure during the inference phase, simplifying it to ordinary convolution, balancing accuracy and deployment efficiency. The synergy between the CBAM and RepConv modules achieves an optimized path of attention perception and efficient convolutional modeling, effectively enhancing the balance between detection accuracy and inference speed.

[0053] After the outputs of the main path and auxiliary branches are spliced ​​together, channel information is integrated through a unified convolutional layer to achieve end-to-end feature enhancement and compression. The introduction of the RepConv module also ensures that the training and inference structures are separable, with good engineering implementation capabilities. The auxiliary branches can be flexibly cropped, channel compressed, or directly integrated into the YOLOv series models, making them suitable for actual defect detection tasks that require both accuracy and speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 is a flow chart of a PCB defect detection method according to an embodiment of the present invention;

[0056] Figure 2 is a PCB image in an embodiment of the present invention;

[0057] Figure 3 is a PCB image after reflection suppression processing in an embodiment of the present invention;

[0058] Figure 4 2 is a diagram of the improved YOLOv11 model architecture in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following is a clear and complete description of the technical solutions of the present invention in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts are within the scope of protection of the present invention.

[0060] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0061] Example 1

[0062] Figure 1 The flowchart of the PCB defect detection method provided by the present invention is shown in FIG. Figure 1 As shown, the PCB defect detection method includes the following steps:

[0063] Step 1: Get the PCB image.

[0064] Use an image acquisition device, such as a camera, to shoot the PCB to obtain a PCB image, such as Figure 2 As shown, Figure 2 The PCB image in the image is reflective and the image quality is poor.

[0065] Step 2: Perform reflection suppression processing on the PCB image.

[0066] Current research generally believes that neural networks have powerful end-to-end learning capabilities and can automatically learn key features when the input image quality is limited. Therefore, they tend to "compensate" for the quality problems of the input image through complex structural stacking, resulting in the marginalization of the role of image enhancement technology in target detection. In particular, in industrial tasks, there is a lack of a systematic technical framework for the active processing of "reflection suppression and contrast equalization."

[0067] It is not easy to achieve efficient coupling of image enhancement technology and the YOLOv detection framework. If the image is over-processed, it will lead to side effects such as loss of image texture and blurred edges, which will reduce the detection accuracy; if the image is under-processed, it will not be able to improve the quality of the input image.

[0068] To improve image quality degradation caused by uneven lighting or reflections and achieve efficient coupling of image enhancement technology with the YOLOv detection framework, this paper employs an improved single-scale Retinex image enhancement algorithm (i.e., the R3SSR algorithm) to suppress reflections in PCB images. This provides higher-quality, higher-contrast input images for subsequent defect detection models, effectively improving detection sensitivity for minor defects. In a specific embodiment of the present invention, the reflection suppression process for PCB images includes:

[0069] Step 2.1: Calculate the reflection components of the R, G, and B channels respectively according to the PCB image, and merge the reflection components of the R, G, and B channels to obtain an enhanced image.

[0070] In this embodiment, the specific calculation formulas for the reflection components of the R, G, and B channels are:

[0071] (1)

[0072] (2)

[0073] (3)

[0074] in, 、 、 Respectively represent the R channel, G channel, and B channel in position The reflection component at ; 、 、 Respectively represent the R channel, G channel, and B channel in the PCB image at positions The pixel value at ; 、 、 Respectively represent the R channel, G channel, and B channel in position Gaussian filter kernel at .

[0075] The specific formula for merging the reflection components of R, G, and B channels is:

[0076] (4)

[0077] in, Indicates that an enhanced image is at position The pixel value at ; Represents a merge operation.

[0078] Step 2.2: Perform fused intensity mapping (FIM) on the primary enhanced image.

[0079] Fusion Intensity Mapping is coming soon Stretched to the range of 0~255, the specific formula is:

[0080] (5)

[0081] in, In the image after fusion intensity mapping, the c channel is at position The pixel value at ; Represents the weight of the c channel, which is used to control the response intensity of the corresponding color channel; Indicates that in an enhanced image, the c channel is at position The pixel value at ; Indicates the offset, used to prevent division by zero and make subsequent adjustments.

[0082] Step 2.3: Use partition-aware histogram enhancement (ZHE) to partition the image after fusion intensity mapping according to brightness, adopt differentiated enhancement strategies for different partitions, and fuse the enhanced partitions to obtain a secondary enhanced image, that is, the PCB image after reflection suppression processing.

[0083] The partition-aware histogram enhancement (ZHE) method improves the traditional CLAHE (i.e., limited contrast adaptive histogram equalization). Compared with the traditional CLAHE, the partition-aware histogram enhancement divides the input image into multiple sub-regions (i.e., partitions) according to brightness and assigns independent enhancement parameters to each sub-region. , and finally fuse the enhanced sub-regions. The specific formula of partition-aware histogram enhancement is:

[0084] (6)

[0085] (7)

[0086] in, Indicates location Whether the pixel value at belongs to the mask of the kth partition, 1, indicating the position The pixel value at belongs to the kth partition (hit); 0, indicating the position The pixel value at does not belong to the kth partition (miss); It represents the kth partition obtained by dividing the image after fusion intensity mapping according to brightness; Represents the position in the secondary enhanced image The pixel value at ; N represents the number of partitions; ZHE represents the partition-aware histogram enhancement strategy; Represents the position in the image after fusion intensity mapping The pixel value at ; represents the enhancement parameters of the kth partition, which include histogram clipping threshold, grid size and enhancement coefficient. Indicates the use of enhanced parameters for each partition For this partition Perform enhancement processing.

[0087] Partition-aware histogram enhancement adaptively adjusts contrast in different partitions based on image content, preserving more effective texture features and significantly improving object detection accuracy in complex lighting conditions. This not only improves detail recovery in reflective areas but also avoids the over-enhancement and information loss that can occur with traditional CLAHE using uniform parameters. Figure 3 Shows the PCB image after reflection suppression treatment, Figure 3 It can be seen that after the reflection suppression post-processing of the present invention, there is no reflection phenomenon in the PCB image, and the feature contrast is improved while retaining the original image features.

[0088] The R3SSR algorithm proposed in the present invention first adopts RGB three channels and a specific enhancement factor, which has better effect in actual reflection suppression; secondly, it performs fusion intensity mapping on the once enhanced image, adjusts the once enhanced image to the standard range, and performs brightness adaptation and numerical stability improvement processing; then, it uses partition-aware histogram enhancement to partition the image after fusion intensity mapping according to brightness, and adopts differentiated enhancement strategies for different areas, so that the image can highlight the features to the greatest extent without distortion; finally, the image with highlights removed and the features of the original image retained is obtained, which is better than the feature extraction of the original image.

[0089] The R3SSR algorithm of the present invention achieves precise suppression of local reflective areas and detail restoration. By performing independent illumination separation operations on each channel, it effectively alleviates the problems of color cast and detail loss in color images caused by the traditional single-scale Retinex image enhancement algorithm.

[0090] Step 3: Use the pre-trained defect detection model to perform defect detection on the PCB image after reflection suppression processing.

[0091] The defect detection model of the present invention adopts the improved YOLOv11 model, such as Figure 4 As shown in the figure, where Conv represents the convolutional layer, Concat represents the concatenation layer, SPPF represents Spatial Pyramid Pooling Fast, Attention represents the attention mechanism, Upsample represents upsampling, and Detect represents the detection head. The YOLOv11 model has two specific improvements. The first improvement is the addition of an RGB channel branching module (i.e., RGBSplit) to the backbone network. The RGB channel branching module consists of a first convolutional layer, a first concatenation layer, and a second convolutional layer connected in sequence. The first convolutional layer is used to separate the R, G, and B channels of the PCB image after reflection suppression processing. The first concatenation layer is used to concatenate the G and B channel images output by the first convolutional layer to obtain a GB-concatenated image. The second convolutional layer is used to extract features from the GB-concatenated image to obtain a GB-fused feature map. The GB-fused feature map is concatenated with the output features of the first convolutional layer (i.e., the third convolutional layer) in the backbone network. The GB-fused feature map is also concatenated with the features after secondary concatenation in the neck network (i.e., the features output by the third upsampling module).

[0092] PCB image features are primarily reflected in the green channel G, with minimal influence from the red channel R. Based on this characteristic of PCB images, the present invention introduces an RGB channel branching module. This module independently separates the three color channels of the PCB image after reflection suppression processing, then concatenates and extracts features from the G and B channels. This is equivalent to performing "feature extraction within the color space," enhancing the model's ability to express color information and detail contrast, reducing interference from the R channel, and making the feature information in the G and B channels more prominent. This structure effectively focuses on areas of significant information, breaking away from the traditional one-shot convolutional processing of the entire image. This allows the model to learn more discernible features from different color channels, making it particularly suitable for defect detection tasks sensitive to color variations (such as PCB defects and metal scratches). The GB fused feature map is not only spliced ​​into the backbone network (i.e., concatenated with the output features of the third convolutional layer), preserving the salient features of each color channel while achieving complementarity and synergy, but is also spliced ​​into the neck network (i.e., concatenated with the output features of the third upsampling module), enabling subsequent convolutional layers in the neck network to continue learning on the fused feature map, facilitating the capture of cross-channel contextual dependencies.

[0093] The second improvement is to use the C3K2-CA module to replace the original C3K2 module, that is, to add an auxiliary branch on the basis of the original C3K2 module. The auxiliary branch includes the CBAM module (Convolutional Block Attention Module), the first RepConv module, the second RepConv module and the third RepConv module connected in sequence. The input of the CBAM module is connected to the output of the first convolutional layer in the C3K2 module, and the outputs of the CBAM module, the first RepConv module, the second RepConv module and the third RepConv module are respectively connected to the splicing layer in the C3K2 module.

[0094] In this embodiment, the CBAM module includes a channel attention module and a spatial attention module. The channel attention module is used to integrate the spatial information of the input feature map through average pooling and maximum pooling operations to obtain two third intermediate feature maps. The two third intermediate feature maps are then extracted by a shared multi-layer perceptron and then element-by-element addition is performed. The channel attention map is obtained by applying a sigmoid activation function. The spatial attention module is used to perform channel-based global maximum pooling and global average pooling operations on the channel attention map to obtain two fourth intermediate feature maps. The two fourth intermediate feature maps are then spliced ​​in the channel dimension and convolved to obtain a fifth intermediate feature map. The fifth intermediate feature map is subjected to a sigmoid operation to obtain a spatial attention feature map.

[0095] The present invention adds an auxiliary branch in parallel with the original C3K2 module to form a main-auxiliary dual-path structure. The main path (i.e., the C3K2 module) retains the traditional C3K module stacking method to enhance the semantic modeling capability. The auxiliary branch introduces the CBAM module and the RepConv module to strengthen the channel and spatial feature expression, thereby enhancing detail perception while maintaining the context extraction capability; through the Split-parallel processing-Concat fusion form, the model's modeling capability for defects, small targets, and weak texture areas is effectively improved.

[0096] In the auxiliary branch, spatial + channel attention learning is implemented through the CBAM module to guide the model to focus on the key features of the defect area. At the same time, the RepConv module is introduced to improve the modeling depth in the training phase. In the inference phase, the structure can be reparameterized and simplified to ordinary convolution, taking into account both accuracy and deployment efficiency. Through the synergy of the CBAM module and the RepConv module, an optimized path of attention perception + efficient convolution modeling is achieved, effectively enhancing the balance between detection accuracy and inference speed.

[0097] After the outputs of the main path and auxiliary branches are spliced ​​together, channel information is integrated through a unified convolutional layer to achieve end-to-end feature enhancement and compression. The introduction of the RepConv module also ensures that the training and inference structures are separable, with good engineering implementation capabilities. The auxiliary branches can be flexibly cropped, channel compressed, or directly integrated into the YOLOv series models, making them suitable for actual defect detection tasks that require both accuracy and speed.

[0098] The present invention effectively enhances the model's ability to perceive color details and tiny defects through the RGB channel branch module, making it particularly suitable for PCB defect detection with obvious color differences. An auxiliary branch is added to the C3K2 module of the backbone network and the neck network. Through the CBAM and RepConv in the auxiliary branch, the model's spatial and channel attention expression capabilities are improved, while taking into account both diverse modeling during training and high efficiency during inference. The two improved points are synergistically optimized, achieving the unity of accurate detection of small targets and high-density detail areas and deployment friendliness for PCB defect detection.

[0099] like Figure 4 As shown in the figure, the backbone network includes the third convolutional layer, the second splicing layer, the fourth convolutional layer, the first C3K2-CA module, the fifth convolutional layer, the second C3K2-CA module, the sixth convolutional layer, the third C3K2-CA module, the seventh convolutional layer, the fourth C3K2-CA module, the SPPF module, the C2PSA module and the Attention mechanism, which are connected in sequence.

[0100] The neck network includes a first upsampling module, a third splicing layer, a fifth C3K2-CA module, a second upsampling module, a fourth splicing layer, a sixth C3K2-CA module, a third upsampling module, a fifth splicing layer, a seventh C3K2-CA module, an eighth C3K2-CA module, an eighth convolutional layer, a sixth splicing layer, a ninth C3K2-CA module, a ninth convolutional layer, a seventh splicing layer and a tenth C3K2-CA module, which are connected in sequence.

[0101] The second convolutional layer is connected to the second splicing layer and the fifth splicing layer; the second C3K2-CA module is connected to the fourth splicing layer, and the third C3K2-CA module is connected to the third splicing layer; the Attention mechanism is connected to the first upsampling module and the seventh splicing layer, and the fifth C3K2-CA module is connected to the sixth splicing layer.

[0102] In a specific embodiment of the present invention, the specific training process of the defect detection model includes:

[0103] Construct a sample dataset and divide it into a training set and a validation set in a ratio of 7:3. Each sample in the sample dataset includes a PCB image after reflection suppression processing and its defect label.

[0104] An improved YOLOv11 model is constructed, the improved YOLOv11 model is trained using the training set, and the trained improved YOLOv11 model is verified using the validation set to obtain a defect detection model.

[0105] In a specific embodiment of the present invention, defect detection is performed on a PCB image after reflection suppression using a pre-trained defect detection model, including:

[0106] Step 3.1: Use the RGB channel branch module to perform channel separation, splicing, and feature extraction on the PCB image after reflection suppression processing to obtain the GB fusion feature map;

[0107] Step 3.2: Use the first convolutional layer (i.e., the third convolutional layer) in the backbone network to extract features from the PCB image after reflection suppression, obtaining the first intermediate feature map. Use the first concatenation layer (i.e., the second concatenation layer) in the backbone network to concatenate the first intermediate feature map with the GB fusion feature map to obtain the second intermediate feature map. Use the rest of the backbone network (i.e., the fourth convolutional layer, the first C3K2-CA module, the fifth convolutional layer, the second C3K2-CA module, the sixth convolutional layer, the third C3K2-CA module, the seventh convolutional layer, the fourth C3K2-CA module, the SPPF module, the C2PSA module, and the Attention mechanism) to perform multi-scale feature extraction on the second intermediate feature map, obtaining the first, second, and third scale feature maps. The second C3K2-CA module outputs the first scale feature map, the third C3K2-CA module outputs the second scale feature map, and the Attention mechanism outputs the third scale feature map.

[0108] Step 3.3: Use the neck network to fuse the low-level semantic features and high-level semantic features in the first-scale feature map, the second-scale feature map, and the third-scale feature map to obtain the first-scale fused feature map, the second-scale fused feature map, and the third-scale fused feature map; and in the fusion process, splice the GB fusion feature map with the feature map output by the third upsampling module (i.e., the third upsampling module) in the neck network, so that the subsequent fusion process pays more attention to the GB channel features.

[0109] Specifically, after the first upsampling module upsamples the third scale feature map, the sixth intermediate feature map is obtained; the sixth intermediate feature map and the second scale feature map are spliced ​​using the third splicing layer to obtain the seventh intermediate feature map; the fifth C3K2-CA module is used to extract features from the seventh intermediate feature map to obtain the eighth intermediate feature map; after the second upsampling module upsamples the eighth intermediate feature map, the ninth intermediate feature map is obtained; the fourth splicing layer is used to splice the ninth intermediate feature map and the first scale feature map to obtain the tenth intermediate feature map; the sixth C3K2-CA module is used to extract features from the tenth intermediate feature map to obtain the eleventh intermediate feature map; after the third upsampling module upsamples the eleventh intermediate feature map, the twelfth intermediate feature map is obtained; the fifth splicing layer is used to extract features from the tenth intermediate feature map The twelfth intermediate feature map is concatenated with the GB fusion feature map to obtain the thirteenth intermediate feature map. The seventh C3K2-CA module, the eighth C3K2-CA module, and the eighth convolutional layer are sequentially used to extract features from the thirteenth intermediate feature map to obtain the fourteenth intermediate feature map. The sixth concatenation layer is used to concatenate the fourteenth intermediate feature map with the eighth intermediate feature map to obtain the fifteenth intermediate feature map. The ninth C3K2-CA module and the ninth convolutional layer are sequentially used to extract features from the fifteenth intermediate feature map to obtain the sixteenth intermediate feature map. The seventh concatenation layer is used to concatenate the sixteenth intermediate feature map with the third-scale feature map to obtain the seventeenth intermediate feature map. The tenth C3K2-CA module is used to extract features from the seventeenth intermediate feature map to obtain the third-scale fusion feature map. The intermediate feature map output by the ninth C3K2-CA module is the second-scale fusion feature map, and the intermediate feature map output by the eighth C3K2-CA module is the first-scale fusion feature map.

[0110] Step 3.4: Use the detection network to detect the first-scale fusion feature map, the second-scale fusion feature map, and the third-scale fusion feature map respectively to obtain the first-scale detection frame, the second-scale detection frame, and the third-scale detection frame.

[0111] The improved YOLOv11 model (i.e., YOLOv11-CA) in this paper has stronger color sensitivity and detail recognition capabilities, and can effectively capture tiny defects and weakly textured targets; the RGB channel branch module improves the color feature modeling effect, and the auxiliary branch composed of CBAM+RepConv enhances attention guidance and reasoning efficiency. The entire model achieves a good balance between accuracy and deployment performance, and is suitable for high-precision defect detection tasks.

[0112] To verify the effectiveness of the present invention, the PCB defect detection accuracy of the present invention (i.e., YOLOv11-CA + reflection suppression) was compared with that of the original YOLOv11 model. The comparison results are shown in Table 1. In Table 1, mAP@0.5 represents the mean average precision (mAP), which is the detection accuracy calculated with an intersection-over-union threshold of 0.5; mAP@0.5:0.95 represents the average mAP calculated with an intersection-over-union threshold ranging from 0.5 to 0.95; APs represents the mean average precision for small objects; APm represents the mean average precision for medium objects; and API represents the mean average precision for large objects. As shown in Table 1, compared with the original YOLOv11 model, the present invention improves small object detection accuracy by 11.4%. The detection accuracy of medium and large objects also improves significantly, demonstrating that the present invention effectively improves PCB defect detection accuracy, especially for small defects, and reduces the missed detection rate. Compared with the original YOLOv11 model, the reflection suppression processing of the present invention also significantly improves various accuracy indicators, demonstrating the effectiveness of the reflection suppression processing of the present invention.

[0113]

[0114] Example 2

[0115] The PCB defect detection device provided by the present invention includes an image processing unit and a defect detection unit. The image processing unit is used to suppress reflections on PCB images, and the defect detection unit is used to detect defects on the PCB images after reflection suppression using a pre-trained defect detection model.

[0116] The defect detection model of the present invention adopts the improved YOLOv11 model, such as Figure 4 As shown in the figure, the YOLOv11 model has two specific improvements. The first improvement is to add an RGB channel branch module (i.e., RGBSplit) to the backbone network. The RGB channel branch module includes a first convolutional layer, a first splicing layer, and a second convolutional layer connected in sequence. The first convolutional layer is used to separate the R, G, and B channels of the PCB image after reflection suppression processing. The first splicing layer is used to splice the G and B channel images output by the first convolutional layer to obtain a GB spliced ​​image. The second convolutional layer is used to extract features from the GB spliced ​​image to obtain a GB fusion feature map. The GB fusion feature map is spliced ​​with the output features of the first convolutional layer (i.e., the third convolutional layer) in the backbone network. The GB fusion feature map is also spliced ​​with the features after secondary splicing in the neck network (i.e., the features output by the third upsampling module).

[0117] The second improvement is to use the C3K2-CA module to replace the original C3K2 module, that is, to add an auxiliary branch on the basis of the original C3K2 module. The auxiliary branch includes the CBAM module (Convolutional Block Attention Module), the first RepConv module, the second RepConv module and the third RepConv module connected in sequence. The input of the CBAM module is connected to the output of the first convolutional layer in the C3K2 module, and the outputs of the CBAM module, the first RepConv module, the second RepConv module and the third RepConv module are respectively connected to the splicing layer in the C3K2 module.

[0118] In this embodiment, the CBAM module includes a channel attention module and a spatial attention module. The channel attention module is used to integrate the spatial information of the input feature map through average pooling and maximum pooling operations to obtain two third intermediate feature maps. The two third intermediate feature maps are then extracted by a shared multi-layer perceptron and then element-by-element addition is performed. The channel attention map is obtained by applying a sigmoid activation function. The spatial attention module is used to perform channel-based global maximum pooling and global average pooling operations on the channel attention map to obtain two fourth intermediate feature maps. The two fourth intermediate feature maps are then spliced ​​in the channel dimension and convolved to obtain a fifth intermediate feature map. The fifth intermediate feature map is subjected to a sigmoid operation to obtain a spatial attention feature map.

[0119] The present invention effectively enhances the model's ability to perceive color details and tiny defects through the RGB channel branch module, making it particularly suitable for PCB defect detection with obvious color differences. An auxiliary branch is added to the C3K2 module of the backbone network and the neck network. Through the CBAM and RepConv in the auxiliary branch, the model's spatial and channel attention expression capabilities are improved, while taking into account both diverse modeling during training and high efficiency during inference. The two improved points are synergistically optimized, achieving the unity of accurate detection of small targets and high-density detail areas and deployment friendliness for PCB defect detection.

[0120] In some specific embodiments of the present invention, the PCB defect detection device may be combined with the features of the PCB defect detection method in the first embodiment of the present invention, and vice versa, which will not be repeated here.

[0121] Example 3

[0122] An embodiment of the present invention further provides an electronic device, which includes: a memory, a processor, and a computer program / instructions stored in the memory, and the processor executes the computer program / instructions to implement the PCB defect detection method in the embodiment of the present invention.

[0123] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes based on programs and / or data stored in a read-only memory (ROM) or programs and / or data loaded from a storage portion into a random access memory (RAM). The processor can be a multi-core processor or can include multiple processors. In some embodiments, the processor can include a general-purpose main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. Various programs and data required for device operation are also stored in the RAM. The processor, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0124] The processor and memory are used together to execute the program / instructions stored in the memory. When the program / instructions are executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0125] Although not shown, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the PCB defect detection method in the embodiment of the present invention is implemented.

[0126] Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be used to store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in the present invention, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0127] The above disclosure is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or modifications within the technical scope disclosed in the present invention, and they should all be covered by the scope of protection of the present invention.

Claims

1. A PCB defect detection method, characterized in that: The detection method comprises: Perform reflection suppression processing on PCB images; Use the pre-trained defect detection model to perform defect detection on the PCB image after reflection suppression processing; The defect detection model adopts an improved YOLOv11 model. The improved YOLOv11 model is based on the original YOLOv11 model, adds an RGB channel branch module to the backbone network, and adopts a C3K2-CA module to replace the original C3K2 module; The RGB channel branch module includes a first convolution layer, a first splicing layer, and a second convolution layer connected in sequence; the first convolution layer is used to separate the R, G, and B channels of the PCB image after the reflection suppression process; the first splicing layer is used to splice the G and B channel images output by the first convolution layer to obtain a GB spliced ​​image; the second convolution layer is used to extract features from the GB spliced ​​image to obtain a GB fusion feature map; wherein the GB fusion feature map is spliced ​​with the output features of the first convolution layer in the backbone network, and the GB fusion feature map is also spliced ​​with the features after secondary splicing in the neck network; The C3K2-CA module is based on the original C3K2 module and adds an auxiliary branch. The auxiliary branch includes a CBAM module, a first RepConv module, a second RepConv module and a third RepConv module connected in sequence. The input end of the CBAM module is connected to the output end of the first convolutional layer in the C3K2 module, and the output ends of the CBAM module, the first RepConv module, the second RepConv module and the third RepConv module are respectively connected to the splicing layer in the C3K2 module.

2. The PCB defect detection method according to claim 1, characterized in that: The performing reflection suppression processing on the PCB image includes: According to the PCB image, the reflection components of the R, G, and B channels are calculated respectively, and the reflection components of the R, G, and B channels are merged to obtain an enhanced image. The specific calculation formulas of the reflection components of the R, G, and B channels are: ； ； ； in, 、 、 Respectively represent the R channel, G channel, and B channel in position The reflection component at ; 、 、 Respectively represent the R channel, G channel, and B channel in the PCB image at positions The pixel value at ; 、 、 Respectively represent the R channel, G channel, and B channel in position Gaussian filter kernel at ; The fusion intensity mapping is performed on the primary enhanced image. The specific formula is: ； in, In the image after fusion intensity mapping, the c channel is at position The pixel value at ; represents the weight of the c channel; Indicates that in an enhanced image, the c channel is at position The pixel value at ; Indicates the offset; The partition-aware histogram enhancement is used to perform partition masking on the image after fusion intensity mapping according to brightness, and differentiated enhancement strategies are adopted for different partitions. The enhanced partitions are fused to obtain a secondary enhanced image, that is, a PCB image after reflection suppression processing. The specific formula is: ； ； in, Represents the position in the secondary enhanced image The pixel value at ; N represents the number of partitions; Indicates location Whether the pixel value at belongs to the mask of the kth partition; ZHE represents the partition-aware histogram enhancement strategy; Represents the position in the image after fusion intensity mapping The pixel value at ; represents the enhancement parameter of the kth partition; Represents the kth partition.

3. The PCB defect detection method according to claim 1, wherein: The specific training process of the defect detection model includes: Constructing a sample data set and dividing the sample data set into a training set and a validation set in proportion; wherein each sample in the sample data set includes a PCB image after reflection suppression processing and its defect label; Build an improved YOLOv11 model; The improved YOLOv11 model is trained using the training set, and the trained improved YOLOv11 model is verified using the verification set to obtain a defect detection model.

4. The PCB defect detection method according to claim 1, wherein: The backbone network includes a third convolutional layer, a second splicing layer, a fourth convolutional layer, a first C3K2-CA module, a fifth convolutional layer, a second C3K2-CA module, a sixth convolutional layer, a third C3K2-CA module, a seventh convolutional layer, a fourth C3K2-CA module, an SPPF module, a C2PSA module and an Attention mechanism connected in sequence; The neck network includes a first upsampling module, a third splicing layer, a fifth C3K2-CA module, a second upsampling module, a fourth splicing layer, a sixth C3K2-CA module, a third upsampling module, a fifth splicing layer, a seventh C3K2-CA module, an eighth C3K2-CA module, an eighth convolutional layer, a sixth splicing layer, a ninth C3K2-CA module, a ninth convolutional layer, a seventh splicing layer and a tenth C3K2-CA module, which are connected in sequence; The second convolutional layer is connected to the second splicing layer and the fifth splicing layer; The second C3K2-CA module is connected to the fourth splicing layer, and the third C3K2-CA module is connected to the third splicing layer; the Attention mechanism is connected to the first upsampling module and the seventh splicing layer, and the fifth C3K2-CA module is connected to the sixth splicing layer.

5. The PCB defect detection method according to claim 1, wherein: The CBAM module includes a channel attention module and a spatial attention module; The channel attention module is used to integrate the spatial information of the input feature map through average pooling and maximum pooling operations to obtain two third intermediate feature maps; then, the two third intermediate feature maps are subjected to feature extraction by a shared multi-layer perceptron and then element-wise addition, and a sigmoid activation function is applied to obtain a channel attention map; The spatial attention module is used to perform channel-based global maximum pooling and global average pooling operations on the channel attention map to obtain two fourth intermediate feature maps; then the two fourth intermediate feature maps are spliced ​​in the channel dimension, and then a convolution operation is performed to obtain a fifth intermediate feature map; and a sigmoid operation is performed on the fifth intermediate feature map to obtain a spatial attention feature map.

6. The PCB defect detection method according to any one of claims 1 to 5, characterized in that: Use the pre-trained defect detection model to perform defect detection on the PCB image after reflection suppression, including: The RGB channel branch module is used to perform channel separation, splicing, and feature extraction on the PCB image after reflection suppression processing to obtain the GB fusion feature map; The first convolutional layer in the backbone network is used to extract features from the PCB image after reflection suppression processing to obtain a first intermediate feature map; the first splicing layer in the backbone network is used to splice the first intermediate feature map and the GB fusion feature map to obtain a second intermediate feature map; the rest of the backbone network is used to perform multi-scale feature extraction on the second intermediate feature map to obtain a first scale feature map, a second scale feature map, and a third scale feature map; The neck network is used to fuse the low-level semantic features and high-level semantic features in the first-scale feature map, the second-scale feature map, and the third-scale feature map to obtain a first-scale fused feature map, a second-scale fused feature map, and a third-scale fused feature map; and during the fusion process, the GB fused feature map is spliced ​​with the feature map output by the third upsampling module in the neck network, so that the subsequent fusion process pays more attention to the GB channel features; The first-scale fusion feature map, the second-scale fusion feature map, and the third-scale fusion feature map are respectively detected using a detection network to obtain a first-scale detection frame, a second-scale detection frame, and a third-scale detection frame.

7. A PCB defect detection device, characterized in that: The detection device comprises: An image processing unit, used for performing reflection suppression processing on PCB images; A defect detection unit, configured to perform defect detection on the PCB image after the reflection suppression process using a pre-trained defect detection model; The defect detection model adopts an improved YOLOv11 model. The improved YOLOv11 model is based on the original YOLOv11 model, adds an RGB channel branch module to the backbone network, and adopts a C3K2-CA module to replace the original C3K2 module; The RGB channel branch module includes a first convolution layer, a first splicing layer, and a second convolution layer connected in sequence; the first convolution layer is used to separate the R, G, and B channels of the PCB image after the reflection suppression process; the first splicing layer is used to splice the G and B channel images output by the first convolution layer to obtain a GB spliced ​​image; the second convolution layer is used to extract features from the GB spliced ​​image to obtain a GB fusion feature map; wherein the GB fusion feature map is spliced ​​with the output features of the first convolution layer in the backbone network, and the GB fusion feature map is also spliced ​​with the features after secondary splicing in the neck network; The C3K2-CA module is based on the original C3K2 module and adds an auxiliary branch. The auxiliary branch includes a CBAM module, a first RepConv module, a second RepConv module and a third RepConv module connected in sequence. The input end of the CBAM module is connected to the output end of the first convolutional layer in the C3K2 module, and the output ends of the CBAM module, the first RepConv module, the second RepConv module and the third RepConv module are respectively connected to the splicing layer in the C3K2 module.

8. An electronic device comprising a memory, a processor, and a computer program / instruction stored in the memory, characterized in that: The processor executes the computer program / instructions to implement the PCB defect detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the PCB defect detection method according to any one of claims 1 to 6 is implemented.

Images