Printed circuit board defect detection method and system based on image recognition

Abstract

The invention relates to the technical field of printed circuit board design, in particular to a printed circuit board defect detection method and system based on image recognition. The method comprises the following steps: carrying out image preprocessing on an original PCB image to obtain a preprocessed PCB image; performing scale feature fusion on the preprocessed PCB image to obtain a multi-scale feature map; performing background texture feature learning on the multi-scale feature map to obtain a background suppression feature map; performing background texture suppression according to the background suppression feature map to obtain a significant defect map; and performing local statistical feature calculation according to the pre-processed PCB image and the saliency defect graph to obtain a local statistical feature graph. Through a series of technical means of multi-scale feature fusion, background texture feature learning and suppression, adaptive threshold segmentation and morphological optimization, the problem of complex background texture interference is effectively solved, and the accuracy and robustness of PCB defect detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of printed circuit board design, and in particular to a printed circuit board defect detection method and system based on image recognition. Background Art

[0002] At present, PCB circuits are becoming more and more dense, and the difficulty and requirements of defect detection are becoming higher and higher. The traditional method based on manual visual inspection can no longer meet the needs of modern industrial production, so the automatic optical inspection (AOI) technology based on image recognition has emerged. The early methods were mainly based on template matching, comparing the image to be inspected with a defect-free standard template, and judging whether there are defects by pixel differences. This method is simple and easy to implement, but it is sensitive to image alignment and lighting changes, has poor robustness, and is difficult to deal with complex background textures.

[0003] In PCB images, some background textures (such as pad edges, small wires, etc.) and defect features (such as open circuits, short circuits, pinholes, etc.) have similar texture features in the image, making them difficult to distinguish. Different PCB designs, manufacturing processes, and lighting conditions will cause changes in background textures, increasing the difficulty of detection. Most existing methods focus on extracting defect features, while ignoring the modeling and suppression of complex background textures, resulting in greater interference from background textures on defect detection. Due to the complexity and diversity of background textures, existing methods are often not robust enough when facing different types of background textures, and are prone to false detection and missed detection. Summary of the invention

[0004] Based on this, it is necessary to provide a printed circuit board defect detection method and system based on image recognition to solve at least one of the above technical problems.

[0005] To achieve the above object, a printed circuit board defect detection method based on image recognition comprises the following steps: Step S1: performing image preprocessing on the original PCB image to obtain a preprocessed PCB image; performing scale feature fusion on the preprocessed PCB image to obtain a multi-scale feature map; Step S2: performing background texture feature learning on the multi-scale feature map to obtain a background suppression feature map; performing background texture suppression according to the background suppression feature map to obtain a significant defect map; Step S3: Calculate local statistical features based on the preprocessed PCB image and the significant defect map to obtain a local statistical feature map; calculate an adaptive threshold based on the local statistical feature map, and perform binary segmentation to obtain a candidate defect area; Step S4: performing preliminary hole detection on the candidate defect area to obtain a preliminary hole area; performing distance transformation calculation on the preliminary hole area to obtain a distance transformation map; performing geodesic transformation calculation on the distance transformation map to obtain a geodesic distance map; constructing a geodesic reconstruction map using the geodesic distance map and the distance transformation map to obtain a geodesic reconstruction map; refining the defect area on the geodesic reconstruction map to obtain a refined defect area; Step S5: classify the refined defect area to obtain defect category data; locate the defect according to the defect category data to obtain defect position and size data; integrate the defect information of the defect category data and the defect position and size data to obtain defect detection results to realize the printed circuit board defect detection operation.

[0006] The present invention enhances image quality and consistency through image preprocessing, laying the foundation for subsequent processing; scale feature fusion combines local texture information of wavelet transform and global semantic information of deep learning, constructs a more representative multi-scale feature map, and improves the robustness and accuracy of defect detection. By learning background texture features through the attention mechanism and guiding feature learning, the interference of complex background textures is effectively suppressed, the defect area is highlighted, and the final significant defect map makes the defect more significant, improving the sensitivity of defect detection. The adaptive threshold calculation method based on local statistical features can dynamically adjust the segmentation threshold according to the characteristics of different regions, effectively overcomes the limitations of the fixed threshold method under complex backgrounds, improves the accuracy of defect segmentation, and obtains more accurate candidate defect areas. Through a series of morphological operations, including noise removal, broken connection, hole detection, distance transformation, geodesic transformation and reconstruction, the defect area is effectively refined, the hole is filled, the edge is smoothed, the noise is removed, and finally more accurate defect shape and location information is obtained. Through feature extraction, classifier training and defect location, automatic classification and precise location of defects are achieved, and the category, location and size information of the defects are integrated, and finally structured defect detection results are output, providing effective data support for PCB production quality control. Therefore, the present invention provides a printed circuit board defect detection method based on image recognition, which solves the shortcomings of existing methods that are difficult to deal with complex background textures. Through a series of technical means such as multi-scale feature fusion, background texture feature learning and suppression, adaptive threshold segmentation, and morphological optimization, the problem of complex background texture interference is effectively solved, and the accuracy and robustness of PCB defect detection are improved. The invention method has important theoretical significance and practical application value, and provides effective technical support for quality control and efficiency improvement in the electronics manufacturing industry, so as to achieve more efficient and intelligent PCB defect detection.

[0007] Preferably, step S1 comprises the following steps: Step S11: performing image preprocessing on the original PCB image to obtain a preprocessed PCB image; Step S12: performing multi-scale wavelet transform on the pre-processed PCB image to obtain multi-scale wavelet coefficients; Step S13: Use a pre-trained deep convolutional neural network to extract deep features from the pre-processed PCB image to obtain a deep feature map; Step S14: reconstructing the multi-scale wavelet coefficients to obtain a reconstructed wavelet feature map; Step S15: upsampling the depth feature map according to the reconstructed wavelet feature map to obtain an upsampled depth feature map; Step S16: performing feature fusion on the reconstructed wavelet feature map and the upsampled depth feature map to obtain a multi-scale feature map.

[0008] The present invention effectively removes image noise through Gaussian filtering through image preprocessing, thereby improving the robustness of subsequent feature extraction; histogram equalization enhances image contrast, highlights detail features in PCB images, and is beneficial to defect identification; scaling the image to a fixed size unifies the specifications of the input image, reduces the amount of calculation, and improves algorithm efficiency. Multiscale wavelet transform decomposes the image into sub-bands of different frequencies, respectively capturing the contour information and detail texture information of the image, thereby realizing the extraction of multi-scale features of the PCB image and providing rich feature representation for subsequent feature fusion. The use of a pre-trained deep convolutional neural network to extract deep feature maps can effectively capture high-level semantic information and global features of PCB images, such as component layout, line direction, etc. These features help to distinguish defects from normal textures. Wavelet coefficient reconstruction recombines the coefficients after wavelet transform into an image form, retains multi-scale features, and expresses image information in a more intuitive way, facilitating fusion with deep feature maps. Feature map upsampling adjusts the size of the deep feature map to match the reconstructed wavelet feature. Figure 1 This ensures the alignment of the two feature maps in the spatial dimension, creating conditions for subsequent feature fusion. Feature fusion combines the local texture information of the wavelet feature map with the global semantic information of the deep feature map to form a more comprehensive multi-scale feature representation, improving the accuracy and robustness of defect detection.

[0009] Preferably, step S2 comprises the following steps: Step S21: constructing an attention module for background texture on the multi-scale feature map to obtain an attention weight map; Step S22: performing attention-guided feature learning on the attention weight map and the multi-scale feature map to obtain a background suppression feature map; Step S23: performing residual connection and feature fusion on the multi-scale feature map and the background suppression feature map to obtain a fused feature map; Step S24: Perform nonlinear activation and normalization on the fused feature map to obtain a significant defect map.

[0010] The present invention generates an attention weight map by learning the background texture features in the multi-scale feature map through the attention module. The weight map highlights the background texture area and provides guidance for subsequent background suppression. Attention-guided feature learning uses the attention weight map to weight the multi-scale feature map, effectively suppresses the feature response of the background texture area, and highlights the features of the potential defect area, thereby improving the sensitivity of defect detection. The residual connection fuses the original multi-scale feature map with the background suppression feature map, retains the original feature information, avoids information loss, and enhances the learning ability of the network; feature fusion further integrates the information of the two feature maps and improves the integrity of feature representation. The nonlinear activation function ReLU introduces nonlinearity and enhances the expressive power of the model; the normalization operation maps the eigenvalues ​​to a unified range, improves the stability and robustness of the algorithm, and the final significant defect map highlights the potential defect area, providing a reliable basis for subsequent defect segmentation.

[0011] Preferably, step S3 comprises the following steps: Step S31: determining the size of the local area according to the pre-processed PCB image and the significant defect map to obtain the local area size data; Step S32: Calculate the local statistical features of the sliding window of the significant defect map according to the local area size data to obtain a local statistical feature map; Step S33: performing adaptive threshold calculation according to the local statistical feature map to obtain an adaptive threshold map, wherein the local statistical feature map includes a local histogram, a local mean map, and a local standard deviation map; Step S34: Binarize and segment the adaptive threshold map and the significant defect map to obtain candidate defect regions.

[0012] The present invention can adaptively adjust the size of the local area to match defects of different sizes by calculating the average size of the connected area according to the gradient information of the significant defect map, and determining the size of the local area in combination with the proportional factor, thereby improving the adaptability of the algorithm to different defect types. The sliding window calculates the local histogram, mean and standard deviation, which can capture the statistical characteristics of the local area. These characteristics can effectively distinguish the defect area from the background area, providing a basis for the subsequent adaptive threshold calculation. The adaptive threshold is calculated using the local histogram, mean and standard deviation, and the segmentation threshold can be dynamically adjusted according to the statistical characteristics of the local area, avoiding the limitations of the fixed threshold segmentation method and improving the accuracy of segmentation, especially in the case of complex background. Comparing the significant defect map with the adaptive threshold map and performing binary segmentation can effectively separate the potential defect area from the background, generate candidate defect areas, and provide input for subsequent morphological optimization.

[0013] Preferably, step S33 includes the following steps: Step S331: performing local Otsu threshold calculation on the local histogram to obtain a local Otsu threshold list; Step S332: converting the local Otsu threshold list into a threshold map according to the local area size data, and performing threshold smoothing to obtain a smoothed local Otsu threshold map; Step S333: performing local contrast calculation according to the local mean map and the local standard deviation map to obtain a local contrast map; Step S334: performing contrast normalization on the local contrast map to obtain a normalized local contrast map; Step S335: Calculate the threshold adjustment weight using the normalized local contrast map to obtain a threshold adjustment weight map; Step S336: Using the threshold adjustment weight map, threshold adjustment is performed on the smoothed local Otsu threshold map to obtain an adaptive threshold map.

[0014] The present invention can obtain a series of local optimal thresholds by applying the Otsu algorithm to each local histogram. These thresholds preliminarily reflect the grayscale distribution characteristics of each local area, laying the foundation for the subsequent generation of an adaptive threshold map. Converting the local Otsu threshold list into a threshold map and performing Gaussian smoothing can effectively remove noise interference and make the threshold map smoother, thereby avoiding the phenomenon of over-segmentation in the binary segmentation result and improving the stability of the segmentation. Calculating the local contrast map can reflect the texture characteristics of the local area of ​​the image. The area with high contrast usually corresponds to the defect or edge area, which provides a basis for adjusting the threshold according to the contrast. Contrast normalization maps the contrast value to between 0 and 1, eliminates the difference in contrast values ​​between different areas, and makes the subsequent threshold adjustment weight calculation more standardized. Calculating the threshold adjustment weight by an exponential function can amplify the influence of contrast on the threshold, so that the threshold adjustment amplitude of the area with high contrast (such as the defect area) is larger, thereby segmenting the defect more accurately. The smoothed local Otsu threshold map is multiplied by the threshold adjustment weight map to obtain the final adaptive threshold map, which comprehensively considers the local grayscale distribution and local contrast information and can more accurately reflect the boundary of the defect area, thereby improving the accuracy of binary segmentation.

[0015] Preferably, step S4 comprises the following steps: Step S41: removing noise from the candidate defect region to obtain a denoised candidate region; connecting the broken defects in the denoised candidate region to obtain a connected defect region; Step S42: performing preliminary void detection on the defective area after connection to obtain a preliminary void area; Step S43: performing distance transformation calculation on the preliminary hole area to obtain a distance transformation map; Step S44: marking the hole boundary of the preliminary hole area to obtain a hole boundary map; performing geodesic transformation calculation on the distance transformation map using the hole boundary map as a mark to obtain a geodesic distance map; Step S45: constructing a geodesic reconstruction map using the geodesic distance map and the distance transformation map to obtain a geodesic reconstruction map; Step S46: performing region merging and filling on the geodesic reconstruction image and the connected defect region to obtain a filled defect region; Step S47: performing edge smoothing on the filled defective region to obtain a smoothed defective region; performing region screening on the smoothed defective region to obtain a refined defective region.

[0016] The present invention effectively removes noise in the candidate defect area through median filtering, reducing false detection; the morphological closing operation connects the broken defect area, making the defect area more complete, which is conducive to subsequent hole detection and defect refinement. Through connected region analysis and Euler number calculation, the holes in the connected defect area can be effectively identified and marked, providing the target area for the subsequent distance transformation and geodesic transformation. The distance transformation calculates the distance from the pixel in each hole area to the nearest pixel in the non-hole area, converts the shape information of the hole into distance information, and provides basic data for subsequent geodesic transformation and reconstruction. The hole boundary marking accurately determines the outline of the hole; the geodesic transformation calculates the shortest path along the hole surface, which can more accurately describe the shape of the hole and provide key information for geodesic reconstruction. Geodesic reconstruction uses the geodesic distance map and the distance transformation map to propagate the hole boundary information to the inside of the hole, effectively filling the hole area while retaining the original shape of the hole, providing more accurate shape information for the refinement of the defect area. The geodesic reconstruction map is merged with the connected defect area to fill the hole area, making the defect area more complete and preparing for the subsequent edge smoothing and area screening. Gaussian filtering smoothes the edges of the filled defect area, removes the jagged edges, and makes the outline of the defect area clearer; area screening removes areas with too small an area, which are usually noise or artifacts, thereby improving the accuracy of defect detection.

[0017] Preferably, step S43 includes the following steps: Step S431: performing defect region inversion on the preliminary void region to obtain an inverted defect region; Step S432: using a pre-selected distance measurement method to calculate an initial distance map of the defective area after inversion to obtain an initial distance map; Step S433: normalizing the initial distance map to obtain a normalized distance map; Step S434: performing distance map enhancement on the normalized distance map to obtain a distance transformation map.

[0018] The present invention inverts the preliminary hole area, converts the hole area into the background, and converts the background area into the foreground, which makes it possible to use a standard distance transformation algorithm to calculate the distance from each pixel to the nearest original hole area pixel, simplifying the calculation process. The initial distance map is calculated using the Euclidean distance metric method, which can accurately reflect the distance from each pixel to the nearest original hole area pixel, providing basic data for subsequent normalization and enhancement operations. Distance map normalization scales the distance value to between 0 and 1, eliminating the difference in distance values ​​between different holes, making subsequent distance enhancement operations more standardized, and facilitating comparisons between different images. Distance map enhancement highlights the change in distance through power law transformation, making pixel values ​​with larger distances more obvious, thereby enhancing the boundary information of the hole area, which is beneficial to subsequent geodesic transformation and reconstruction, and improving the accuracy of defect area refinement.

[0019] Preferably, step S45 includes the following steps: Step S451: constructing an initial reconstruction image according to the distance transformation image to obtain an initial reconstruction image; Step S452: determining a dynamic threshold coefficient according to the geodesic distance map to obtain a dynamic threshold coefficient; Step S453: traverse each pixel in the distance transformation map, and multiply the distance value of the corresponding pixel in the geodesic distance map by the dynamic threshold coefficient to obtain a dynamic threshold; Step S454: comparing pixel values ​​according to the distance transformation map and the dynamic threshold, and labeling the initial reconstructed map to obtain a labeled map; Step S455: Perform morphological processing on the labeled image to obtain a geodesic reconstruction image.

[0020] The present invention provides initial data for subsequent iterative reconstruction operations by copying the distance transform map as the initial reconstruction map, and ensures that the size of the reconstructed image is consistent with the original image. The dynamic threshold coefficient is determined according to the statistical characteristics (mean and standard deviation) of the geodesic distance map, and the sensitivity of the reconstruction process can be adaptively adjusted to make the reconstruction result more in line with the shape of the actual hole. The dynamic threshold is calculated for each pixel, and the geodesic distance information is integrated into the reconstruction process, which provides a basis for subsequent pixel value comparison and marking. By comparing the distance transform map and the dynamic threshold, and marking the initial reconstruction map, the geodesic distance information is propagated to the distance transform map, and the reconstructed image is gradually constructed, so that the filling of the hole area is more accurate. The morphological closing operation smoothes the reconstruction result, removes small noise and artifacts, and connects possible fracture areas, so that the final geodesic reconstruction map is more complete and accurate, and more accurately reflects the shape and size of the hole.

[0021] Preferably, step S5 comprises the following steps: Step S51: extracting features from the refined defect area to obtain a defect feature vector; Step S52: performing lightweight classifier training according to the defect feature vector and the preset defect label to obtain a defect classifier; Step S53: using a defect classifier to classify the defect feature vector to obtain defect category data; Step S54: locating the defect according to the refined defect area and defect category data to obtain defect position and size data; Step S55: Integrate the defect information of the defect category data and the defect position and size data to obtain the defect detection result.

[0022] The present invention can comprehensively describe the geometric shape and grayscale characteristics of the defect by extracting multiple features of the refined defect area, including area, perimeter, circularity, Hu moment, grayscale mean and grayscale standard deviation, and provide effective feature representation for subsequent defect classification. Using a pre-labeled data set to train a lightweight classifier (such as SVM or random forest) can effectively learn the mapping relationship between defect features and defect categories, and realize automatic classification of different types of defects. Using the trained classifier to classify the defect feature vector, the refined defect area can be divided into different defect categories, such as open circuit, short circuit, missing component, etc., to provide category information for subsequent defect analysis and processing. Calculating the minimum circumscribed rectangle of the refined defect area can accurately obtain the position coordinates and size information of the defect, and provide accurate positioning information for subsequent defect repair or other processing. The defect category, position and size information are integrated together to form the final defect detection result, and the detection result is output in a structured manner, which is convenient for subsequent analysis, processing and recording, and improves the practicality of defect detection.

[0023] Preferably, the present invention further provides a printed circuit board defect detection system based on image recognition, which is used to execute the printed circuit board defect detection method based on image recognition as described above, and the printed circuit board defect detection system based on image recognition comprises: The multi-scale feature fusion module is used to perform image preprocessing on the original PCB image to obtain a preprocessed PCB image; perform scale feature fusion on the preprocessed PCB image to obtain a multi-scale feature map; A background texture suppression module is used to perform background texture feature learning on a multi-scale feature map to obtain a background suppression feature map; background texture suppression is performed based on the background suppression feature map to obtain a significant defect map; The adaptive threshold segmentation module is used to calculate local statistical features based on the pre-processed PCB image and the significant defect map to obtain a local statistical feature map; perform adaptive threshold calculation based on the local statistical feature map, and perform binary segmentation to obtain a candidate defect area; The morphological optimization module is used to perform preliminary hole detection based on the candidate defect area to obtain a preliminary hole area; perform distance transformation calculation on the preliminary hole area to obtain a distance transformation map; perform geodesic transformation calculation on the distance transformation map to obtain a geodesic distance map; use the geodesic distance map and the distance transformation map to construct a geodesic reconstruction map to obtain a geodesic reconstruction map; refine the defect area on the geodesic reconstruction map to obtain a refined defect area; The defect classification and positioning module is used to classify the refined defect area and obtain defect category data; locate the defect according to the defect category data to obtain defect position and size data; integrate the defect information of the defect category data and the defect position and size data to obtain the defect detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of the steps of a printed circuit board defect detection method based on image recognition; Figure 2 for Figure 1 Detailed implementation steps of step S3 in FIG. Figure 3 for Figure 1 Detailed implementation steps of step S4 in FIG.

[0025] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0026] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0027] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different network and / or processor methods and / or microcontroller methods.

[0028] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0029] To achieve this, please refer to Figures 1 to 3 , a printed circuit board defect detection method based on image recognition, comprising the following steps: Step S1: performing image preprocessing on the original PCB image to obtain a preprocessed PCB image; performing scale feature fusion on the preprocessed PCB image to obtain a multi-scale feature map; Step S2: performing background texture feature learning on the multi-scale feature map to obtain a background suppression feature map; performing background texture suppression according to the background suppression feature map to obtain a significant defect map; Step S3: Calculate local statistical features based on the preprocessed PCB image and the significant defect map to obtain a local statistical feature map; calculate an adaptive threshold based on the local statistical feature map, and perform binary segmentation to obtain a candidate defect area; Step S4: performing preliminary hole detection on the candidate defect area to obtain a preliminary hole area; performing distance transformation calculation on the preliminary hole area to obtain a distance transformation map; performing geodesic transformation calculation on the distance transformation map to obtain a geodesic distance map; constructing a geodesic reconstruction map using the geodesic distance map and the distance transformation map to obtain a geodesic reconstruction map; refining the defect area on the geodesic reconstruction map to obtain a refined defect area; Step S5: classify the refined defect area to obtain defect category data; locate the defect according to the defect category data to obtain defect position and size data; integrate the defect information of the defect category data and the defect position and size data to obtain defect detection results to realize the printed circuit board defect detection operation.

[0030] In the embodiment of the present invention, reference Figure 1 FIG. 1 is a schematic diagram of the steps of the printed circuit board defect detection method based on image recognition of the present invention. In this example, the printed circuit board defect detection method based on image recognition includes the following steps: Step S1: performing image preprocessing on the original PCB image to obtain a preprocessed PCB image; performing scale feature fusion on the preprocessed PCB image to obtain a multi-scale feature map; In the embodiment of the present invention, in the image preprocessing stage, the original PCB image is first subjected to Gaussian filtering for denoising, then subjected to histogram equalization for contrast enhancement, and finally scaled to a fixed size. In the scale feature fusion stage, the preprocessed image is first subjected to multi-scale wavelet transform, and the wavelet coefficients are reconstructed to obtain a reconstructed wavelet feature map; at the same time, a pre-trained ResNet-50 network is used to extract a depth feature map, and the map is upsampled to the same size as the wavelet feature map. Finally, the reconstructed wavelet feature map and the upsampled depth feature map are spliced ​​in the channel dimension, and then 1 1 Convolution is used to perform feature fusion and obtain a multi-scale feature map.

[0031] Step S2: performing background texture feature learning on the multi-scale feature map to obtain a background suppression feature map; performing background texture suppression according to the background suppression feature map to obtain a significant defect map; In the embodiment of the present invention, the attention module is used to learn the background texture features and generate an attention weight map. The attention weight map is multiplied with the multi-scale feature map to suppress the background texture and obtain a background suppression feature map. The multi-scale feature map and the background suppression feature map are residually connected and 1 1 Convolution fusion features. Finally, the fused feature map is ReLU activated and normalized to obtain the significant defect map.

[0032] Step S3: Calculate local statistical features based on the preprocessed PCB image and the significant defect map to obtain a local statistical feature map; calculate an adaptive threshold based on the local statistical feature map, and perform binary segmentation to obtain a candidate defect area; In an embodiment of the present invention, the size of the local region is calculated according to the gradient amplitude of the significant defect map. The local histogram, local mean and local standard deviation are calculated on the significant defect map using a sliding window to form a local statistical feature map. Based on the local statistical feature map, the initial threshold is calculated using the Otsu algorithm, and adaptive adjustment is performed according to the local mean and standard deviation to obtain an adaptive threshold map. Finally, the significant defect map is compared with the adaptive threshold map, and binary segmentation is performed to obtain a candidate defect region.

[0033] Step S4: performing preliminary hole detection on the candidate defect area to obtain a preliminary hole area; performing distance transformation calculation on the preliminary hole area to obtain a distance transformation map; performing geodesic transformation calculation on the distance transformation map to obtain a geodesic distance map; constructing a geodesic reconstruction map using the geodesic distance map and the distance transformation map to obtain a geodesic reconstruction map; refining the defect area on the geodesic reconstruction map to obtain a refined defect area; In the embodiment of the present invention, median filtering and morphological closing operations are performed on the candidate defect regions to connect the broken defects. The preliminary void region is detected by connected region analysis and Euler number calculation. The preliminary void region is subjected to distance transformation to obtain a distance transformation map. Then, geodesic transformation is performed to obtain a geodesic distance map. The distance transformation map and the geodesic distance map are used to perform geodesic reconstruction to obtain a geodesic reconstruction map. Finally, the reconstructed map is merged with the connected defect region, and Gaussian filtering and area screening are performed to obtain a refined defect region.

[0034] Step S5: classify the refined defect area to obtain defect category data; locate the defect according to the defect category data to obtain defect position and size data; integrate the defect information of the defect category data and the defect position and size data to obtain defect detection results, so as to realize the printed circuit board defect detection operation; In the embodiment of the present invention, the features of the refined defect area, including area, perimeter, circularity, Hu moment, grayscale mean and grayscale standard deviation, are extracted to form a defect feature vector. A lightweight classifier (such as SVM or random forest) is trained using a pre-labeled data set. The defect feature vector is classified using the trained classifier to obtain defect category data. The minimum enclosing rectangle of each defect area is calculated to obtain the position and size information. Finally, the defect category, position and size information are integrated to output the final defect detection result.

[0035] Preferably, step S1 comprises the following steps: Step S11: performing image preprocessing on the original PCB image to obtain a preprocessed PCB image; Step S12: performing multi-scale wavelet transform on the pre-processed PCB image to obtain multi-scale wavelet coefficients; Step S13: Use a pre-trained deep convolutional neural network to extract deep features from the pre-processed PCB image to obtain a deep feature map; Step S14: reconstructing the multi-scale wavelet coefficients to obtain a reconstructed wavelet feature map; Step S15: upsampling the depth feature map according to the reconstructed wavelet feature map to obtain an upsampled depth feature map; Step S16: performing feature fusion on the reconstructed wavelet feature map and the upsampled depth feature map to obtain a multi-scale feature map.

[0036] In the embodiment of the present invention, the original PCB image is first read, which is usually an RGB color image or a grayscale image. The image data is read using the OpenCV library and converted into a NumPy array format. Subsequently, a Gaussian filter is applied to smooth the image to remove high-frequency noise and retain the edge information of the image. The convolution kernel size of the Gaussian filter is set to 5. 5, and the standard deviation is set to 1.5. Next, histogram equalization is performed to enhance the image contrast, making the grayscale distribution of the image more uniform and highlighting the details. Finally, in order to reduce the amount of calculation and unify the image size, the image is scaled to a fixed size, such as 512 512 pixels. The scaling algorithm uses bilinear interpolation to ensure image quality.

[0037] The PyWavelets library is used to perform multi-scale wavelet transform on the preprocessed PCB image. The 'db4' wavelet basis function is selected for three-level decomposition to obtain the wavelet coefficients of the low-frequency subband and the high-frequency subband respectively. The low-frequency subband mainly contains the contour information of the image, while the high-frequency subband contains the details and texture information of the image. The wavelet coefficients obtained at each level of decomposition are stored as NumPy arrays for subsequent processing.

[0038] Load a pre-trained deep convolutional neural network model, such as ResNet-50, which has been trained on the ImageNet dataset and has strong feature extraction capabilities. Use a deep learning framework such as TensorFlow or PyTorch to load the model. Remove the classification layer of the model and input the pre-processed PCB image into the model for forward propagation. Extract the output of the last convolutional layer of the model as a deep feature map. The deep feature map captures the high-level semantic information of the PCB image.

[0039] The multi-scale wavelet coefficients are reconstructed using the PyWavelets library. The same wavelet basis function 'db4' and decomposition level are used for inverse wavelet transform. The reconstructed wavelet feature map is stored as a NumPy array, which contains the multi-scale texture information of the PCB image.

[0040] The deep feature map is upsampled using bilinear interpolation. The upsampled target size is the same as the wavelet feature map size. The upsampling operation enlarges the size of the deep feature map so that it is aligned with the wavelet feature map in the spatial dimension.

[0041] The reconstructed wavelet feature map and the upsampled depth feature map are concatenated in the channel dimension. The concatenated feature map contains the multi-scale wavelet features and deep semantic features of the PCB image. Subsequently, a 1×1 convolution is used to reduce the channel dimension of the concatenated feature map to reduce the amount of calculation and fuse different types of features. The resulting multi-scale feature map will be used for subsequent background texture feature learning.

[0042] Preferably, step S2 comprises the following steps: Step S21: constructing an attention module for background texture on the multi-scale feature map to obtain an attention weight map; Step S22: performing attention-guided feature learning on the attention weight map and the multi-scale feature map to obtain a background suppression feature map; Step S23: performing residual connection and feature fusion on the multi-scale feature map and the background suppression feature map to obtain a fused feature map; Step S24: Perform nonlinear activation and normalization on the fused feature map to obtain a significant defect map.

[0043] In the embodiment of the present invention, the multi-scale feature map is input into the attention module. The attention module is composed of two parallel convolution layers, and the convolution kernel size is 3 3, stride 1, padding 'same'. The first convolutional layer outputs half the number of channels of the multi-scale feature map, and the second convolutional layer outputs 1. ReLU activation function is applied after each convolutional layer. The output of the second convolutional layer is activated by Sigmoid function to obtain the attention weight map, which ranges from 0 to 1, indicating the saliency of the background texture at each pixel position.

[0044] The attention weight map is element-wise multiplied with the multi-scale feature map. This operation assigns attention weights to the multi-scale feature map, so that the features of the background texture area are suppressed, while the features of the potential defect area are enhanced. Subsequently, the multiplication result is input into a 3×3 convolutional layer with a step size of 1, a padding of 'same', and the number of output channels is the same as the multi-scale feature map. After passing through the ReLU activation function, the background suppression feature map is obtained.

[0045] The multi-scale feature map and the background suppression feature map are added element by element to achieve residual connection. Residual connection helps to retain the original feature information and avoid the gradient vanishing problem. The result after addition is passed through a 1×1 convolution layer with a step size of 1 and a padding mode of 'same'. The number of output channels is the same as that of the multi-scale feature map. The function of this convolution layer is to fuse the features after residual connection to obtain a fused feature map.

[0046] The fused feature map is input into the ReLU activation function to introduce nonlinearity and enhance the expressiveness of the model. Subsequently, the activated feature map is normalized. The Min-Ma× normalization method is used to scale the pixel value range of the feature map to between 0 and 1 to obtain the final significant defect map. The significant defect map highlights the potential defect area and suppresses the interference of background texture, providing a basis for subsequent defect segmentation.

[0047] Preferably, step S3 comprises the following steps: Step S31: determining the size of the local area according to the pre-processed PCB image and the significant defect map to obtain the local area size data; Step S32: Calculate the local statistical features of the sliding window of the significant defect map according to the local area size data to obtain a local statistical feature map; Step S33: performing adaptive threshold calculation according to the local statistical feature map to obtain an adaptive threshold map, wherein the local statistical feature map includes a local histogram, a local mean map, and a local standard deviation map; Step S34: Binarize and segment the adaptive threshold map and the significant defect map to obtain candidate defect regions.

[0048] As an example of the present invention, refer to Figure 2 As shown, in this example, step S3 includes: Step S31: determining the size of the local area according to the pre-processed PCB image and the significant defect map to obtain the local area size data; In an embodiment of the present invention, the preprocessed PCB image and the significant defect map are analyzed to determine the size of the local region. Considering the diversity of PCB defect sizes, a statistical method based on the significant defect map is used to determine the size of the local region. First, the gradient amplitude map of the significant defect map is calculated. Then, the connected areas of the pixels whose gradient amplitude is greater than a preset threshold (e.g., 0.1) are counted. The average size of these connected areas is calculated and used as the initial value of the local region size. In order to adapt to the characteristics of different PCB images, the initial value is multiplied by a scaling factor (e.g., 1.5) to obtain the final local region size data. The local region size data is in pixels, e.g., 15 15 pixels.

[0049] Step S32: Calculate the local statistical features of the sliding window of the significant defect map according to the local area size data to obtain a local statistical feature map; In an embodiment of the present invention, a sliding window operation is performed on a significant defect map using the local region size. The sliding step size is set to 1 pixel. For each pixel in the window, the local histogram, local mean, and local standard deviation are calculated. The local histogram divides the pixel value into 256 gray levels and counts the number of pixels at each gray level. The local mean calculates the average value of all pixel values ​​in the window. The local standard deviation calculates the standard deviation of all pixel values ​​in the window. The local histogram, local mean, and local standard deviation of each window are stored as a local statistical feature map. The size of the local statistical feature map is the same as that of the significant defect map, and each pixel position corresponds to a feature vector, which contains local histogram, local mean, and local standard deviation information.

[0050] Step S33: performing adaptive threshold calculation according to the local statistical feature map to obtain an adaptive threshold map, wherein the local statistical feature map includes a local histogram, a local mean map, and a local standard deviation map; In an embodiment of the present invention, for each pixel position in the local statistical feature map, the adaptive threshold is calculated using its corresponding local histogram, local mean and local standard deviation. First, the local histogram is threshold segmented using the Otsu algorithm to obtain an initial threshold. Then, the initial threshold is adjusted according to the local mean and local standard deviation. The adjustment formula is: `T=T0+α*(μ-β*σ)`, where T0 is the initial threshold, μ is the local mean, σ is the local standard deviation, α and β are preset parameters, for example α=0.5, β=1.0. The adaptive threshold of each pixel position is stored as an adaptive threshold map, the size of which is the same as the significant defect map.

[0051] Step S34: performing binary segmentation on the adaptive threshold map and the significant defect map to obtain candidate defect regions; In the embodiment of the present invention, the significant defect map is compared with the adaptive threshold map. If the value of a pixel in the significant defect map is greater than or equal to the corresponding adaptive threshold, the pixel is marked as 1, otherwise it is marked as 0. The marking result is stored as a binary image, i.e., a candidate defect area. In the candidate defect area, pixels with a value of 1 represent potential defect areas, and pixels with a value of 0 represent background areas.

[0052] Preferably, step S33 includes the following steps: Step S331: performing local Otsu threshold calculation on the local histogram to obtain a local Otsu threshold list; Step S332: converting the local Otsu threshold list into a threshold map according to the local area size data, and performing threshold smoothing to obtain a smoothed local Otsu threshold map; Step S333: performing local contrast calculation according to the local mean map and the local standard deviation map to obtain a local contrast map; Step S334: performing contrast normalization on the local contrast map to obtain a normalized local contrast map; Step S335: Calculate the threshold adjustment weight using the normalized local contrast map to obtain a threshold adjustment weight map; Step S336: Using the threshold adjustment weight map, threshold adjustment is performed on the smoothed local Otsu threshold map to obtain an adaptive threshold map.

[0053] In the embodiment of the present invention, the local histogram corresponding to each pixel position in the local statistical feature map is traversed. For each local histogram, the Otsu algorithm is applied to calculate the optimal segmentation threshold. The Otsu algorithm determines the optimal threshold by maximizing the inter-class variance. The Otsu threshold calculated for each local histogram is stored in a list to obtain a local Otsu threshold list.

[0054] The local Otsu threshold list is converted into a threshold map of the same size as the significant defect map. The threshold at each pixel position corresponds to the Otsu threshold calculated from its local histogram. In order to remove noise and obtain a smoother threshold map, the threshold map is smoothed using Gaussian filtering. The Gaussian kernel size is set to the same size as the local region (e.g. 15 15), the standard deviation is set to 1 / 3 of the local region size (e.g. 5). The smoothed local Otsu threshold map is obtained.

[0055] The local contrast map is calculated using the local mean map and the local standard deviation map. The local contrast at each pixel location is defined as the local standard deviation divided by the local mean: `C=σ / μ`, where C is the local contrast, σ is the local standard deviation, and μ is the local mean. To avoid division by zero errors, C is set to 0 when μ is 0.

[0056] Normalize the local contrast map. Use the Min-Ma× normalization method to scale the local contrast value to between 0 and 1: `C'=(C-Cmin) / (Cma×-Cmin)`, where C' is the normalized local contrast, C is the original local contrast, and Cmin and Cma× are the minimum and maximum values ​​in the local contrast map, respectively.

[0057] The threshold adjustment weight map is calculated based on the normalized local contrast map. The threshold adjustment weight is defined as an exponential function of the normalized local contrast: `W=e×p(γ*C')`, where W is the threshold adjustment weight, C' is the normalized local contrast, and γ is a parameter that controls the magnitude of the weight change, for example, γ=2.0.

[0058] The smoothed local Otsu threshold map is multiplied element by element with the threshold adjustment weight map to obtain the final adaptive threshold map: `T=T0*W`, where T is the adaptive threshold, T0 is the smoothed local Otsu threshold, and W is the threshold adjustment weight. Each pixel value in the adaptive threshold map represents the segmentation threshold at that pixel position.

[0059] Preferably, step S4 comprises the following steps: Step S41: removing noise from the candidate defect region to obtain a denoised candidate region; connecting the broken defects in the denoised candidate region to obtain a connected defect region; Step S42: performing preliminary void detection on the defective area after connection to obtain a preliminary void area; Step S43: performing distance transformation calculation on the preliminary hole area to obtain a distance transformation map; Step S44: marking the hole boundary of the preliminary hole area to obtain a hole boundary map; performing geodesic transformation calculation on the distance transformation map using the hole boundary map as a mark to obtain a geodesic distance map; Step S45: constructing a geodesic reconstruction map using the geodesic distance map and the distance transformation map to obtain a geodesic reconstruction map; Step S46: performing region merging and filling on the geodesic reconstruction image and the connected defect region to obtain a filled defect region; Step S47: performing edge smoothing on the filled defective region to obtain a smoothed defective region; performing region screening on the smoothed defective region to obtain a refined defective region.

[0060] As an example of the present invention, refer to Figure 3 As shown, in this example, step S4 includes: Step S41: removing noise from the candidate defect region to obtain a denoised candidate region; connecting the broken defects in the denoised candidate region to obtain a connected defect region; In the embodiment of the present invention, median filtering is applied to the candidate defect area to remove noise. The median filter kernel size is set to 3 3. Subsequently, the morphological closing operation is used to connect the broken defective areas. The closing operation is a combination of dilation followed by erosion operations, which can effectively connect the broken areas and fill small holes. Structural element selection 5 5 rectangular structural elements.

[0061] Step S42: performing preliminary void detection on the defective area after connection to obtain a preliminary void area; In the embodiment of the present invention, a connected region analysis is performed on the defective region after connection. The area and Euler number of each connected region are calculated. The Euler number is defined as the number of objects in the connected region minus the number of holes. If the Euler number of a connected region is less than 1, it is considered that the connected region contains a hole and is marked as a hole region. All pixels marked as hole regions are merged to obtain a preliminary hole region.

[0062] Step S43: performing distance transformation calculation on the preliminary hole area to obtain a distance transformation map; In the embodiment of the present invention, the preliminary hole region is subjected to distance transformation. The distance from each pixel in the hole region to the nearest pixel in the non-hole region is calculated using the Euclidean distance measurement method. The calculated distance values ​​are stored as a distance transformation map, in which the value of each pixel represents its distance to the nearest pixel in the non-hole region.

[0063] Step S44: marking the hole boundary of the preliminary hole area to obtain a hole boundary map; performing geodesic transformation calculation on the distance transformation map using the hole boundary map as a mark to obtain a geodesic distance map; In an embodiment of the present invention, a morphological gradient operation is used to extract the boundary of the preliminary hole area. The morphological gradient is defined as the dilation operation result minus the erosion operation result. The structural element selects a 3×3 rectangular structural element. The extracted boundary is marked as a hole boundary map. Using the hole boundary map as a marker, a geodesic transformation is performed on the distance transformation map. The geodesic transformation calculates the shortest geodesic distance from each pixel on the hole boundary to other boundary pixels, that is, the shortest path along the hole surface.

[0064] Step S45: constructing a geodesic reconstruction map using the geodesic distance map and the distance transformation map to obtain a geodesic reconstruction map; In the embodiment of the present invention, a geodesic reconstruction map is constructed using a distance transform map and a geodesic distance map. The geodesic distance map is used as a marker to perform geodesic reconstruction on the distance transform map. The geodesic reconstruction operation iteratively propagates the values ​​in the marker image to the distance transform map until a stable state is reached. The reconstructed image retains the shape of the original hole and fills the area inside the hole.

[0065] Step S46: performing region merging and filling on the geodesic reconstruction image and the connected defect region to obtain a filled defect region; In the embodiment of the present invention, a logical OR operation is performed on the geodesic reconstruction image and the connected defective region to merge the two regions. The merged region includes the connected defective region and the filled hole region.

[0066] Step S47: smoothing the edges of the filled defective region to obtain a smoothed defective region; performing region screening on the smoothed defective region to obtain a refined defective region; In the embodiment of the present invention, Gaussian filtering is applied to the defective area after filling to perform edge smoothing. The Gaussian kernel size is set to 5 5, and the standard deviation is set to 1. Then, the area is screened according to the area of ​​the connected area. The connected areas with an area smaller than the preset threshold (e.g. 10 pixels) are removed and considered to be noise. The connected areas finally obtained are the refined defect areas.

[0067] Preferably, step S43 includes the following steps: Step S431: performing defect region inversion on the preliminary void region to obtain an inverted defect region; Step S432: using a pre-selected distance measurement method to calculate an initial distance map of the defective area after inversion to obtain an initial distance map; Step S433: normalizing the initial distance map to obtain a normalized distance map; Step S434: performing distance map enhancement on the normalized distance map to obtain a distance transformation map.

[0068] In the embodiment of the present invention, the preliminary hole region is binary inverted. In the preliminary hole region, pixels with a value of 1 represent the hole region, and pixels with a value of 0 represent the background region. After inversion, pixels with a value of 0 represent the hole region, and pixels with a value of 1 represent the background region. This operation is to facilitate the subsequent distance transformation calculation.

[0069] Perform distance transformation calculation on the inverted defective area. Use the exact Euclidean distance as the distance metric. Calculate the Euclidean distance from each pixel in the inverted defective area to the nearest non-zero pixel (i.e., the pixel in the original hole area). Store the calculated distance values ​​as the initial distance map. In the initial distance map, the value of each pixel represents its Euclidean distance to the nearest pixel in the original hole area.

[0070] Normalize the initial distance map. Use the Min-Ma× normalization method to scale the distance value to between 0 and 1. The normalization formula is: `D'=(D-Dmin) / (Dma×-Dmin)`, where D' is the normalized distance value, D is the original distance value, and Dmin and Dma× are the minimum and maximum values ​​in the initial distance map, respectively.

[0071] The normalized distance map is enhanced to highlight the change in distance. The power law transformation is used for enhancement, and the formula is: `D''=D'^γ`, where D'' is the enhanced distance value, D' is the normalized distance value, and γ is a parameter that controls the degree of enhancement, for example, γ=2.0. The enhanced distance value is stored as the final distance transformation map. In the distance transformation map, the larger the pixel value, the farther the pixel is from the original hole area. This enhancement step helps the subsequent geodesic transformation and reconstruction, and improves the refinement of the defect area.

[0072] Preferably, step S45 includes the following steps: Step S451: constructing an initial reconstruction image according to the distance transformation image to obtain an initial reconstruction image; Step S452: determining a dynamic threshold coefficient according to the geodesic distance map to obtain a dynamic threshold coefficient; Step S453: traverse each pixel in the distance transformation map, and multiply the distance value of the corresponding pixel in the geodesic distance map by the dynamic threshold coefficient to obtain a dynamic threshold; Step S454: comparing pixel values ​​according to the distance transformation map and the dynamic threshold, and labeling the initial reconstructed map to obtain a labeled map; Step S455: Perform morphological processing on the labeled image to obtain a geodesic reconstruction image.

[0073] In the embodiment of the present invention, the distance transformation map is copied and used as the initial reconstruction map. The size and content of the initial reconstruction map are exactly the same as those of the distance transformation map. Subsequent operations will modify the initial reconstruction map to finally obtain a geodesic reconstruction map.

[0074] Analyze the geodesic distance map and determine the dynamic threshold coefficient. Calculate the mean μ and standard deviation σ of the geodesic distance map. The dynamic threshold coefficient is defined as: `k=1-α*(σ / μ)`, where α is a parameter that controls the size of the coefficient, for example, α=0.5. This coefficient is used to control the sensitivity of geodesic reconstruction.

[0075] Traverse each pixel in the distance transformation map. For each pixel, multiply the distance value of the corresponding position in the geodesic distance map by the dynamic threshold coefficient k to obtain the dynamic threshold of the pixel.

[0076] Traverse each pixel of the initial reconstruction image. Compare the distance value of the corresponding pixel in the distance transform image with the dynamic threshold. If the value in the distance transform image is greater than the dynamic threshold, set the value of the corresponding pixel in the initial reconstruction image to the value of the corresponding pixel in the geodesic distance image, otherwise keep the value of the corresponding pixel in the initial reconstruction image unchanged. Get a labeled image, in which the values ​​of some pixels come from the geodesic distance map and the values ​​of other pixels come from the distance transform image.

[0077] Perform morphological closing operation on the marked image. The structural element is a 3×3 rectangular structural element. The closing operation can smooth the reconstruction result, remove small noise and artifacts, and connect the broken areas. The result after the closing operation is the final geodesic reconstruction image. The geodesic reconstruction image accurately describes the shape and size of the void and fills the area inside the void, providing a basis for the subsequent refinement of the defect area.

[0078] Preferably, step S5 comprises the following steps: Step S51: extracting features from the refined defect area to obtain a defect feature vector; Step S52: performing lightweight classifier training according to the defect feature vector and the preset defect label to obtain a defect classifier; Step S53: using a defect classifier to classify the defect feature vector to obtain defect category data; Step S54: locating the defect according to the refined defect area and defect category data to obtain defect position and size data; Step S55: Integrate the defect information of the defect category data and the defect position and size data to obtain the defect detection result.

[0079] In the embodiment of the present invention, for each refined defect area, a series of features are extracted to form a defect feature vector. The extracted features include: 1. Area: the number of pixels in the defect area; 2. Perimeter: the number of pixels at the boundary of the defect area; 3. Circularity: 4π*area / perimeter squared; 4. Hu moment: 7 invariant moments that describe the shape characteristics of the defect area; 5. Grayscale mean: the average value of the grayscale value of pixels in the defect area (based on the preprocessed PCB image); 6. Grayscale standard deviation: the standard deviation of the grayscale value of pixels in the defect area (based on the preprocessed PCB image). These features are combined into a vector as the feature vector of the defect area.

[0080] Using a pre-prepared dataset containing various PCB defect types, the defect feature vector is extracted and preset defect labels are annotated for each defect area, such as open circuit, short circuit, missing component, rat bite, etc. Use these annotated data to train a lightweight classifier, such as a support vector machine (SVM) or a random forest. SVM uses a radial basis kernel function (RBF) with a penalty coefficient C set to 1.0. Random forest uses 100 decision trees with a maximum depth of 10. During the training process, 5-fold cross validation is used for model selection and parameter optimization.

[0081] The defect feature vector is classified using a defect classifier. The classifier outputs the defect category label to which each defect area belongs, such as open circuit, short circuit, missing component, rat bite, etc. The classification results are stored as defect category data.

[0082] For each refined defect area, calculate the upper left corner coordinates, width, and height of its minimum bounding rectangle. The minimum bounding rectangle can accurately define the position and size of the defect area. The center coordinates, width, and height of the defect area, as well as the corresponding defect category label, are stored as defect position and size data.

[0083] The defect category data and defect location and size data are integrated together to form the final defect detection results. The defect detection results are output in a table or JSON format, including the category, location coordinates, width, height and other information of each defect, which is convenient for subsequent defect analysis and processing.

[0084] Preferably, the present invention further provides a printed circuit board defect detection system based on image recognition, which is used to execute the printed circuit board defect detection method based on image recognition as described above, and the printed circuit board defect detection system based on image recognition comprises: The multi-scale feature fusion module is used to perform image preprocessing on the original PCB image to obtain a preprocessed PCB image; perform scale feature fusion on the preprocessed PCB image to obtain a multi-scale feature map; A background texture suppression module is used to perform background texture feature learning on a multi-scale feature map to obtain a background suppression feature map; background texture suppression is performed based on the background suppression feature map to obtain a significant defect map; The adaptive threshold segmentation module is used to calculate local statistical features based on the pre-processed PCB image and the significant defect map to obtain a local statistical feature map; perform adaptive threshold calculation based on the local statistical feature map, and perform binary segmentation to obtain a candidate defect area; The morphological optimization module is used to perform preliminary hole detection based on the candidate defect area to obtain a preliminary hole area; perform distance transformation calculation on the preliminary hole area to obtain a distance transformation map; perform geodesic transformation calculation on the distance transformation map to obtain a geodesic distance map; use the geodesic distance map and the distance transformation map to construct a geodesic reconstruction map to obtain a geodesic reconstruction map; refine the defect area on the geodesic reconstruction map to obtain a refined defect area; The defect classification and positioning module is used to classify the refined defect area and obtain defect category data; locate the defect according to the defect category data to obtain defect position and size data; integrate the defect information of the defect category data and the defect position and size data to obtain the defect detection result.

[0085] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0086] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A printed circuit board defect detection method based on image recognition, characterized in that: The following steps are involved: Step S1: performing image preprocessing on the original PCB image to obtain a preprocessed PCB image; Perform scale feature fusion on the pre-processed PCB image to obtain a multi-scale feature map; Step S2: performing background texture feature learning on the multi-scale feature map to obtain a background suppression feature map; Perform background texture suppression based on the background suppression feature map to obtain a significant defect map; Step S3: Calculate local statistical features based on the preprocessed PCB image and the significant defect map to obtain a local statistical feature map; calculate an adaptive threshold based on the local statistical feature map, and perform binary segmentation to obtain a candidate defect area; Step S4: performing preliminary hole detection on the candidate defect area to obtain a preliminary hole area; performing distance transformation calculation on the preliminary hole area to obtain a distance transformation map; performing geodesic transformation calculation on the distance transformation map to obtain a geodesic distance map; constructing a geodesic reconstruction map using the geodesic distance map and the distance transformation map to obtain a geodesic reconstruction map; refining the defect area on the geodesic reconstruction map to obtain a refined defect area; Step S5: classify the refined defect area to obtain defect category data; Defect location is performed according to defect category data to obtain defect position and size data; The defect information is integrated on the defect category data as well as the defect location and size data to obtain the defect detection results, so as to realize the printed circuit board defect detection operation.

2. The printed circuit board defect detection method based on image recognition according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: performing image preprocessing on the original PCB image to obtain a preprocessed PCB image; Step S12: performing multi-scale wavelet transform on the pre-processed PCB image to obtain multi-scale wavelet coefficients; Step S13: Use a pre-trained deep convolutional neural network to extract deep features from the pre-processed PCB image to obtain a deep feature map; Step S14: reconstructing the multi-scale wavelet coefficients to obtain a reconstructed wavelet feature map; Step S15: upsampling the depth feature map according to the reconstructed wavelet feature map to obtain an upsampled depth feature map; Step S16: performing feature fusion on the reconstructed wavelet feature map and the upsampled depth feature map to obtain a multi-scale feature map.

3. The printed circuit board defect detection method based on image recognition according to claim 1, characterized in that: Step S2 includes the following steps: Step S21: constructing an attention module for background texture on the multi-scale feature map to obtain an attention weight map; Step S22: performing attention-guided feature learning on the attention weight map and the multi-scale feature map to obtain a background suppression feature map; Step S23: performing residual connection and feature fusion on the multi-scale feature map and the background suppression feature map to obtain a fused feature map; Step S24: Perform nonlinear activation and normalization on the fused feature map to obtain a significant defect map.

4. The printed circuit board defect detection method based on image recognition according to claim 1, characterized in that: Step S3 includes the following steps: Step S31: determining the size of the local area according to the pre-processed PCB image and the significant defect map to obtain the local area size data; Step S32: Calculate the local statistical features of the sliding window of the significant defect map according to the local area size data to obtain a local statistical feature map; Step S33: performing adaptive threshold calculation according to the local statistical feature map to obtain an adaptive threshold map, wherein the local statistical feature map includes a local histogram, a local mean map, and a local standard deviation map; Step S34: Binarize and segment the adaptive threshold map and the significant defect map to obtain candidate defect regions.

5. The printed circuit board defect detection method based on image recognition according to claim 4 is characterized in that: Step S33 includes the following steps: Step S331: performing local Otsu threshold calculation on the local histogram to obtain a local Otsu threshold list; Step S332: converting the local Otsu threshold list into a threshold map according to the local area size data, and performing threshold smoothing to obtain a smoothed local Otsu threshold map; Step S333: performing local contrast calculation according to the local mean map and the local standard deviation map to obtain a local contrast map; Step S334: performing contrast normalization on the local contrast map to obtain a normalized local contrast map; Step S335: Calculate the threshold adjustment weight using the normalized local contrast map to obtain a threshold adjustment weight map; Step S336: Using the threshold adjustment weight map, threshold adjustment is performed on the smoothed local Otsu threshold map to obtain an adaptive threshold map.

6. The printed circuit board defect detection method based on image recognition according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: removing noise from the candidate defect region to obtain a denoised candidate region; connecting the broken defects in the denoised candidate region to obtain a connected defect region; Step S42: performing preliminary void detection on the defective area after connection to obtain a preliminary void area; Step S43: performing distance transformation calculation on the preliminary hole area to obtain a distance transformation map; Step S44: marking the hole boundary of the preliminary hole area to obtain a hole boundary map; performing geodesic transformation calculation on the distance transformation map using the hole boundary map as a mark to obtain a geodesic distance map; Step S45: constructing a geodesic reconstruction map using the geodesic distance map and the distance transformation map to obtain a geodesic reconstruction map; Step S46: performing region merging and filling on the geodesic reconstruction image and the connected defect region to obtain a filled defect region; Step S47: performing edge smoothing on the filled defective region to obtain a smoothed defective region; performing region screening on the smoothed defective region to obtain a refined defective region.

7. The printed circuit board defect detection method based on image recognition according to claim 6, characterized in that: Step S43 includes the following steps: Step S431: performing defect region inversion on the preliminary void region to obtain an inverted defect region; Step S432: using a pre-selected distance measurement method to calculate an initial distance map of the defective area after inversion to obtain an initial distance map; Step S433: normalizing the initial distance map to obtain a normalized distance map; Step S434: performing distance map enhancement on the normalized distance map to obtain a distance transformation map.

8. The printed circuit board defect detection method based on image recognition according to claim 6, characterized in that: Step S45 includes the following steps: Step S451: constructing an initial reconstruction image according to the distance transformation image to obtain an initial reconstruction image; Step S452: determining a dynamic threshold coefficient according to the geodesic distance map to obtain a dynamic threshold coefficient; Step S453: traverse each pixel in the distance transformation map, and multiply the distance value of the corresponding pixel in the geodesic distance map by the dynamic threshold coefficient to obtain a dynamic threshold; Step S454: comparing pixel values ​​according to the distance transformation map and the dynamic threshold, and labeling the initial reconstructed map to obtain a labeled map; Step S455: Perform morphological processing on the labeled image to obtain a geodesic reconstruction image.

9. The printed circuit board defect detection method based on image recognition according to claim 1, characterized in that: Step S5 includes the following steps: Step S51: extracting features from the refined defect area to obtain a defect feature vector; Step S52: performing lightweight classifier training according to the defect feature vector and the preset defect label to obtain a defect classifier; Step S53: using a defect classifier to classify the defect feature vector to obtain defect category data; Step S54: locating the defect according to the refined defect area and defect category data to obtain defect position and size data; Step S55: Integrate the defect information of the defect category data and the defect position and size data to obtain the defect detection result.

10. A printed circuit board defect detection system based on image recognition, characterized in that: Used to execute the printed circuit board defect detection method based on image recognition as claimed in claim 1, the printed circuit board defect detection system based on image recognition comprises: The multi-scale feature fusion module is used to perform image preprocessing on the original PCB image to obtain a preprocessed PCB image; perform scale feature fusion on the preprocessed PCB image to obtain a multi-scale feature map; A background texture suppression module is used to perform background texture feature learning on a multi-scale feature map to obtain a background suppression feature map; background texture suppression is performed based on the background suppression feature map to obtain a significant defect map; The adaptive threshold segmentation module is used to calculate local statistical features based on the pre-processed PCB image and the significant defect map to obtain a local statistical feature map; perform adaptive threshold calculation based on the local statistical feature map, and perform binary segmentation to obtain a candidate defect area; The morphological optimization module is used to perform preliminary hole detection based on the candidate defect area to obtain a preliminary hole area; perform distance transformation calculation on the preliminary hole area to obtain a distance transformation map; perform geodesic transformation calculation on the distance transformation map to obtain a geodesic distance map; use the geodesic distance map and the distance transformation map to construct a geodesic reconstruction map to obtain a geodesic reconstruction map; refine the defect area on the geodesic reconstruction map to obtain a refined defect area; The defect classification and positioning module is used to classify the refined defect area and obtain defect category data; locate the defect according to the defect category data to obtain defect position and size data; integrate the defect information of the defect category data and the defect position and size data to obtain the defect detection result.

Images