Method and system for detecting defects of target printed circuit board based on AI

Abstract

The invention discloses a method and a system for detecting defects of a target printed circuit board based on AI. The method comprises the following steps: inputting an image of the target printed circuit board into a pre-trained primary neural network; extracting a plurality of feature maps from the image input to a predetermined number of layers according to semantic and spatial information, and generating feature map layers from higher to lower arranged in a path from top to bottom; processing at least one predetermined feature layer by a pyramid structure model coupled with selective feature interest; transmitting the processed feature layer to an ROI (Region of Interest) aligner for normalization processing; and outputting and visualizing the processed layer through an output layer, and displaying a defect result on the target printed circuit board by the output layer. The method has the advantages that the method is easy to implement, especially when the small defects on the printed circuit board are detected with various resolutions, the small defects in the printed circuit board are detected with a high cross joint threshold value, and the method is simple, easy to operate, accurate and effective.

Description

Technical Field

[0001] The present invention belongs to the field of power electronics technology and relates to a method and system for detecting defects of a target printed circuit board based on AI. Background Art

[0002] The miniaturization of electronic products and the widespread use of small printed circuit boards (PCBs) present significant challenges for current AI-based defect detection methods. These methods often struggle to effectively detect minute defects in these small electronic components. This limitation has far-reaching consequences, potentially compromising safety, reducing overall manufacturing quality, and ultimately increasing production costs. The trend toward miniaturization of electronic products means that traditional defect detection systems can miss defects in tiny components or traces on PCBs. These defects may be invisible to the naked eye but can have serious consequences, especially in safety-critical applications such as medical devices or automotive electronics. In these situations, even minor faults can lead to catastrophic failures, necessitating improved defect detection accuracy.

[0003] A reduced ability to detect these minute defects can also compromise the overall quality of electronic products. Even minor defects can affect the functionality and lifespan of a device, leading to increased customer dissatisfaction and potentially costly recalls or warranty claims. Maintaining high manufacturing quality is not only a matter of reputation but also a key factor in maintaining a competitive advantage in the market. Failure to detect small defects early in the production process can lead to increased production costs, as defects discovered further down the production chain often require more cost and time to correct. Early defect detection can streamline the production process, reduce scrap rates, and ultimately improve cost efficiency.

[0004] Traditional Feature Pyramid Networks (FPNs) may not provide sufficiently rich semantic information for certain tasks, such as printed circuit board defect detection. Many attention mechanisms primarily focus on selecting different channels of feature maps and individually suppressing different spatial regions in the feature maps, which may not be sufficient to solve the complex problem of detecting tiny defects on printed circuit boards.

[0005] Squeeze-Excitation Network (SE-Net) captures some semantic information by adaptively weighting different channels, focusing mainly on the relationship between channels rather than spatial information, but they do not directly focus on spatial relationships and dependencies with feature maps.

[0006] In addition to the aforementioned issues, patent publication TWI718573B may also disclose an automated optical inspection (AOI) system that uses a deep learning algorithm to detect defects. This system consists of inspection equipment and a processor with an inspection system. The inspection system includes an inspection module, an artificial intelligence extraction module, and a judgment module. A deep neural network is used to extract features from normalized images of candidate frames, generating feature vectors representing the inspected features of the printed circuit board. However, the results produced by this invention cannot be displayed visually, as this would require highly skilled operators to operate the system and analyze the results. Due to the complexity of the technology, the overall cost would be high.

[0007] Chinese patent publication CN115100098A discloses an AI-enabled system for processing captured images. The system and method detect defects on printed circuit boards by calculating the difference between a standard image and each captured image and generating a corresponding difference image. However, this technology lacks the ability to process spatial and semantic information, resulting in images of defective printed circuit boards that lack high resolution.

[0008] In addition to the aforementioned technologies, a patent document published by the World Intellectual Property Organization under the publication number WO2022098488A1 may also disclose a general technology that may be applicable to various industries, including electronics. While it incorporates artificial intelligence and has the ability to process spatial and semantic information, it may lack accuracy in displaying defective images because it uses a camera to capture the initial image before processing, rather than using a sensor (which has high sensitivity compared to a camera).

[0009] US Patent Publication No. US9524450B may employ a convolutional neural network (CNN) to classify objects in digital images. However, due to the small size of printed circuit boards, this technology may not be effective in capturing defects in them. Furthermore, the technology utilizes images captured only by a mobile phone camera, making it unlikely that a mobile phone camera can accurately capture defects at high resolution.

[0010] Because electronic products are made up of interconnected electronic components on printed circuit boards (PCBs), the quality of these boards is crucial to determining the overall performance and reliability of the product. Therefore, detecting any defects on PCBs during the manufacturing process with low cost and high accuracy is crucial. This paper aims to provide a system with a learnable framework for detecting small defects in PCBs with a high intersection-over-union (IoU) threshold. Summary of the Invention

[0011] The purpose of the present invention is to overcome the above-mentioned shortcomings and provide a method and system for detecting defects of a target printed circuit board based on AI, which can solve various problems existing in the prior art.

[0012] In order to achieve the above-mentioned object, the technical solution adopted by the present invention is: a method for detecting defects of a target printed circuit board based on AI, characterized by comprising: Input the image of the target printed circuit board into the pre-trained primary neural network; Extract multiple feature maps from the image input into a predetermined number of layers based on semantic and spatial information, generating feature layers arranged from higher to lower levels in a top-down path; Processing at least one predetermined feature layer through a pyramid structure model coupled with selective feature attention; Pass the processed feature layer to the ROI aligner for normalization; and The layer after outputting and visualization processing is outputted through the output layer, and the defect results on the target printed circuit board are displayed by the output layer.

[0013] Furthermore, the generating of the feature layers from higher to lower arranged in a top-down path includes: generating the feature layers from higher to lower with progressive semantic and spatial information.

[0014] Furthermore, the processing of at least one predetermined feature layer by the pyramid structure model coupled with selective feature attention includes: selecting at least one feature layer from a higher feature layer, and passing the feature map in the feature layer to the deep convolution kernel.

[0015] Furthermore, the processing of at least one predetermined feature layer by the pyramid structure model coupled with selective feature attention includes: Select at least one feature layer from the upper feature layer and the lower feature layer respectively; Upsampling feature maps in a feature layer selected from a higher feature layer; Fusing the upsampled feature map with a feature map in a feature layer selected from the lower feature layer to form a fused feature map; Passing the fused feature map to global pooling to form a vector; Compress the vector from c dimension to z dimension; Separating the compressed vector into a first separated vector and a second separated vector of c dimension; Multiplying the first separation vector and the upsampled feature map to generate a weighted first feature map; Multiplying the second separating vector by a selected feature map in a lower feature layer to generate a weighted second feature map; The weighted first feature map and the second weighted feature map are fused to form a fused feature map.

[0016] Furthermore, the selecting at least one feature layer from the higher feature layer and the lower feature layer further comprises the following steps: The feature maps in the feature layers selected from the higher feature layers have higher semantic information and lower spatial information; The feature maps in the feature layer selected from the lower feature layer have lower semantic information and higher spatial information.

[0017] Furthermore, the method further includes the step of excluding at least one feature map in the lowest feature layer that has a large memory footprint and low semantic information.

[0018] Furthermore, upsampling the feature map in the feature layer selected from the higher feature layer includes: Pass the selected feature map to the learnable block for processing; Outputting the processed feature maps to a channel size similar to four times the size of the selected feature maps from the lower feature layers; and The processed feature map is magnified by 2 times through the pyramid structure model to form an upsampled feature map.

[0019] Furthermore, the processing of at least one predetermined feature layer by the pyramid structure model coupled with selective feature attention further comprises the following steps: At least one fused feature map is generated from the corresponding feature map in the preset feature layer, and the generated fused feature map has no direct interaction with its corresponding feature map.

[0020] Furthermore, the processing of at least one predetermined feature layer by the pyramid structure model coupled with selective feature attention further comprises the following steps: Selected feature maps are learned and utilized from higher feature layers with higher semantic and spatial information to detect minute defects in target printed circuit boards.

[0021] Furthermore, the pre-trained primary neural network is trained using training images having multiple semantic and spatial information from multiple defective printed circuit boards, and the pre-training of the primary neural network further includes the following steps: Downsample the training images to multiple scales; Fusing downsampled training images using a pyramid structure model coupled with selective feature attention to fuse feature maps from higher and lower feature layers; Passing the fused feature layer of the training image to the region of interest alignment; generating a result that includes the defect; and The results are stored in the primary neural network for further learning.

[0022] Another object of the present invention is to provide a system for detecting defects in a target printed circuit board based on AI, characterized by comprising: A processor and a non-transitory medium including machine-readable instructions executable by the processor, including an extraction module for extracting a plurality of feature maps from an image input into a predetermined number of layers based on semantic and spatial information, generating feature map layers arranged from higher to lower layers in a top-down path; a processing module, configured to process at least one predetermined feature layer through a pyramid structure model coupled with selective feature attention; A normalization module, which is used to pass the processed feature map to the ROI aligner for normalization; and The output module is used to output and visualize the processed layer through the output layer, and the output layer displays the defect results on the target printed circuit board.

[0023] Furthermore, the AI ​​further includes a pre-trained primary neural network, which is a deep convolutional neural network including ResNet50.

[0024] Furthermore, the image of the target printed circuit board is collected using a camera or an image file.

[0025] Furthermore, the feature layers from higher to lower include: feature layers from higher to lower with progressive semantic and spatial information, when transitioning from the top layer to the lower layer, the sampling rate of the top layer with the most semantic information and the lower layer with the least semantic information is reduced by 2 times, and the number of feature map channels is increased by 2 times.

[0026] Furthermore, the pyramid structure model coupled with selective feature attention includes: at least one learnable block having 3×3 or 5×5 dimensions, the learnable block configured to perform feature map extraction and upsampling on a feature map having imperfect spatial information; and At least one depth-wise separable convolution kernel with 3×3 dimensions, configured to send feature maps with poor semantic information to further processing for selective feature attention.

[0027] Furthermore, the learnable block is a CP block.

[0028] Furthermore, the selective feature attention is configured to fuse multiple weighted feature maps to form a fused feature map with balanced semantic and spatial information.

[0029] Furthermore, the output module displays the defect classification and location on the target printed circuit board in the form of open circuit, short circuit and rat bite.

[0030] Compared with the prior art, the present invention has the following advantages: The implementation is simple, including: inputting an image of a target printed circuit board into a pre-trained primary neural network; extracting multiple feature maps from the image input based on semantic and spatial information into a predetermined number of layers, generating feature layers arranged from higher to lower levels in a top-down path; processing at least one predetermined feature map through a pyramid structure model coupled with selective feature attention; passing the processed feature map to a ROI aligner for normalization; and outputting and visualizing the processed layers through an output layer, with the output layer displaying defect results on the target printed circuit board. The system has a learnable framework that focuses on the spatial relationships and dependencies of feature maps, particularly when detecting small defects on printed circuit boards at various resolutions. It detects small defects in printed circuit boards with a high cross-union threshold, is simple and easy to operate, and processes semantic and spatial information for certain tasks, particularly for detecting small defects in printed circuit boards at various resolutions, with accuracy and effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The features of the present invention will be more readily understood and appreciated when the following detailed description is read in conjunction with the accompanying drawings of preferred embodiments of the present invention, in which: Figure 1 A flow chart of a method for detecting defects in a target printed circuit board based on AI according to the present invention is shown; Figure 2 Shows a schematic structural diagram of the CP block of the present invention; Figure 3 A schematic diagram showing a process of forming a final feature map through selective feature attention (SF) according to the present invention; Figure 4 The present invention shows a schematic diagram of a system for detecting defects in a target printed circuit board based on a pre-trained primary neural network; Figure 5a The first output terminal on the printed circuit board is shown, more specifically, as a missing hole; Figure 5b The second output on the printed circuit board is shown, more specifically, the rat bite; Figure 5c The third output terminal on the printed circuit board is shown, more specifically, as an open circuit; Figure 5d A fourth output terminal on the printed circuit board is shown, more specifically, a short-circuited output; Figure 5eThe fifth output on the printed circuit board is shown, more specifically, a spurious; Figure 5f The sixth output on the printed circuit board is shown, more specifically, a pseudo copper; Figure 6 shows the comparison of the results of different models, namely (1) ground truth; (2) the present invention and (3) Faster R-CNN: where A: open circuit output; B: short circuit output; C: mouse bite output. DETAILED DESCRIPTION

[0032] As requested, specific embodiments of the present invention are disclosed herein. However, it should be understood that the disclosed embodiments are merely examples of the present invention, which can be implemented in many different forms. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limiting, but only as the basis for the claims. It should be understood that the drawings and their detailed description are not intended to limit the present invention to the specific forms disclosed herein. On the contrary, the present invention covers all modifications, equivalents and alternatives that fall within the scope defined by the claims. As used throughout this application, the word "may" means optional (i.e., meaning possible), rather than mandatory (i.e., meaning must). Similarly, the words "include" and "comprising" are meant to include but are not limited to. In addition, unless otherwise noted, the word "one" means "at least one" and the word "a plurality" means one or more. When abbreviations or technical terms are used, these refer to the generally accepted meanings known in the art.

[0033] like Figure 1 As shown, the present invention discloses a method for detecting defects of a target printed circuit board based on AI, comprising: An image of a target printed circuit board is input into a pre-trained primary neural network 102; multiple feature maps are extracted from the image input according to semantic and spatial information into a predetermined number of layers and arranged in a top-down path 104; at least one predetermined feature layer is processed through a pyramid structure model coupled with selective feature attention (SF-PSPyramid) 106; the processed feature layer is passed to a region of interest (ROI) aligner for normalization 108; and the processed layer is output and visualized through an output layer, wherein the output layer displays defect results on the target printed circuit board 110. Figure 1 The process overview of the method is further explained.

[0034] In one embodiment of the present invention, the step of extracting the feature map into a predetermined number of feature layers, arranged in a top-down path 104, further includes: generating higher feature layers with progressively higher semantic and spatial information; and generating lower feature layers with progressively lower semantic and spatial information.

[0035] In another embodiment of the present invention, step 106 of processing at least one preset feature layer by SF-PSPyramid further includes: selecting at least one feature map from a higher feature layer; passing the feature map to the deep convolution kernel (MConv) for further processing by the ROI aligner.

[0036] The most critical aspects of the present invention involve upsampling (e.g. Figure 2 ) and fuse upsampled feature maps to detect defects (as Figure 3 as described), such as Figure 2 As shown in the figure, where “c_in” and “c_out” represent the number of input channels and output channels respectively, and “h” and “w” represent the height and width of the feature map during upsampling respectively; More specifically, the defects that SF-PSPyramid can detect include tiny defects in printed circuit boards. The following embodiment further elaborates on this, wherein step 106 of processing at least one feature layer by SF-PSPyramid further includes: selecting at least one feature layer from a higher feature layer and a lower feature layer respectively; upsampling the selected feature layer from the higher feature layer by PSPyramid; fusing the upsampled feature map with the feature map selected from the lower feature layer by selective feature attention to form a fused feature map; passing the fused feature map to global pooling to form a vector; compressing the vector from c dimension to z dimension; separating the compressed vector into a first separated and a second separated vector of c dimension; multiplying the first separated vector with the upsampled feature map to generate a weighted first feature map; multiplying the second separated vector with the selected feature map in the lower layer feature map to generate a weighted second feature map; and fusing the first and second weighted feature maps to form a fused feature map for further processing by the ROI aligner.

[0037] Based on the above embodiment, selecting at least one feature layer from a higher layer and a lower layer also includes: selecting a feature layer with higher semantic information and lower spatial information from a higher layer; selecting a feature layer with lower semantic information and higher spatial information from a lower layer.

[0038] In another embodiment of the present invention, the step of selecting at least one feature layer from the high layer and the low layer respectively further includes: excluding at least one feature layer from the lowest layer, and the feature layer in the lowest layer has a larger memory footprint and lower semantic information.

[0039] See also Figure 2, which involves the steps of upsampling a selected feature map from a higher feature layer via PSPyramid, further comprising: passing the selected feature map to a learnable block for processing; outputting the processed feature map with a channel size four times that of the selected feature map from the lower feature layer; and upscaling the processed feature map by a factor of 2 via PSPyramid to form an upsampled feature map. The size of the upsampled feature map is now doubled.

[0040] Figure 3 The pipeline for fusing upsampled images via selective feature attention is described, where selective feature attention allows the feature maps to be split into two streams using either a 3x3 convolution kernel or a 5x5 convolution kernel. The merged output is then used to determine the weights for the weighted summation to optimally merge the feature maps. Figure 3 In [1], the input is two feature maps from two adjacent layers. The feature map of the higher layer has a lower resolution and is upsampled to the same resolution as the lower layer. Selective feature attention adaptively weights the two feature maps based on their semantic and spatial information, generating an optimal fused feature map that incorporates both information. Since the two feature maps used by selective feature attention are semantically distinct, it can align the feature maps and inject useful information from both. When combined with PSPyramid, this architecture enables cross-scale attention within the pyramid.

[0041] An embodiment of the present invention further relates to step 106 of processing at least one feature layer through SF-PSPyramid, which further includes: generating at least one fused feature map from its corresponding feature map in the preset feature layer, and the generated fused feature map has no direct interaction with its corresponding feature map.

[0042] In another embodiment of the present invention, the step of processing at least one feature layer by the SF-PSPyramid 106 further includes learning and utilizing selected feature maps from higher feature layers having higher semantic and spatial information to detect minute defects in the target printed circuit board.

[0043] More specifically, refer to Figure 4 , the present invention does not directly receive information from the lateral convolutional layers. The Pi feature maps (i = 2, 3, 4) primarily receive information from the corresponding Cj feature maps (j = 3, 4, 5). No Pi feature map is directly linked to the Ci feature map. This forces the present invention to learn and utilize semantics from deeper (higher-layer) feature maps. This design is motivated by the fact that PCB defects are not natural images and most objects are small. Therefore, focusing on shallow (lower-layer) feature maps while leveraging the semantics of deeper feature maps facilitates the detection of subtle PCB defects.

[0044] In another embodiment of the present invention, wherein a pre-trained primary neural network is trained using training images having multiple semantic and spatial information from multiple defective printed circuit boards, the pre-training of the primary neural network further includes downsampling the training images to multiple scales; fusing the downsampled training images using SF-PSPyramid to fuse feature maps from a higher feature layer and a lower feature layer; passing the fused feature layer of the training image to region of interest (ROI) alignment; generating a result containing defects; and storing the result in the primary neural network for further learning.

[0045] The present invention also discloses a system 400 for detecting defects on a target printed circuit board (PCB) based on artificial intelligence (AI) 403, the system comprising: a processor and a non-transitory medium comprising machine-readable instructions executable by the processor, including an extraction module that extracts multiple feature maps from an image input 402 into a predetermined number of layers 404 using a top-down approach based on semantic and spatial information; a processing module that processes at least one feature layer using a pyramid structure model SF-PSPyramid 406a coupled with selective feature attention 406b; a normalization module that includes feature map region of interest (ROI) alignment 408; and an output device that displays output defects on the PCB on a display screen 410. Figure 4 An overview of the above system is shown and is described in more detail below.

[0046] In another embodiment of the present invention, the artificial intelligence further includes a pre-trained primary neural network 403, which is a deep convolutional neural network such as ResNet50. ResNet50 is known for its residual learning technology, which can capture a wide range of features and learn complex patterns in data. Key features of ResNet50 include: (a) residual connections and skip connections introduced through residual learning, which mitigate the vanishing gradient problem in deep networks; (b) a bottleneck architecture. ResNet50's design reduces computational complexity by using 1x1 convolutional layers before and after the 3x3 layers of each residual block. This architecture reduces the number of parameters and computational cost, and improves network efficiency; and (c) robustness. The deep architecture and residual learning technology make the network more robust to overfitting.

[0047] In one embodiment of the present invention, image input 402 is in the form of RGB image data blocks, where the image data is captured from a target printed circuit board using a camera or directly input from an image file. The captured data is then passed to ResNet50 403 for feature extraction.

[0048] In another embodiment of the present invention, in a top-down approach, when each layer transitions from the top layer to the lower layer, the sampling rate of the top layer with the most semantic information and the lower layer with the least semantic information is reduced by a factor of 2, and the number of feature map channels is increased by a factor of 2. More specifically, the top-down approach is a key component of the present invention. For example, referring to Figure 4 In this paper, feature maps are extracted from the neural network backbone Cs, where s = 2 to 5 layers. The bottom layer is not used due to its high memory usage and low semantic information. Higher layers are used to provide rich semantic information. More specifically, the top layer P5 is generated by passing C5 through a 3x3 depthwise separable convolution kernel (MConv) and then sent directly to the region of interest alignment layer because it contains sufficient semantic information. The CP block, on the other hand, is a learnable module that performs feature extraction and upsampling of the feature maps. Therefore, the deep or top feature map P4 inherits the high semantics of C5 and has higher or satisfactory spatial resolution or information. P4-6 contain high semantics but have lower or suboptimal spatial resolution. The limited ability to detect small defects relies on these three layers. The bottom layer P2 is specifically designed to handle the problem of detecting small defects in printed circuit board images and is generated from three feature maps: C2, P2 (extracted from C3), and P3 (extracted from C4 and C5). This invention focuses on the largest feature map, C2, for detecting small objects. This is because spatial information is as important as semantic information.

[0049] In another embodiment of the present invention, the system further includes at least one learnable block 405 with 3x3 or 5x5 dimensions that directly communicates with the PSPyramid 406a, and the learnable block 405 is configured to perform feature map extraction and upsampling on feature maps with imperfect spatial information; and at least one depthwise separable convolution kernel (MConv) 401 with 3x3 dimensions that directly communicates with the PSPyramid 406a, and the convolution kernel is configured to send feature maps with imperfect semantic information to the PSPyramid for further processing by selective feature attention.

[0050] Based on the above embodiment, the learnable block 405 is a channel pruning block (CP block).

[0051] Combining ResNet50, learnable blocks, and MConv, the pyramid structure model (PSPyramid) becomes a learnable pyramid structure that can effectively extract semantic information and embed it into a larger feature map.

[0052] In another embodiment of the present invention, the selective feature attention is configured to fuse multiple weighted feature maps to form a fused feature map with balanced semantic and spatial information.

[0053] According to the above embodiment, the output 410 displays the defect classification and location on the target printed circuit board in the form of open circuit, short circuit and rat bite. The present invention can classify the defects on the printed circuit board into one of the following examples: Figure 5a The first output terminal on the printed circuit board is shown, more specifically, as a missing hole; Figure 5b The second output on the printed circuit board is shown, more specifically, the rat bite; Figure 5c The third output terminal on the printed circuit board is shown, more specifically, as an open circuit; Figure 5d A fourth output terminal on the printed circuit board is shown, more specifically, a short-circuited output; Figure 5e The fifth output on the printed circuit board is shown, more specifically, a spurious; Figure 5f The sixth output on the PCB is shown, more specifically, a pseudo copper; a detailed image of the result is shown in Figure 6.

[0054] Figure 6 depicts a comparison between different types of outputs produced by (1) ground truth, (2) our method, and (3) Faster R-CNN: A: open-circuit output; B: short-circuit output; C: mouse-bite output.

[0055] test To evaluate the performance of our algorithm, we conducted extensive experiments on two well-known datasets of printed circuit boards (PCBs) containing tiny defects, namely the Deep PCB and TDD datasets. In the Deep PCB dataset, we used both non-reference and reference methods, which are more feasible in industrial applications. In the TDD dataset, our method achieved a 36% reduction in error compared to the state-of-the-art TDDnet method. The experimental results demonstrate the effectiveness of our method in improving the quality assurance process for PCBs in the electronics industry. We compared the results on the Deep and TDD PCB datasets using different detection methods. The performance of different non-reference methods on the Deep PCB dataset is shown in Table 1, and it is clear that our method outperforms the standard model architecture, especially when using a high intersection-of-union threshold for the precise bounding box detection required in real-world applications.

[0056] Table 1 This method also outperformed most existing methods on the Deep Printed Circuit Board dataset. By aggregating multi-scale features, it significantly improved the performance of Faster R-CNN, increasing the average precision at 85% intersection-over-union (AP85) by over 10%. Furthermore, non-inference methods achieved an average precision of 98.7% at a high intersection-over-union of 33%.

[0057] On the other hand, the results obtained by the reference method according to the present invention are listed in Table 2. All the results were also calculated using AP33.

[0058] From the results, it is clear that the present invention achieves better performance in each category.

[0059] Table 2 Table 3 presents a comparative analysis of the mean average precision (mAP) of our method and several baseline methods at different intersection-of-union thresholds. Compared to the state-of-the-art model TDDNet, which achieves a very satisfactory AP50, our Faster R-CNN reduces the AP error of intersection-of-union from 1.1% (100%-98.9%) to 0.7% (100%-99.3%). This represents a 36% reduction in AP50 error. In terms of mAP, our method significantly outperforms previous models. In particular, when intersection-of-union is above 70 (a challenge for baseline models), our method also outperforms the previous best model, CornerNet, by 1.7%.

[0060] Table 3 This highlights the effectiveness of the invention in handling difficult situations where precise localization of tiny defects is required. PSPyramid helps aggregate multi-scale information without causing upsampling distortion, and selective feature attention can dynamically weight information to better predict tiny defects.

[0061] The present invention offers several advantages and is simple to implement, including: inputting an image of a target printed circuit board into a pretrained primary neural network; extracting multiple feature maps from the image input based on semantic and spatial information into a predetermined number of layers, generating layers of feature maps arranged from higher to lower levels in a top-down path; processing at least one predetermined feature map via a pyramid structure model coupled with selective feature attention; passing the processed feature map to a ROI aligner for normalization; and outputting and visualizing the processed layers via an output layer, which displays defect results on the target printed circuit board. The present invention has a learnable framework that focuses on the spatial relationships and dependencies of feature maps, particularly when detecting small defects on printed circuit boards at various resolutions, using a high cross-union threshold. The framework is simple and easy to operate, processes semantic and spatial information for certain tasks, and is particularly accurate and effective in detecting small defects on printed circuit boards at various resolutions.

[0062] The above explanation of the present invention is not limited to the aforementioned embodiments and drawings, and it is obvious to those skilled in the art that various substitutions, modifications and changes can be made without departing from the scope of the present invention.

Claims

1. A method for detecting defects in a target printed circuit board based on AI, characterized in that: include: Input the image of the target printed circuit board into the pre-trained primary neural network; Extract multiple feature maps from the image input into a predetermined number of layers based on semantic and spatial information, generating feature layers arranged from higher to lower levels in a top-down path; Processing at least one predetermined feature layer through a pyramid structure model coupled with selective feature attention; Pass the processed feature layer to the ROI aligner for normalization; as well as The layer after outputting and visualization processing is outputted through the output layer, and the defect results on the target printed circuit board are displayed by the output layer.

2. The method for detecting defects in a target printed circuit board based on AI according to claim 1, characterized in that: The generating of feature layers arranged from higher to lower in a top-down path includes: generating feature layers from higher to lower with progressive semantic and spatial information.

3. The method for detecting defects in a target printed circuit board based on AI according to claim 2, characterized in that: The method of processing at least one predetermined feature layer by a pyramid structure model coupled with selective feature attention includes: selecting at least one feature layer from a higher feature layer, and passing a feature map in the feature layer to a deep convolution kernel.

4. The method for detecting defects in a target printed circuit board based on AI according to claim 2, characterized in that: The processing of at least one predetermined feature layer by a pyramid structure model coupled with selective feature attention includes: Select at least one feature layer from the upper feature layer and the lower feature layer respectively; Upsampling feature maps in a feature layer selected from a higher feature layer; Fusing the upsampled feature map with a feature map in a feature layer selected from the lower feature layer to form a fused feature map; Passing the fused feature map to global pooling to form a vector; Compress the vector from c dimension to z dimension; Separating the compressed vector into a first separated vector and a second separated vector of c dimension; Multiplying the first separation vector and the upsampled feature map to generate a weighted first feature map; Multiplying the second separating vector by a selected feature map in a lower feature layer to generate a weighted second feature map; The weighted first feature map and the second weighted feature map are fused to form a fused feature map.

5. The method for detecting defects of a target printed circuit board based on AI according to claim 4, characterized in that: The selecting at least one feature layer from the higher feature layer and the lower feature layer further comprises the following steps: The feature maps in the feature layers selected from the higher feature layers have higher semantic information and lower spatial information; The feature maps in the feature layer selected from the lower feature layer have lower semantic information and higher spatial information.

6. The method for detecting defects in a target printed circuit board based on AI according to claim 5, characterized in that: Also includes: At least one feature map in the lowest feature layer with large memory footprint and low semantic information is excluded.

7. The method for detecting defects of a target printed circuit board based on AI according to claim 4, 5 or 6, characterized in that: Upsampling feature maps in a feature layer selected from a higher feature layer, including: Pass the selected feature map to the learnable block for processing; Outputting the processed feature maps to a channel size similar to four times the size of the selected feature maps from the lower feature layers; and The processed feature map is magnified by 2 times through the pyramid structure model to form an upsampled feature map.

8. The method for detecting defects of a target printed circuit board based on AI according to claim 4, characterized in that: The processing of at least one predetermined feature layer by the pyramid structure model coupled with selective feature attention further comprises the following steps: At least one fused feature map is generated from the corresponding feature map in the preset feature layer, and the generated fused feature map has no direct interaction with its corresponding feature map.

9. The method for detecting defects of a target printed circuit board based on AI according to claim 8, characterized in that: The processing of at least one predetermined feature layer by the pyramid structure model coupled with selective feature attention further comprises the following steps: Selected feature maps are learned and utilized from higher feature layers with higher semantic and spatial information to detect minute defects in target printed circuit boards.

10. The method for detecting defects of a target printed circuit board based on AI according to claim 1, characterized in that: The pre-trained primary neural network is trained using a plurality of training images having a plurality of semantic and spatial information from a plurality of defective printed circuit boards, and the pre-training of the primary neural network further comprises the following steps: Downsample the training images to multiple scales; Fusing downsampled training images using a pyramid structure model coupled with selective feature attention to fuse feature maps from higher and lower feature layers; Passing the fused feature layer of the training image to the region of interest alignment; generating a result that includes the defect; and The results are stored in the primary neural network for further learning.

11. A system for detecting defects in a target printed circuit board based on AI, characterized in that: include: A processor and a non-transitory medium including machine-readable instructions executable by the processor, including an extraction module for extracting a plurality of feature maps from an image input into a predetermined number of layers based on semantic and spatial information, generating feature map layers arranged from higher to lower layers in a top-down path; a processing module, configured to process at least one predetermined feature layer through a pyramid structure model coupled with selective feature attention; Normalization module, used to pass the processed feature map to the ROI aligner for normalization; as well as The output module is used to output and visualize the processed layer through the output layer, and the output layer displays the defect results on the target printed circuit board.

12. The system for detecting defects in a target printed circuit board based on AI according to claim 11, characterized in that: The AI ​​further includes a pre-trained primary neural network, which is a deep convolutional neural network including ResNet50.

13. The system for detecting defects in a target printed circuit board based on AI according to claim 11, characterized in that: The image of the target printed circuit board is collected using a camera or an image file.

14. The system for detecting defects in a target printed circuit board based on AI according to claim 11, characterized in that: The feature layers from higher to lower include: feature layers from higher to lower with progressive semantic and spatial information. When transitioning from the top layer to the lower layer, the sampling rate of the top layer with the most semantic information and the lower layer with the least semantic information is reduced by 2 times, and the number of feature map channels is increased by 2 times.

15. The system for detecting defects in a target printed circuit board based on AI according to claim 11, wherein: The pyramid structure model coupled with selective feature attention includes: at least one learnable block having 3×3 or 5×5 dimensions, the learnable block configured to perform feature map extraction and upsampling on a feature map having imperfect spatial information; and At least one depth-wise separable convolution kernel with 3×3 dimensions, configured to send feature maps with poor semantic information to further processing for selective feature attention.

16. The system for detecting defects of a target printed circuit board based on AI according to claim 11 or 15, characterized in that: The learnable block is a CP block.

17. The system for detecting defects in a target printed circuit board based on AI according to claim 11 or 15, characterized in that: The selective feature attention is configured to fuse multiple weighted feature maps to form a fused feature map with balanced semantic and spatial information.

18. The system for detecting defects of a target printed circuit board based on AI according to claim 11 or 15, characterized in that: The output module displays the defect classification and location on the target printed circuit board in the form of open circuit, short circuit and rat bite.

Images