Circuit board defect detection method and electronic device

Abstract

The application discloses a circuit board defect detection method and electronic equipment, and relates to the technical field of defect detection. Image information of a to-be-detected circuit board is collected, a deep convolutional neural network is used to extract a multi-resolution feature map, and feature information of different scales of the circuit board is captured. Further, the feature map is input into a multi-scale feature fusion network to generate a multi-layer pyramid feature map, and the multi-scale perception ability for defects such as virtual welding is enhanced. Then, the multi-layer pyramid feature map is input into a cross-layer feature fusion network to generate a fusion feature map, and the feature information of different levels is further integrated to improve the expression ability of the features. Finally, the fusion feature map is input into a detector network to accurately detect the defect position and defect type of the to-be-detected circuit board, effectively solves the problem that internal virtual welding and small target virtual welding are prone to be missed, and significantly improves the accuracy and reliability of detection.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and in particular to a method for detecting defects in circuit boards and electronic devices. Background Technology

[0002] As the carrier of electronic components and the bridge for electrical connections, the soldering quality of circuit boards directly affects the performance and reliability of equipment. Poor soldering is a common and extremely harmful problem in the circuit board soldering process. Essentially, it occurs when there is an isolation layer between the solder and the pin at the solder joint, resulting in only a small amount of solder pillars, leading to poor contact and unstable connections.

[0003] In related technologies, industrial cameras are used to capture visible light images of circuit boards, and algorithms such as edge detection and template matching are used to identify solder joint morphological defects (such as solder pad detachment and insufficient solder), but it is difficult to detect internal cold solder joints; infrared detection based on machine learning extracts manual features such as temperature mean, gradient, and texture from infrared images, and identifies cold solder joints through classifiers such as random forests, but it has insufficient generalization ability for complex heat distribution scenarios, and small target cold solder joints are easy to miss. Summary of the Invention

[0004] This application provides a circuit board defect detection method and electronic equipment to at least solve the problems in related technologies where it is difficult to detect internal cold solder joints and small target cold solder joints are easily missed.

[0005] This application provides a circuit board defect detection method, comprising: acquiring image information of the circuit board to be inspected; inputting the image information into a deep convolutional neural network to extract multi-resolution feature maps; inputting the multi-resolution feature maps into a multi-scale feature fusion network to generate multi-layer pyramid feature maps, wherein the multi-layer pyramid feature maps are arranged from top to bottom, and the resolution of the upper pyramid feature map is lower than that of the lower pyramid feature map; inputting the multi-layer pyramid feature maps into a cross-layer feature fusion network to generate a fused feature map; and inputting the fused feature map into a detector network to detect the defect location and defect type of the circuit board to be inspected.

[0006] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the aforementioned circuit board defect detection method.

[0007] This application discloses a circuit board defect detection method and electronic device, relating to the field of defect detection technology. It involves acquiring image information of the circuit board to be inspected, extracting multi-resolution feature maps using a deep convolutional neural network to capture feature information at different scales of the circuit board; further, the feature maps are input into a multi-scale feature fusion network to generate multi-layer pyramid feature maps, enhancing the multi-scale perception capability for defects such as cold solder joints; subsequently, the multi-layer pyramid feature maps are input into a cross-layer feature fusion network to generate fused feature maps, further integrating feature information at different levels and improving the expressive power of the features; finally, the fused feature maps are input into a detector network to accurately detect the defect location and defect type of the circuit board to be inspected, effectively solving the problem of easily missed detection of internal cold solder joints and small-target cold solder joints, and significantly improving the accuracy and reliability of detection. Attached Figure Description

[0008] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a flowchart of a circuit board defect detection method according to some embodiments of this application;

[0010] Figure 2 This is a schematic diagram of the feature pyramid structure according to some embodiments of this application;

[0011] Figure 3 This is a schematic diagram illustrating cross-layer feature fusion according to some embodiments of this application;

[0012] Figure 4 This is a block diagram of an electronic device according to some embodiments of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0014] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0015] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] The specific application environment architecture or specific hardware architecture on which the circuit board defect detection method depends is described here.

[0017] The circuit board defect detection method and electronic device according to embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0018] Figure 1 This is a flowchart of a circuit board defect detection method according to some embodiments of this application. (Refer to...) Figure 1 The circuit board defect detection method of this application embodiment may include the following steps:

[0019] S110 acquires image information of the circuit board to be tested.

[0020] Specifically, image information of the circuit board to be inspected can be acquired using industrial cameras, such as line scan cameras and area scan cameras, or infrared image information of the circuit board to be inspected can be acquired using a high-precision infrared thermal imager.

[0021] In this embodiment of the application, the infrared image information of the circuit board to be tested is acquired by a high-precision infrared thermal imager as an example for illustration, but this is not intended to limit the scope of the application.

[0022] S120 inputs image information into a deep convolutional neural network to extract multi-resolution feature maps.

[0023] Specifically, after acquiring infrared image information of the circuit board to be inspected, the infrared image information is input into a deep convolutional neural network for feature extraction, such as a ResNet-50 (Residual Network-50) network, to extract multi-level feature maps, with each level of feature map having a different resolution, for example, referring to... Figure 2 C2 is a low-level feature map with high resolution and low semantic information, and C5 is a high-level feature map with low resolution and high semantic information.

[0024] During feature extraction, the deep convolutional neural network uses small-sized convolutional kernels and high-resolution feature maps to capture basic visual features such as the edge contours of solder joints, surface texture details, and local hot spots caused by temperature gradient changes. These low-level features provide key information for identifying subtle morphological differences in solder joint defects. As the network layers deepen, the feature map resolution decreases while the number of channels increases. The network begins to integrate low-level features to form a more abstract semantic expression.

[0025] Deep convolutional neural networks, based on residual learning mechanisms, effectively avoid the gradient vanishing and degradation problems common in deep network training through a unique skip connection structure. This ensures training stability while enabling efficient mining of infrared image features.

[0026] S130: Input the multi-resolution feature map into the multi-scale feature fusion network to generate a multi-layer pyramid feature map. The multi-layer pyramid feature maps are arranged from top to bottom, and the resolution of the upper pyramid feature map is lower than that of the lower pyramid feature map.

[0027] Specifically, based on feature extraction from deep convolutional neural networks, a multi-scale feature fusion network, such as a feature pyramid network, is introduced to generate multi-layered pyramid feature maps, as shown below. Figure 2 The pyramid feature maps from top to bottom are P5-P2, with P5 having the lowest resolution and P2 having the highest resolution.

[0028] For example, feature pyramid networks such as Figure 2As shown, the feature pyramid network includes two paths, bottom-up and top-down, to achieve the fusion of feature maps of different resolutions. The bottom-up path utilizes feature maps from each layer of the ResNet-50 network, from the high-resolution, low-semantic-information low-level feature map C2 to the low-resolution, high-semantic-information high-level feature map C5. The top-down path upsamples the high-level feature maps and then horizontally connects and fuses them with the lower-level feature maps of the same resolution. For example, if the pyramid feature map P5 is the same as feature map C5, the pyramid feature layer P5 is upsampled to match the resolution of feature map C4, and then horizontally connected and fused with feature map C4 to generate pyramid feature map P4. Then, pyramid feature map P4 is upsampled to match the resolution of feature map C3, and then horizontally connected and fused with feature map C3 to generate pyramid feature map P3. Finally, pyramid feature map P3 is upsampled to match the resolution of feature map C2, and then horizontally connected and fused with feature map C2 to generate pyramid feature map P2. In this way, the pyramid feature map at each level contains rich semantic information and retains high-resolution detailed features, which can effectively meet the detection needs of solder joint defects at different scales on the circuit board. Solder joint defects from small component solder joints to large solder joints can be accurately captured.

[0029] S140, input the multi-layer pyramid feature map into the cross-layer feature fusion network to generate a fused feature map.

[0030] Specifically, after generating multi-layer pyramid feature maps, these maps (e.g., pyramid feature maps P5, P4, P3, and P2) can be input into a cross-layer feature fusion network. The network then fuses these pyramid feature maps, for example, by upsampling P5 and P4, downsampling P2, and then performing pointwise convolutions between the upsampled P5 and P4, the downsampled P2, and P3 to generate a fused feature map. Ultimately, the fused features possess both detailed accuracy and global integrity, achieving a complementary information fusion effect.

[0031] S150 inputs the fused feature map into the detector network to detect the location and type of defects on the circuit board to be inspected.

[0032] Specifically, after generating the fused feature map, the fused feature map is input into the detector network, such as the YOLOX (YouOnly Look Once – eXtended) detector head. Through a series of convolution and pooling operations, the fused features are further extracted and integrated. Using the anchor box mechanism and classification and regression network, the defect location and defect type of the circuit board to be inspected are determined, such as the location, type and probability of solder joints, and finally the fast and accurate detection of solder joints on the circuit board is achieved.

[0033] This application addresses the problem of difficulty in detecting internal cold solder joints and the tendency to miss small-target cold solder joints. It acquires image information of the circuit board under test and uses a deep convolutional neural network to extract multi-resolution feature maps, capturing feature information at different scales of the circuit board. Furthermore, the feature maps are input into a multi-scale feature fusion network to generate multi-layer pyramid feature maps, enhancing the multi-scale perception capability of defects such as cold solder joints. Subsequently, the multi-layer pyramid feature maps are input into a cross-layer feature fusion network to generate a fused feature map, further integrating feature information from different levels and improving feature expressiveness. Finally, the fused feature map is input into a detector network to accurately detect the location and type of defects on the circuit board under test, effectively solving the problem of easily missed internal cold solder joints and small-target cold solder joints, and significantly improving the accuracy and reliability of detection.

[0034] In some embodiments, the cross-layer feature fusion network includes a first convolutional neural network, a second convolutional neural network, and a third convolutional neural network. Inputting multi-layer pyramid feature maps into the cross-layer feature fusion network to generate a fused feature map includes: arbitrarily selecting three pyramid feature maps from the multi-layer pyramid feature maps, wherein the three pyramid feature maps include a first pyramid feature map, a second pyramid feature map, and a third pyramid feature map, the resolution of the first pyramid feature map being less than the resolution of the second pyramid feature map, and the resolution of the second pyramid feature map being less than the resolution of the third pyramid feature map; performing upsampling processing on the first pyramid feature map based on the first convolutional neural network to determine a first target pyramid feature map, the first target pyramid feature map having the same resolution as the second pyramid feature map; performing downsampling processing on the third pyramid feature map based on the second convolutional neural network to determine a second target pyramid feature map, the second target pyramid feature map having the same resolution as the second pyramid feature map; and inputting the first target pyramid feature map, the second target pyramid feature map, and the second pyramid feature map into the third convolutional neural network to generate a fused feature map.

[0035] Specifically, fusing all pyramid feature maps would significantly increase computational complexity, thus affecting the speed of defect detection. Therefore, three layers of pyramid feature maps can be arbitrarily selected and input into a cross-layer feature fusion network for feature fusion, which reduces computational complexity while increasing the detail accuracy and global completeness of the fused feature map.

[0036] For example, refer to Figure 3 Suppose that three pyramid feature maps are arbitrarily selected from the multi-layer pyramid feature maps as pyramid feature map P4, pyramid feature map P3 and pyramid feature map P2. Among them, the resolution of pyramid feature map P4 is less than the resolution of pyramid feature map P3, and the resolution of pyramid feature map P3 is less than the resolution of pyramid feature map P2. That is to say, pyramid feature map P4 is the first pyramid feature map, pyramid feature map P3 is the second pyramid feature map, and pyramid feature map P2 is the third pyramid feature map.

[0037] The first target pyramid feature map (P4) is upsampled using a first convolutional neural network, such as U-Net, to generate the first target pyramid feature map. The third target pyramid feature map (P2) is downsampled using a second convolutional neural network, such as VGGNet (Visual Geometry Group Network), to generate the second target pyramid feature map. The resolutions of the first and second target pyramid feature maps are the same as those of the second pyramid feature map. Finally, the first, second, and third target pyramid feature maps are fused using a third convolutional neural network to generate a fused feature map.

[0038] In some embodiments, the first convolutional neural network is a deformable convolutional neural network, the second convolutional neural network is a dilated convolutional neural network, and the third convolutional neural network is a pointwise convolutional neural network.

[0039] Specifically, the first convolutional neural network is a deformable convolutional neural network. This network uses an offset learning mechanism to dynamically adapt the sampling positions of the convolutional kernels. For each spatial location in the input feature map, the network autonomously learns a set of two-dimensional offset parameters (Δx, Δy). These parameters are optimized together with the loss function through backpropagation, enabling the sampling points of the convolutional kernels to pay more attention to key details such as irregular contours of solder joint edges and local areas of small hot spots. Compared to conventional upsampling methods (such as bilinear interpolation or deconvolution) that rely on fixed weights for pixel filling, deformable convolution can adjust the sampling strategy according to the spatial distribution characteristics of solder joint details. For tiny hot spots only a few pixels in size, the sampling points will cluster towards the high grayscale region at the center of the hot spot, avoiding the blurring of hot spot contours caused by pixel averaging in conventional upsampling.

[0040] The second convolutional neural network is a dilated convolutional neural network. To better extract semantic information from deep, low-resolution feature maps, dilated convolution introduces a dilation rate parameter, enabling flexible adjustment of the receptive field and information preservation during downsampling. While deep feature maps (e.g., the pyramid feature map P2) already contain rich semantic information, their spatial resolution is low due to downsampling operations. Conventional convolution results in a fixed receptive field, making it difficult to cover defects over a larger spatial area. Dilated convolution, by inserting a fixed number of zeros between kernel elements, can expand the receptive field without increasing the kernel parameter size. This design allows the network to capture global features at different scales by adjusting the dilation rate while maintaining a moderate reduction in feature map resolution during downsampling. This effectively avoids the loss of semantic information caused by conventional downsampling operations, ultimately accurately extracting macroscopic defect features such as the gradient change trend of the overall thermal distribution of the solder joint and large-scale temperature anomaly areas.

[0041] The third convolutional neural network is a pointwise convolutional neural network (1×1 convolution). The three pyramid feature maps correspond to different levels of feature representation. The shallowest layer (first target pyramid feature map) focuses on details such as edge texture and local pixel differences of the solder joint. The middle layer (second pyramid feature map) contains the local structural morphological features of the solder joint. The deepest layer (second target pyramid feature map) provides the overall semantics and general distribution. By aligning the three layers in the spatial dimension and then stacking them in the channel dimension, multi-scale information can be initially integrated. However, this results in excessive redundant information and a significant increase in model computation. The pointwise convolutional neural network can compress the stacked high-dimensional channels to the target dimension, achieving feature dimension unification and reducing computation. At the same time, the pointwise convolutional neural network can filter out the most discriminative features in the channels, suppressing repetitive or noisy information. Ultimately, the fused features possess both detailed accuracy and global integrity, achieving a complementary information fusion effect.

[0042] In some embodiments, inputting multi-resolution feature maps into a multi-scale feature fusion network to generate multi-layer pyramid feature maps includes: performing multi-scale upsampling on the multi-resolution feature maps using a top-down path to generate multi-layer upsampled feature maps; and laterally connecting the multi-layer upsampled feature maps with feature maps of corresponding resolutions to generate multi-layer pyramid feature maps.

[0043] Specifically, in order to make the pyramid feature maps at each level contain richer semantic information and more detailed features, multi-scale upsampling of the multi-resolution feature maps can be performed using a top-down path (e.g., using a deformable convolutional neural network for upsampling) to obtain feature maps of multiple resolutions. The multi-layer upsampled feature maps are then horizontally connected with the feature maps of the corresponding resolutions to generate multi-layer pyramid feature maps.

[0044] For example, refer to Figure 2 For feature map C5, multi-scale upsampling is performed to obtain feature maps of multiple resolutions, specifically generating feature maps VI, VIII, and VIII. Feature map VI has the same resolution as feature map C4, feature map VIII has the same resolution as feature map C3, and feature map VIII has the same resolution as feature map C2. Further, feature map VI is horizontally concatenated with feature map C4 to generate pyramid feature map VIIV, feature map VIII is horizontally concatenated with feature map C3 to generate pyramid feature map VIIIIII, and feature map VIII is horizontally concatenated with feature map C2 to generate pyramid feature map VIIIIII. For feature map C4, multi-scale upsampling is performed to obtain feature maps of multiple resolutions, specifically generating feature maps VIIV. Figure IV I and characteristics Figure IV II, among which, characteristics Figure IV Ⅰ. Same resolution as feature map C3, feature Figure IV II has the same resolution as feature map C2. Furthermore, the features... Figure IV I. Horizontally connect with feature map C3 to generate pyramid features. Figure IV I and III, features Figure IV II is horizontally connected to feature map C2 to generate a pyramid feature. Figure IV II. And so on, generating multi-layered pyramid feature maps.

[0045] In addition, feature maps can be horizontally concatenated with pyramid feature maps of corresponding resolution to generate pyramid feature maps. For example, if the resolution of pyramid feature map VIIV is the same as that of feature map C4, feature map C4 can also be horizontally concatenated with pyramid feature map VIIV to generate a pyramid feature map; upsampled features Figure IV I has the same resolution as pyramid feature maps V, II, and III, therefore the features can also be... Figure IVI is horizontally connected to the pyramid feature maps V, II, and III to generate a pyramid feature map.

[0046] Alternatively, pyramid feature maps can be upsampled at multiple scales, and the upsampled pyramid feature maps can be laterally concatenated with pyramid feature maps of corresponding resolutions to generate a new pyramid feature map. For example, pyramid feature map VIIV can be upsampled, and the upsampled pyramid feature map VIIV can be concatenated with the pyramid feature map of the corresponding resolution. Figure IV ⅠⅢ are horizontally connected to generate a pyramid feature map.

[0047] Thus, by performing multi-scale feature fusion on multi-resolution feature maps, each level of the pyramid feature map can contain rich semantic information and retain more detailed features, thereby significantly improving the detector's ability and accuracy to detect targets at different scales. At the same time, through flexible feature combination and dynamic adjustment, the model is more efficient and accurate in handling complex scenes, enhancing its ability to identify various defects on circuit boards.

[0048] In some embodiments, the multi-layer upsampled feature maps are laterally connected with the feature maps of corresponding resolutions to generate multi-layer pyramid feature maps, including: performing attention recalibration on each upsampled feature map and the feature map of corresponding resolution to obtain corresponding weighted feature maps; and performing pointwise convolution on the corresponding weighted feature maps after element-wise summation to generate multi-layer pyramid feature maps.

[0049] Specifically, attention mechanisms, such as the Squeeze-and-Excitation (SE) module, can be used to calculate the correlation between the upsampled feature map and the feature map at the corresponding resolution, generating a weight matrix. Then, the weight matrix is ​​used to weight both the upsampled and corresponding resolution feature maps for attention recalibration, thereby enhancing important features and suppressing unimportant ones. Next, the attention-recalibrated upsampled and corresponding resolution feature maps are added element-wise to obtain a fused feature map, combining high-level semantic information with low-level spatial detail information. Finally, the fused feature map is convolved pointwise to adjust the number of channels and optimize the feature map quality, generating a pyramid feature map. This process is repeated to generate multi-layered pyramid feature maps.

[0050] Thus, feature fusion technology based on attention mechanism can significantly improve the model's ability to detect targets at multiple scales, increase detection accuracy, and optimize computational efficiency.

[0051] In some embodiments, the fused feature map is input into a detector network to detect the location and type of defects on the circuit board to be inspected, including: dividing the fused feature map into regions to obtain multiple local regions; extracting the grayscale value of each local region and determining a detection threshold for each local region based on the grayscale value; adjusting the classification confidence threshold of the detector network based on the detection threshold, and performing defect detection on the fused feature map based on the adjusted classification confidence threshold to determine the location and type of defects on the circuit board to be inspected.

[0052] Specifically, the fused feature map contains various feature information of the circuit board to be inspected. By dividing the fused feature map into regions, it can be segmented into multiple local regions. Each local region corresponds to a specific part of the circuit board to be inspected, such as a solder joint region, a trace region, or a component region. Region segmentation can be achieved using image segmentation algorithms, such as threshold-based segmentation, edge detection, region growing, or deep learning segmentation algorithms.

[0053] For each local region, the local region can be converted into a grayscale image to read the grayscale value of each pixel in the local region and build a grayscale histogram. By looking at the distribution of grayscale values, the lowest point between two peaks is found as the detection threshold of the local region. The detection threshold is used to distinguish different parts of the local region, such as background and target (such as solder joint).

[0054] Furthermore, the classification confidence threshold of the detector network is adjusted using a detection threshold. The classification confidence threshold is used to determine which predictions are considered valid. If the detection threshold is high (e.g., close to 255), it indicates that the grayscale difference between the background and the target in the local area is large. In this case, the classification confidence threshold can be increased (e.g., adjusted from 0.5 to 0.7 or 0.8) to reduce false alarms. If the detection threshold is low (e.g., close to 0), it indicates that the grayscale difference between the background and the target in the local area is small. In this case, the classification confidence threshold can be decreased (e.g., adjusted from 0.5 to 0.3 or 0.4) to reduce false negatives.

[0055] It should be noted that each local region has a corresponding detection threshold. Therefore, the classification confidence threshold can be adjusted separately for each local region and applied separately during detection.

[0056] Thus, by dividing the fused feature map into regions, extracting the grayscale values ​​of each local region and establishing a grayscale histogram, the detection threshold is determined. Then, the classification confidence threshold of the detector network is adjusted according to the detection threshold to optimize the accuracy of defect detection, reduce false alarms and false negatives, and improve the performance and reliability of circuit board defect detection.

[0057] In some embodiments, the above method further includes: acquiring a historical image dataset of the circuit board, labeling the defect categories and defect locations of the circuit board in the historical image dataset to construct a model training set; using the model training set to jointly train a deep convolutional neural network, a multi-scale feature fusion network, a cross-layer feature fusion network, and a detector network to obtain a preset defect recognition model; and inputting image information into the preset defect recognition model to determine the defect location and defect type of the circuit board to be detected.

[0058] Specifically, a method of artificially simulating solder joint defects is used to manufacture circuit boards containing different types of solder joint defects (such as partial contact, solder joint cracking, solder joint oxidation, etc.), and infrared images of the circuit boards are acquired using a high-precision infrared thermal imager. Then, the acquired images are professionally labeled to accurately mark the location and type of normal solder joints and solder joint defects, ultimately forming a historical infrared image dataset containing a large number of samples. The historical infrared image dataset can then be divided into training set, validation set, and test set in a 7:1:2 ratio, and the data scale can be further expanded through data augmentation techniques (random flipping, elastic deformation, thermal image distortion) to ensure that the model can learn comprehensive solder joint feature patterns during training, effectively improving generalization performance and adaptability to complex scenarios.

[0059] After constructing the model training set, the deep convolutional neural network, multi-scale feature fusion network, cross-layer feature fusion network and detector network are jointly trained using the model training set. The model hyperparameters (such as learning rate, regularization parameter, network structure, etc.) are adjusted using the validation set. After the model training and validation are completed, the preset defect recognition model is evaluated using the test set.

[0060] After the evaluation is passed, the preset defect recognition model is deployed to the electronic device, and the infrared image information of the circuit board to be inspected is input into the preset defect recognition model to determine the defect location and defect type of the circuit board to be inspected.

[0061] Thus, by acquiring and labeling historical infrared image datasets of circuit boards, a model training set is constructed. This allows for the joint training of a deep convolutional neural network, a multi-scale feature fusion network, a cross-layer feature fusion network, and a detector network, resulting in a pre-defined defect recognition model. Using artificially manufactured circuit boards with solder defects and high-precision infrared thermal imagers to acquire images, the location and type of defects are precisely marked through professional annotation, forming a large-scale dataset. Data augmentation techniques are employed to expand the data scale, improving the model's generalization ability and adaptability to complex scenarios. Hyperparameters are adjusted using a validation set to ensure model performance, and the model is evaluated on a test set. Finally, the model is deployed to electronic devices, enabling rapid and accurate detection of circuit board defect locations and types, significantly improving detection accuracy and efficiency.

[0062] In some embodiments, the method further includes: inputting any input data from the model training set into an initial preset defect identification model to output the predicted defect location and predicted defect type of the circuit board; calculating the discrimination loss based on the predicted defect location and predicted defect type of the circuit board and the labeled defect location and labeled defect type of the circuit board corresponding to the model training set to obtain the calculation result; updating the model parameters of the initial preset defect identification model using the calculation result until the updated initial preset defect identification model meets the preset convergence condition to obtain the preset defect identification model.

[0063] For example, any input data from the model training set is fed into an initial preset defect recognition model for forward propagation to obtain the predicted defect location and predicted defect type of the output circuit board. Based on the predicted defect location and type of the circuit board, and the labeled defect location and type corresponding to the input data, a discrimination loss is calculated to obtain the loss value. The gradient of the loss function with respect to each parameter is calculated through backpropagation, and the model parameters of the initial preset defect recognition model are updated according to the gradient using a preset optimization algorithm (such as Adam). The above steps are repeated until the updated preset defect recognition model meets preset convergence conditions, such as the loss value reaching a preset loss threshold or the number of iterations reaching a preset iteration threshold, resulting in a preset defect recognition model used to detect defect locations and defect types.

[0064] In some embodiments, obtaining a historical image dataset of a circuit board includes: acquiring images of the circuit board under different shooting angles, different rated currents of the circuit board, and different input noise conditions to generate a historical image dataset of the circuit board.

[0065] Specifically, by adjusting the shooting angle of the infrared imager (0°-45°), changing the current of the circuit board (5%-150% of the rated current), and introducing a controllable noise source (simulating thermal imaging noise caused by electromagnetic interference), a historical infrared image dataset under multiple environmental conditions can be generated.

[0066] This significantly enhances data diversity, improves the model's generalization ability and detection accuracy, and optimizes the model training process.

[0067] In summary, this application addresses the problem of difficulty in detecting internal cold solder joints and the tendency to miss small-target cold solder joints. It acquires image information of the circuit board under test and uses a deep convolutional neural network to extract multi-resolution feature maps, capturing feature information at different scales of the circuit board. Furthermore, these feature maps are input into a multi-scale feature fusion network to generate multi-layer pyramid feature maps, enhancing the multi-scale perception capability of defects such as cold solder joints. Subsequently, the multi-layer pyramid feature maps are input into a cross-layer feature fusion network to generate a fused feature map, further integrating feature information from different levels and improving feature expressiveness. Finally, the fused feature map is input into a detector network to accurately detect the defect location and defect type of the circuit board under test, effectively solving the problem of easily missing internal cold solder joints and small-target cold solder joints, and significantly improving the accuracy and reliability of detection.

[0068] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0069] Embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned circuit board defect detection method.

[0070] It should be noted that the above explanation of the embodiments and beneficial effects of the circuit board defect detection method also applies to the computer-readable storage medium of the embodiments of this application. To avoid redundancy, it will not be elaborated in detail here.

[0071] Embodiments of this application also provide an electronic device.

[0072] Reference Figure 4 The electronic device 300 includes a memory 310, a processor 320, and a computer program stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program, it implements the aforementioned circuit board defect detection method.

[0073] It should be noted that the above explanation of the embodiments and beneficial effects of the circuit board defect detection method also applies to the electronic devices of the embodiments of this application. To avoid redundancy, they will not be elaborated in detail here.

[0074] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0075] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0076] The present application provides a detailed description of a circuit board defect detection method and electronic device. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for detecting defects in a circuit board, characterized in that, The method includes: Acquire image information of the circuit board to be inspected; The image information is input into a deep convolutional neural network to extract multi-resolution feature maps; The multi-resolution feature map is input into a multi-scale feature fusion network to generate a multi-layer pyramid feature map, which is arranged from top to bottom, and the resolution of the upper pyramid feature map is lower than that of the lower pyramid feature map. The multi-layer pyramid feature maps are input into a cross-layer feature fusion network to generate a fused feature map; The fused feature map is input into the detector network to detect the location and type of defects on the circuit board to be inspected. The cross-layer feature fusion network includes a first convolutional neural network, a second convolutional neural network, and a third convolutional neural network. Multiple layers of the pyramid feature maps are input into the cross-layer feature fusion network to generate a fused feature map, including: Three layers of pyramid feature maps are arbitrarily selected from the multi-layered pyramid feature maps, wherein the three layers of pyramid feature maps include a first pyramid feature map, a second pyramid feature map, and a third pyramid feature map, wherein the resolution of the first pyramid feature map is less than the resolution of the second pyramid feature map, and the resolution of the second pyramid feature map is less than the resolution of the third pyramid feature map. The first pyramid feature map is upsampled based on the first convolutional neural network to determine the first target pyramid feature map, which has the same resolution as the second pyramid feature map. The third pyramid feature map is downsampled based on the second convolutional neural network to determine the second target pyramid feature map, which has the same resolution as the second pyramid feature map. The first target pyramid feature map, the second target pyramid feature map, and the second pyramid feature map are input into the third convolutional neural network to generate the fused feature map; Wherein, the first convolutional neural network is a deformable convolutional neural network, the second convolutional neural network is a dilated convolutional neural network, and the third convolutional neural network is a pointwise convolutional neural network; The process of inputting the multi-resolution feature map into the multi-scale feature fusion network to generate a multi-layer pyramid feature map includes: The multi-resolution feature map is upsampled at multiple scales using a top-down path to generate multi-layer upsampled feature maps. The multi-layer upsampled feature maps are laterally connected with the feature maps of corresponding resolutions to generate the multi-layer pyramid feature maps; wherein, the feature maps of corresponding resolutions include the pyramid feature maps and the multi-resolution feature maps; The process of laterally concatenating the multi-layered upsampled feature maps with feature maps of corresponding resolutions to generate the multi-layered pyramid feature maps includes: Attention recalibration is performed on the upsampled feature map and the feature map of the corresponding resolution for each layer to obtain the corresponding weighted feature map; The corresponding weighted feature maps are summed element by element and then convolved point by point to generate multi-layered pyramid feature maps. The fused feature map is input into a detector network to detect the location and type of defects on the circuit board to be inspected, including: The fused feature map is divided into regions to obtain multiple local regions; Extract the grayscale value of each local region, and determine the detection threshold of each local region based on the grayscale value; The classification confidence threshold of the detector network is adjusted according to the detection threshold, and the fused feature map is used for defect detection based on the adjusted classification confidence threshold to determine the defect location and defect type of the circuit board to be detected.

2. The circuit board defect detection method according to claim 1, characterized in that, The method further includes: Obtain a historical image dataset of circuit boards, and label the defect categories and locations of the circuit boards in the historical image dataset to construct a model training set; The deep convolutional neural network, the multi-scale feature fusion network, the cross-layer feature fusion network, and the detector network are jointly trained using the model training set to obtain a preset defect recognition model. The image information is input into the preset defect recognition model to determine the location and type of defects on the circuit board to be inspected.

3. The circuit board defect detection method according to claim 2, characterized in that, The method further includes: Input any input data from the model training set into the initial preset defect recognition model to output the predicted defect location and predicted defect type of the circuit board; The discrimination loss is calculated based on the predicted defect location and predicted defect type of the circuit board and the labeled defect location and labeled defect type of the circuit board in the model training set to obtain the calculation result; The model parameters of the initial preset defect identification model are updated using the calculation results until the updated initial preset defect identification model meets the preset convergence condition, thus obtaining the preset defect identification model.

4. The circuit board defect detection method according to claim 2, characterized in that, Obtain a historical image dataset of the circuit board, including: Images of the circuit board are acquired under different shooting angles, different rated currents of the circuit board, and different input noise conditions to generate a historical image dataset of the circuit board.

5. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the circuit board defect detection method according to any one of claims 1-4.

Images