PCB welding spot detection method and device, electronic equipment and storage medium

Abstract

The invention provides a PCB welding spot detection method and device, electronic equipment and a storage medium. Obtaining a three-dimensional image of a to-be-detected welding point of the target PCB; acquiring a detection result corresponding to the three-dimensional image of the to-be-detected welding point based on a preset detection model; the detection result comprises normality or abnormity; when the detection result is abnormal, the detection result further comprises a defect image and a defect type; the preset detection model performs forward noise addition on the acquired three-dimensional image through a preset diffusion model, and subtracts the predicted noise from the noise-added three-dimensional image to obtain a reconstructed three-dimensional image; and inputting the reconstructed three-dimensional image and the acquired three-dimensional image into a preset defect detection and identification network model, and outputting a detection result corresponding to the welding point to be detected. According to the method, the welding spot detection of the PCB can be efficiently and accurately realized, and the defects can be positioned and classified.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and particularly to a method, apparatus, electronic device, and storage medium for detecting solder joints of a PCB. Background Art

[0002] Manufacturing is the core pillar of a country's economic system. In recent years, the manufacturing industry in China has developed rapidly, and its global share has been continuously rising. As an indispensable component of electronic devices, the Printed Circuit Board (PCB) plays a very important role in the electronics industry. However, the solder joints of the PCB are prone to defects, which affect the use and safety of the devices. Therefore, the defect detection of the solder joints of the PCB has become an indispensable key step in the manufacturing of electronic devices.

[0003] Currently, in the existing industrial non-destructive testing technology field at home and abroad, especially for the detection of solder joint defects of the PCB, traditional methods mainly rely on manual visual inspection or automatic detection technology based on two-dimensional images. However, when facing complex three-dimensional solder joint structures, these methods often have problems such as limited detection accuracy, difficulty in obtaining abnormal samples, slow detection speed, inability to accurately identify defect types, and unclear defect types. These problems not only affect the automation level and detection efficiency of the production line but also limit the further improvement of the solder joint quality detection technology. Therefore, there is an urgent need for a new method that can efficiently and accurately detect and classify defects in three-dimensional solder joint images to overcome the limitations of the existing technology and promote the innovation and development of the PCB solder joint quality detection technology. Summary of the Invention

[0004] In view of this, the present application provides a method, apparatus, electronic device, and storage medium for detecting solder joints of a PCB, which can efficiently and accurately detect the solder joints of the PCB and can locate and classify defects.

[0005] To solve the above technical problems, the technical solution of the present application is implemented as follows:

[0006] In one embodiment, a method for detecting solder joints of a PCB is provided, and the method includes:

[0007] Obtain a three-dimensional image of a solder joint to be detected on a target PCB;

[0008] Obtain a detection result corresponding to the three-dimensional image of the solder joint to be detected based on a preset detection model; the detection result includes normal or abnormal; when the detection result is abnormal, the detection result further includes a defect image and a defect type;

[0009] Among them, the preset detection model performs forward denoising on the obtained three-dimensional image through a preset diffusion model, and then subtracts the predicted noise from the denoised three-dimensional image to obtain a reconstructed three-dimensional image; and inputs the reconstructed three-dimensional image and the obtained three-dimensional image into a preset defect detection and recognition network model to output the detection result corresponding to the welding point to be detected; the predicted noise is predicted based on the noise added during the forward denoising process and the abnormal voxels in the obtained three-dimensional image disturbed by the added noise.

[0010] Among them, the preset detection model performs forward denoising on the obtained three-dimensional image through a preset diffusion model, and then subtracts the predicted noise from the denoised three-dimensional image to obtain a reconstructed three-dimensional image, including:

[0011] Downsample and encode the obtained three-dimensional image into a latent space representation through an encoder;

[0012] Perform forward denoising on the latent space representation to obtain a latent variable;

[0013] Input the latent variable into a preset denoising network to obtain the latent space representation of the reconstructed three-dimensional image; among them, the preset denoising network subtracts the variable corresponding to the predicted noise from the latent variable to obtain the latent space representation of the reconstructed three-dimensional image;

[0014] Use a decoder to obtain the reconstructed three-dimensional image corresponding to the latent space representation of the reconstructed three-dimensional image.

[0015] Among them, the preset denoising network is an improved U-net;

[0016] The improved U-net is: delete the non-linear activation function after the convolution operation of each convolution layer in the network residual structure of the original U-net, and embed a TmeEmdeding between two convolution operations to encode the time step t into the feature matrix; and after each convolution layer, add a SimpleGate and a CBAM attention layer.

[0017] Among them, the method further includes:

[0018] Generating the preset diffusion model includes:

[0019] Obtain a first training sample; the first training sample is the three-dimensional image of a first reference solder joint; the first reference solder joint is a normal solder joint; the label of the first training sample is 0;

[0020] Generate an initial diffusion model; the initial diffusion model includes an initial denoising network;

[0021] When training the initial diffusion model using the first training sample, optimize the predicted noise in the initial denoising network using a noise prediction loss function, and continuously optimize the parameters of the initial denoising network;

[0022] When the value of the noise prediction loss function is less than a first preset threshold, obtain a preset denoising network and a preset diffusion model;

[0023] Wherein, the value of the noise prediction loss function is calculated based on the label of the first training sample, and the predicted noise and the noise used in the forward noise addition process.

[0024] Wherein, the predicted defect detection and recognition network model includes: a preset 3D-Unet segmentation network and a classification decision layer;

[0025] The preset 3D-Unet segmentation network predicts voxel-level anomaly feature scores and a feature-level anomaly score map using the inconsistency and commonality between the three-dimensional image to be detected and the reconstructed three-dimensional image; and uses the feature anomaly score map to determine the defect location and shape, combines the three-dimensional image to be detected to determine the defect image, and outputs it; if the number of voxel-level anomaly feature scores is less than a preset number threshold, determine that the detection result is normal;

[0026] Perform classification decision processing on the voxel-level anomaly feature scores through the classification decision layer to determine the defect type.

[0027] Wherein, the classification decision layer includes a Softmax layer and an Argmax operation;

[0028] Normalize the voxel-level anomaly feature scores output by the predicted 3D-Unet segmentation network through the Softmax layer;

[0029] Use the Argmax operation to select the category label with the maximum value from the normalized result as the defect type.

[0030] Wherein, the method further includes:

[0031] Generating the preset detection model, including:

[0032] Obtain a second training sample; the second training sample is a three-dimensional image of a reference solder joint; the reference solder joint includes normal solder joints and abnormal solder joints;

[0033] Generate an initial detection model; wherein, the initial detection model includes: a preset diffusion model and an initial defect detection and recognition network model;

[0034] Train the initial prediction model using the second training sample; and after each training session, substitute the reference class label distribution and the predicted class label distribution into the cross-entropy loss function to calculate the value of the cross-entropy loss function;

[0035] When the value of the calculated cross-entropy loss function is less than the second preset threshold, obtain the preset defect detection and recognition network model, and the preset detection model.

[0036] In another embodiment, a solder joint detection device for a PCB is provided. The device includes:

[0037] An acquisition unit for acquiring a three-dimensional image of a solder joint to be detected on a target PCB;

[0038] A detection unit for obtaining a detection result corresponding to the three-dimensional image of the solder joint to be detected based on a preset detection model; the detection result includes normal or abnormal; when the detection result is abnormal, the detection result further includes a defect image and a defect type; wherein, the preset detection model performs forward denoising on the acquired three-dimensional image through a preset diffusion model, and uses the denoised three-dimensional image minus the predicted noise to obtain a reconstructed three-dimensional image; and inputs the reconstructed three-dimensional image and the acquired three-dimensional image into a preset defect detection and recognition network model to output the detection result corresponding to the solder joint to be detected; the predicted noise is predicted according to the noise added during the forward denoising process and the abnormal voxels in the acquired three-dimensional image affected by the added noise.

[0039] In another embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, a solder joint detection method for a PCB is implemented.

[0040] In another embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, a solder joint detection method for a PCB is implemented.

[0041] As can be seen from the above technical solutions, in the above embodiments, by acquiring a three-dimensional image of a solder joint to be detected on a target PCB board and inputting the three-dimensional image into a preset detection model, a corresponding detection result can be obtained. The detection result is normal or abnormal. If it is abnormal, the detection result further includes a defect image and a defect type; wherein the preset detection model inputs the reconstructed three-dimensional image output by the preset diffusion model and the acquired three-dimensional image into a preset defect detection and recognition network model to output the detection result corresponding to the solder joint to be detected; the reconstructed three-dimensional image is a normal image obtained by the diffusion model using the difference between the acquired three-dimensional image and the predicted noise. This solution can efficiently and accurately implement the solder joint detection of a PCB and can locate and classify defects. Brief Description of the Drawings

[0042] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0043] Figure 1 Schematic diagram of the preset detection model structure in the embodiments of the present application;

[0044] Figure 2(a) is a schematic diagram corresponding to each convolutional layer in the network residual structure of the original U-net;

[0045] Figure 2(b) is a schematic diagram corresponding to each convolutional layer in the improved U-net network residual structure in the embodiments of the present application;

[0046] Figure 3 Schematic diagram of the processing flow of the preset diffusion model for three-dimensional images in the embodiments of the present application;

[0047] Figure 4 Schematic diagram of the classification decision layer structure in the embodiments of the present application;

[0048] Figure 5 Schematic diagram of the process for generating the preset diffusion model in the embodiments of the present application;

[0049] Figure 6 Schematic diagram of the training process of the preset detection model in the embodiments of the present application;

[0050] Figure 7 Schematic diagram of the solder joint detection process for the PCB in the embodiments of the present application;

[0051] Figure 8 Schematic diagram of the structure of the solder joint detection device for the PCB in the embodiments of the present application;

[0052] Figure 9 Schematic diagram of the physical structure of the electronic device provided in the embodiments of the present invention. Detailed Description of the Embodiments

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0054] In the description, claims and above-mentioned drawings of the present invention, terms such as "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects, and do not necessarily describe the order or sequence of the target. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0055] The technical solution of the present invention will be described in detail below with specific embodiments. The following several specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0056] Traditional methods mainly rely on manual visual inspection or automatic detection techniques based on two-dimensional images. However, when facing complex three-dimensional solder joint structures, these methods often have problems such as limited detection accuracy, difficulty in obtaining abnormal samples, slow detection speed, inability to accurately identify defect types, and unclear defect types. These problems not only affect the automation level and detection efficiency of the production line, but also limit the further improvement of solder joint quality detection technology.

[0057] Based on the above problems, an embodiment of the present application provides a method for detecting solder joints of a PCB. By obtaining a three-dimensional image of the solder joints to be detected on the target PCB board and inputting the three-dimensional image into a preset detection model, the corresponding detection result can be obtained. The detection result is normal or abnormal. If it is abnormal, the detection result also includes a defect image and a defect type. Among them, the preset detection model inputs the reconstructed three-dimensional image output by the preset diffusion model and the obtained three-dimensional image into a preset defect detection and recognition network model, and outputs the detection result corresponding to the solder joint to be detected. The reconstructed three-dimensional image is a normal image obtained by the diffusion model by using the difference between the obtained three-dimensional image and the predicted noise. This solution can efficiently and accurately detect the solder joints of the PCB and can locate and classify the defects.

[0058] Before detecting the solder joints of the PCB, it is necessary to first obtain a preset detection model through model training. Refer to Figure 1 , Figure 1 which is a schematic diagram of the structure of the preset detection model in the embodiment of the present application. Figure 1 The preset detection model in

[0059] The three-dimensional images of the welding points to be detected on the target PCB obtained are respectively input into a preset diffusion model and a preset defect detection and recognition network model; the reconstructed three-dimensional image is obtained through the processing of the preset diffusion model and input into the preset defect detection and recognition network model;

[0060] The preset defect detection and recognition network model processes the three-dimensional images of the welding points to be detected and the reconstructed three-dimensional image input, and outputs the detection result. The detection result includes: normal and abnormal; when the detection result is abnormal, the detection result also includes the defect image and the defect type.

[0061] Among them, the preset diffusion model includes: an encoder, a noise addition network, a preset denoising network, and a decoder;

[0062] The preset denoising network is an improved U-shaped convolutional neural network (U-net). The improved U-net is: deleting the non-linear activation function after the convolutional operation of each convolutional layer in the network residual structure of the original U-net, and embedding a noise timestamp (TmeEmdeding) between two convolutional operations to encode the time step t into the feature matrix; and after each convolutional layer, adding a sampling gate activation operation (SimpleGate) and a convolutional block attention module (CBAM) attention layer.

[0063] Referring to Fig. 2(a), Fig. 2(a) is a schematic structural diagram corresponding to each convolutional layer in the network residual structure of the original U-net. The network residual structure of the original U-net includes multiple convolutional layers, and Fig. 2(a) is a schematic structural diagram corresponding to one of the convolutional layers. Each convolutional layer includes two convolutional operations and two non-linear activation functions (GELU); there is a pooling layer after each convolutional layer.

[0064] Referring to Fig. 2(b), Fig. 2(b) is a schematic structural diagram corresponding to each convolutional layer in the network residual structure of the improved U-net in the embodiment of the present application. The number of convolutional layers in the network residual structure of the improved U-net in the embodiment of the present application remains unchanged, and there is also a pooling layer after each convolutional layer, that is, the necessary convolutional and normalization layers are retained. What is changed is to delete the non-linear activation function after each convolutional operation, add SimpleGate and CBAM attention layers after two convolutional operations; and embed a TmeEmdeding between two convolutional operations to encode the time step t into the feature matrix.

[0065] In the embodiment of the present application, SimpleGate is used to replace the complex non-linear activation function GELU, which can improve the data processing efficiency. SimpleGate can achieve the effect of non-linear mapping through a single multiplication operation. The calculation formula of SimpleGate is as follows:

[0066] SimpleGate(X, Y) = X ⊙ Y;

[0067] SimpleGate directly divides the features into two parts along the channel dimension and multiplies them; X and Y represent dividing the feature map with channel C, height H, and width W into two parts along the channel dimension. of.

[0068] The CBAM attention mechanism can enhance the network's attention to key information and improve the quality of feature representation by combining channel attention and spatial attention. The CBAM attention layer helps the network to more accurately identify and recover the key details and structures in the image, increasing the network's ability to capture key information. Introducing the CBAM attention mechanism into the U-Net network of the diffusion model can further improve its denoising performance and image restoration quality;

[0069] In the embodiments of this application, by introducing SimpleGate and CBAM, more accurate image denoising reconstruction can be achieved without significantly increasing additional algorithm overhead.

[0070] Embed a TmeEmdeding between two layers of convolutional operations to encode the time step t into the feature matrix; that is, use the current diffusion time step t as the input so that the overall system network can perceive the noise at different time steps t and better match the diffusion model.

[0071] See Figure 3 , Figure 3 is a schematic diagram of the processing flow of the preset diffusion model for three-dimensional images in the embodiments of this application. The specific steps are as follows:

[0072] Step 301, downsample and encode the obtained three-dimensional image into a latent space representation through an encoder.

[0073] The encoder here can be a VQVAE encoder. Input the obtained three-dimensional image X0 into the VQVAE encoder ε for downsampling and encoding, and the obtained latent space representation is Z 0 = ε(X 0 ).

[0074] Step 302, perform forward noise addition on the latent space representation to obtain latent variables.

[0075] During the forward noise addition process, Gaussian noise is gradually added to obtain the latent variable Z T , and the specific formula used is:

[0076]

[0077] where t belongs to {1, 2, 3.......T}, representing the time step, ∈ tdenotes a Gaussian noise vector subject to a standard normal distribution, is a parameter that controls the proportion of noise and belongs to [0, 1].

[0078] Step 303: Input the latent variable into a preset denoising network to obtain the latent space representation of the reconstructed three-dimensional image; wherein, the preset denoising network obtains the latent space representation of the reconstructed three-dimensional image by subtracting the variable corresponding to the predicted noise from the latent variable.

[0079] Here, the predicted noise is the prediction of the noise added during the noise addition process; if there are defects in the three-dimensional image, the defective parts in the three-dimensional image will also be regarded as added noise during prediction. Therefore, subtracting the variable corresponding to the predicted noise from the latent variable will obtain a variable corresponding to a normal three-dimensional image, which will then be used as a reference image for comparison with the original input three-dimensional image in the subsequent process.

[0080] Step 304: Use a decoder to obtain the reconstructed three-dimensional image corresponding to the latent space representation of the reconstructed three-dimensional image.

[0081] The preset defect detection and recognition network model includes: a preset three-dimensional U-shaped convolutional neural network (3D-Unet) segmentation network and a classification decision layer.

[0082] The preset 3D-Unet segmentation network predicts the voxel-level abnormal feature scores and the feature-level abnormal score map by using the inconsistencies and commonalities between the three-dimensional image of the solder joint to be detected and the reconstructed three-dimensional image; and uses the feature abnormal score map to determine the defect location and shape, combines with the three-dimensional image to be detected to determine the defect image, and outputs; if the number of voxel-level abnormal feature scores is less than the preset number threshold, it is determined that the detection result is normal;

[0083] The classification decision layer performs classification decision processing on the voxel-level abnormal feature scores output by the preset 3D-Unet segmentation network to determine the defect type.

[0084] See Figure 4 , Figure 4 which is the structural schematic diagram of the classification decision layer in the embodiment of the present application. Figure 4 The classification decision layer in

[0085] performs normalization processing on the voxel-level abnormal feature scores output by the preset 3D-Unet segmentation network through the Softmax layer;

[0086] uses the Argmax operation to select the category label of the maximum value from the results of the normalization processing as the defect type.

[0087] After the last output layer of the preset 3D-Unet segmentation network, a Softmax layer is added. The Softmax layer normalizes the output of the preset 3D-Unet segmentation network and converts it into a probability distribution, such that the sum of the probabilities of each category is 1.

[0088] The mathematical expression of the Softmax function is:

[0089]

[0090] where z i represents the original output value of the i-th node in the output layer, K is the total number of categories, and the sum of all p i is 1. The above formula shows that the Softmax function converts the original output value of the convolutional layer into a probability value by calculating the proportion of the exponential value of each category in the total sum.

[0091] After obtaining the output of the Softmax function, the Argmax operation is used to select the category label with the maximum value from the probability distribution as the defect type, that is, the category label with the highest probability is selected as the defect label.

[0092] The preset detection model in the embodiments of the present application is obtained through training. First, a preset diffusion model needs to be obtained by training an initial diffusion model, and then the preset defect detection and recognition network model and the preset detection model including the preset diffusion model and the initial defect detection and recognition network model are trained. When training the preset defect detection and recognition network model, it is the process of obtaining the preset 3D-Unet segmentation network.

[0093] See Figure 5 , Figure 5 which is a schematic diagram of the process of generating the preset diffusion model in the embodiments of the present application. The specific steps are as follows:

[0094] Step 501, obtain the first training sample; the first training sample is a three-dimensional image of the first reference solder joint; the first reference solder joint is a normal solder joint; the label of the first training sample is 0.

[0095] Step 502, generate an initial diffusion model; the initial diffusion model includes an initial denoising network.

[0096] The initial diffusion model includes an encoder, a noise addition network, an initial denoising network, and a decoder; among them, the encoder, the noise addition network, and the decoder can be implemented without training according to related technologies. The initial denoising network in the present application uses the structure of an improved U-net and needs to be trained. Therefore, the training of the initial diffusion model is the training of the initial denoising network, that is, the initial denoising network is trained to obtain the preset denoising network, and then the preset diffusion network is obtained.

[0097] Steps 501 and 502 are executed without a sequential order.

[0098] Step 503, when training the initial diffusion model using the first training sample, optimize the predicted noise in the initial denoising network using the noise prediction loss function, and continuously optimize the parameters of the initial denoising network.

[0099] Among them, the value of the noise prediction loss function is calculated based on the label of the first training sample, as well as the predicted noise and the noise used in the forward noise addition process.

[0100] During the training process, use the noise prediction loss function to optimize the noise prediction target in the initial denoising network, continuously train the network, and continuously optimize the parameters of the network. The finally obtained preset denoising network can predict noise more accurately, thereby improving the image reconstruction ability of the preset diffusion model and reducing the error of subsequent defect detection. The noise prediction loss function L d is expressed as follows:

[0101] L d =(1 - y)||∈ t - ∈ θ (z t , t)|| 2

[0102] Specifically, y represents the sample label of the input first training sample; ∈ t is the true noise at time step t, which is the noise added during the forward diffusion process, ∈ θ (z t , t) is the prediction of the noise at time step t, that is, the predicted noise, and ||·||^2 represents calculating the L2 norm.

[0103] In this embodiment, during training, normal samples are used. In subsequent use, whether it is a normal 3D image or an abnormal 3D image, accurate noise prediction can be performed. For abnormal samples, during prediction after adding noise, the defect features are also regarded as noise. In this way, by subtracting the predicted noise from the input 3D image, a normal image can be obtained regardless of whether it is a normal image, for subsequent comparison.

[0104] Step 504, when the value of the noise prediction loss function is less than the first preset threshold, obtain the preset denoising network and the preset diffusion model.

[0105] Thus, the training of the preset diffusion model is completed.

[0106] See Figure 6 , Figure 6 which is the schematic diagram of the training process of the preset detection model in the embodiments of this application. The specific steps are as follows:

[0107] Step 601: Obtain the second training sample; the second training sample is a three-dimensional image of a reference solder joint; the reference solder joint includes normal solder joints and abnormal solder joints.

[0108] The second training sample here can be a training sample different from the first training sample, or a training sample that completely or partially includes the first training sample.

[0109] Step 602: Generate an initial detection model; wherein, the initial detection model includes: a preset diffusion model and an initial defect detection and recognition network model.

[0110] Steps 601 and 602 can be executed in any order.

[0111] Step 603: Use the second training sample to train the initial prediction model; and after each training session, substitute the reference class label distribution and the predicted class label distribution into the cross-entropy loss function to calculate the value of the cross-entropy loss function.

[0112] When training the initial defect detection and recognition network, use the cross-entropy loss function to optimize the network. Substitute the true class label distribution and the predicted class distribution into the cross-entropy loss function to calculate the loss value, and use this loss value to continuously train the network until more accurate defect segmentation and recognition are achieved. The cross-entropy loss function L seg is expressed as follows:

[0113]

[0114] where N is the number of samples, C is the number of defect types. In solder joint defects, there are types such as voids, bridging, and impurities. y i,c is the true label (0 or 1) indicating whether the i-th sample belongs to the c-th type of defect. is the probability that the i-th sample is predicted to be the c-th type of defect.

[0115] Step 604: When the value of the calculated cross-entropy loss function is less than the second preset threshold, obtain the preset defect detection and recognition network model and the preset detection model.

[0116] During Figure 6 the training process, if there are few abnormal samples, three-dimensional images can be generated by expanding two-dimensional images and used as abnormal training samples for training the preset detection model.

[0117] Specifically, the expansion of two-dimensional images to three-dimensional images can be achieved through the pix2pix network. The pix2pix network learns the mapping relationship from the defect mask to the three-dimensional defect data and can reconstruct the three-dimensional defect data from the two-dimensional image. This can solve the problem of few three-dimensional abnormal samples.

[0118] So far, the training of the preset detection model is completed. The preset detection model can be used to detect the solder joints of the PCB.

[0119] See Figure 7 , Figure 7 , which is a schematic diagram of the solder joint detection process of the PCB in the embodiment of the present application. The specific steps are as follows:

[0120] Step 701, obtain a three-dimensional image of the solder joint to be detected on the target PCB.

[0121] Step 702, obtain the detection result corresponding to the solder joint to be detected based on the preset detection model; the detection result is normal or abnormal; when the detection result is abnormal, the detection result further includes a defect image and a defect type.

[0122] Among them, the preset detection model performs forward noise addition on the obtained three-dimensional image through a preset diffusion model, and then subtracts the predicted noise from the noise-added three-dimensional image to obtain a reconstructed three-dimensional image; and inputs the reconstructed three-dimensional image and the obtained three-dimensional image into a preset defect detection and recognition network model to output the detection result corresponding to the solder joint to be detected; the predicted noise is predicted based on the noise added during the forward noise addition process and the abnormal voxels in the obtained three-dimensional image disturbed by the added noise.

[0123] All the above optional technical solutions can be combined arbitrarily to form alternative embodiments of the present disclosure, which will not be elaborated here one by one.

[0124] Based on the same inventive concept, an apparatus for detecting solder joints of a PCB is further provided in the embodiment of the present application. See Figure 8 , Figure 8 , which is a schematic structural diagram of the apparatus for detecting solder joints of the PCB in the embodiment of the present application. The apparatus includes:

[0125] An acquisition unit 801, configured to obtain a three-dimensional image of the solder joint to be detected on the target PCB;

[0126] A detection unit 802, configured to obtain the detection result corresponding to the three-dimensional image of the solder joint to be detected based on the preset detection model; the detection result includes normal or abnormal; when the detection result is abnormal, the detection result further includes a defect image and a defect type; among them, the preset detection model performs forward noise addition on the obtained three-dimensional image through a preset diffusion model, and then subtracts the predicted noise from the noise-added three-dimensional image to obtain a reconstructed three-dimensional image; and inputs the reconstructed three-dimensional image and the obtained three-dimensional image into a preset defect detection and recognition network model to output the detection result corresponding to the solder joint to be detected; the predicted noise is predicted based on the noise added during the forward noise addition process and the abnormal voxels in the obtained three-dimensional image disturbed by the added noise.

[0127] In another embodiment,

[0128] After the preset detection model performs forward noise addition on the obtained three-dimensional image through the preset diffusion model, the reconstructed three-dimensional image is obtained by subtracting the predicted noise from the noise-added three-dimensional image, including:

[0129] Downsample and encode the obtained three-dimensional image into a latent space representation through an encoder;

[0130] Perform forward noise addition on the latent space representation to obtain a latent variable;

[0131] Input the latent variable into the preset denoising network to obtain the latent space representation of the reconstructed three-dimensional image; wherein, the preset denoising network obtains the latent space representation of the reconstructed three-dimensional image by subtracting the variable corresponding to the predicted noise from the latent variable;

[0132] Use a decoder to obtain the reconstructed three-dimensional image corresponding to the latent space representation of the reconstructed three-dimensional image.

[0133] In another embodiment, the preset denoising network is an improved U-net;

[0134] The improved U-net is: delete the non-linear activation function after the convolution operation of each convolution layer in the network residual structure of the original U-net, and embed a TmeEmdeding between the two convolution operations to encode the time step t into the feature matrix; and after each convolution layer, add a SimpleGate and a CBAM attention layer.

[0135] In another embodiment, the device further includes:

[0136] A training unit 803, configured to generate a preset diffusion model, specifically:

[0137] Obtain a first training sample; the first training sample is a three-dimensional image of a first reference solder joint; the first reference solder joint is a normal solder joint; the label of the first training sample is 0;

[0138] Generate an initial diffusion model; the initial diffusion model includes an initial denoising network;

[0139] When using the first training sample to train the initial diffusion model, use a noise prediction loss function to optimize the predicted noise in the initial denoising network, and continuously optimize the parameters of the initial denoising network;

[0140] When the value of the noise prediction loss function is less than a first preset threshold, obtain a preset denoising network and a preset diffusion model;

[0141] Wherein, the value of the noise prediction loss function is calculated according to the label of the first training sample, and the predicted noise and the noise used in the forward noise addition process.

[0142] In another embodiment, the prediction defect detection and recognition network model includes: a preset 3D-Unet segmentation network and a classification decision layer;

[0143] The 3D-Unet segmentation network predicts voxel-level abnormal feature scores and feature-level abnormal score maps using the inconsistencies and commonalities between the three-dimensional image to be detected and the reconstructed three-dimensional image; and determines the defect position and shape using the feature abnormal score map, combines the three-dimensional image to be detected to determine the defect image, and outputs it; if the number of voxel-level abnormal feature scores is less than a preset number threshold, the detection result is determined to be normal;

[0144] The classification decision layer performs classification decision processing on the voxel-level abnormal feature scores to determine the defect type.

[0145] In another embodiment, the classification decision layer includes a Softmax layer and an Argmax operation;

[0146] The Softmax layer normalizes the voxel-level abnormal feature scores output by the 3D-Unet segmentation network;

[0147] The Argmax operation is used to select the category label with the maximum value from the normalized result as the defect type.

[0148] In another embodiment,

[0149] The training unit 803 is further configured to generate a preset detection model, specifically:

[0150] Obtain a second training sample; the second training sample is a three-dimensional image of a reference solder joint; the reference solder joint includes normal solder joints and abnormal solder joints;

[0151] Generate an initial detection model; wherein, the initial detection model includes: a preset diffusion model and an initial defect detection and recognition network model;

[0152] Use the second training sample to train the initial prediction model; and after each training ends, substitute the reference category label distribution and the predicted category label distribution into the cross-entropy loss function to calculate the value of the cross-entropy loss function;

[0153] When the value of the calculated cross-entropy loss function is less than a second preset threshold, obtain a preset defect detection and recognition network model and a preset detection model.

[0154] The units in the above embodiments can be integrated into one body or separately deployed; they can be combined into one unit or further split into multiple sub-units.

[0155] In another embodiment, an electronic device is further provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it is a solder joint detection method for a PCB.

[0156] In another embodiment, a computer-readable storage medium is further provided, on which computer instructions are stored. When the instructions are executed by a processor, a solder joint detection method for a PCB is implemented.

[0157] Figure 9 It is a schematic diagram of the physical structure of the electronic device provided by the embodiment of the present invention. As Figure 9 shown, the electronic device may include: a processor (Processor) 910, a communication interface (Communications Interface) 920, a memory (Memory) 930, and a communication bus 940. Among them, the processor 910, the communication interface 920, and the memory 930 complete mutual communication through the communication bus 940. The processor 910 can call the logical instructions in the memory 930 to execute the following method:

[0158] Obtain a three-dimensional image of the solder joints to be detected on the target PCB;

[0159] Obtain a detection result corresponding to the three-dimensional image of the solder joints to be detected based on a preset detection model; the detection result includes normal or abnormal; when the detection result is abnormal, the detection result further includes a defect image and a defect type;

[0160] Among them, the preset detection model performs forward noise addition on the obtained three-dimensional image through a preset diffusion model, and uses the three-dimensional image after noise addition to subtract the predicted noise to obtain a reconstructed three-dimensional image; and inputs the reconstructed three-dimensional image and the obtained three-dimensional image into a preset defect detection and recognition network model to output a detection result corresponding to the solder joints to be detected; the predicted noise is predicted based on the noise added during the forward noise addition process and the abnormal voxels in the obtained three-dimensional image disturbed by the added noise.

[0161] In addition, when the logical instructions in the above-mentioned memory 930 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0162] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative effort.

[0163] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments.

[0164] The flowcharts and block diagrams in the accompanying drawings of the present application illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments disclosed in the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the different drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as combinations of blocks in the block diagram or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0165] Those skilled in the art can understand that the features recited in the various embodiments and / or claims of the present application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly recited in the present application. In particular, without departing from the spirit and teachings of the present application, the features recited in the various embodiments and / or claims of the present application can be combined and / or combined in various ways, and all such combinations and / or combinations fall within the scope disclosed in the present application.

[0166] Specific embodiments are used herein to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention, and is not used to limit the present application. For those skilled in the art, changes can be made in the specific implementation manners and application scopes according to the ideas, spirits, and principles of the present invention. Any modifications, equivalent replacements, improvements, etc. made by them should be included within the scope protected by the present application.

Claims

1. A method for detecting solder joints of a PCB, characterized in that: The method comprises: Acquire a three-dimensional image of the solder joint to be inspected on the target PCB; Acquire a detection result corresponding to the three-dimensional image of the welding point to be detected based on a preset detection model; the detection result includes normal or abnormal; when the detection result is abnormal, the detection result also includes a defect image and a defect type; Among them, after the preset detection model performs forward noise addition on the acquired three-dimensional image through a preset diffusion model, the predicted noise is subtracted from the noisy three-dimensional image to obtain a reconstructed three-dimensional image; and the reconstructed three-dimensional image and the acquired three-dimensional image are input into a preset defect detection and recognition network model, and the detection result corresponding to the welding point to be detected is output; the predicted noise is predicted based on the noise added in the forward noise addition process and the abnormal voxels in the acquired three-dimensional image after being interfered by the added noise.

2. The method according to claim 1, characterized in that After the preset detection model performs forward noise addition on the acquired three-dimensional image through a preset diffusion model, a reconstructed three-dimensional image is obtained by subtracting the predicted noise from the noisy three-dimensional image, including: Downsampling and encoding the acquired three-dimensional image into a latent space representation through an encoder; Performing forward noise addition on the latent space representation to obtain latent variables; Inputting the latent variables into a preset denoising network to obtain a latent space representation of the reconstructed three-dimensional image; wherein the preset denoising network uses the latent variables to subtract the variables corresponding to the predicted noise to obtain the latent space representation of the reconstructed three-dimensional image; A decoder is used to obtain a reconstructed three-dimensional image corresponding to the latent space representation of the reconstructed three-dimensional image.

3. The method according to claim 2, characterized in that The preset denoising network is an improved U-net; The improved U-net is as follows: the nonlinear activation function after the convolution operation of each convolution layer in the network residual structure of the original U-net is deleted, and a TmeEmdeding is embedded between two convolution operations to encode the time step t into the feature matrix; And after each convolutional layer, add SimpleGate and CBAM attention layers.

4. The method according to claim 1, characterized in that: The method further comprises: Generating the preset diffusion model includes: Acquire a first training sample; the first training sample is a three-dimensional image of a first reference solder joint; the first reference solder joint is a normal solder joint; the label of the first training sample is 0; Generate an initial diffusion model; the initial diffusion model includes an initial denoising network; When the initial diffusion model is trained using the first training sample, the predicted noise in the initial denoising network is optimized using a noise prediction loss function, and the parameters of the initial denoising network are continuously optimized; When the value of the noise prediction loss function is less than a first preset threshold, obtaining a preset denoising network and a preset diffusion model; The value of the noise prediction loss function is calculated based on the label of the first training sample, as well as the prediction noise and the noise used in the forward noise addition process.

5. The method according to claim 1, characterized in that The predicted defect detection and identification network model includes: a preset 3D-Unet segmentation network and a classification decision layer; The preset 3D-Unet segmentation network predicts voxel-level abnormal feature scores and feature-level abnormal score maps using the inconsistencies and commonalities between the three-dimensional image to be detected and the reconstructed three-dimensional image; and uses the feature abnormal score map to determine the defect location and shape, and determines the defect image in combination with the three-dimensional image to be detected, and outputs it; if the number of voxel-level abnormal feature scores is less than a preset number threshold, the detection result is determined to be normal; The voxel-level abnormal feature score is classified and decided by the classification decision layer to determine the defect type.

6. The method according to claim 5, characterized in that The classification decision layer includes a Softmax layer and an Argmax operation; Normalizing the voxel-level abnormal feature scores output by the preset 3D-Unet segmentation network through a Softmax layer; The Argmax operation is used to select the category label with the maximum value from the normalized results as the defect type.

7. The method according to claim 1, characterized in that The method further comprises: Generating the preset detection model includes: Acquire a second training sample; the second training sample is a three-dimensional image of a reference solder joint; the reference solder joint includes a normal solder joint and an abnormal solder joint; Generate an initial detection model; wherein the initial detection model includes: a preset diffusion model and an initial defect detection and identification network model; The initial prediction model is trained using the second training sample; and after each training, the reference category label distribution and the predicted category label distribution are substituted into the cross entropy loss function to calculate the value of the cross quotient loss function; When the calculated value of the cross quotient loss function is less than a second preset threshold, a preset defect detection and recognition network model and the preset detection model are obtained.

8. A PCB solder joint detection device, characterized in that: The device comprises: An acquisition unit, used for acquiring a three-dimensional image of a soldering point to be inspected on a target PCB; A detection unit is used to obtain a detection result corresponding to the three-dimensional image of the welding point to be detected based on a preset detection model; the detection result includes normal or abnormal; when the detection result is abnormal, the detection result also includes a defect image and a defect type; wherein, after the preset detection model performs forward noise addition on the acquired three-dimensional image through a preset diffusion model, a reconstructed three-dimensional image is obtained by subtracting the predicted noise from the noisy three-dimensional image; and the reconstructed three-dimensional image and the acquired three-dimensional image are input into a preset defect detection and recognition network model, and the detection result corresponding to the welding point to be detected is output; the predicted noise is predicted based on the noise added in the forward noise addition process and the abnormal voxels in the acquired three-dimensional image after being interfered by the added noise.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method described in any one of claims 1 to 7 is implemented.

Images