PCB solder joint detection method, device, electronic device and storage medium

Abstract

The present application provides a method, device, electronic device, and storage medium for detecting solder joints on a PCB. A three-dimensional image of the solder joint to be detected on a target PCB is obtained; a detection result corresponding to the three-dimensional image of the solder joint to be detected is obtained based on a preset detection model; the detection result may be normal or abnormal; when the detection result is abnormal, the detection result also includes a defect image and defect type; the preset detection model performs forward noise addition on the acquired three-dimensional image using a preset diffusion model, and then subtracts the predicted noise from the noisy three-dimensional image to obtain a reconstructed three-dimensional image; the reconstructed three-dimensional image and the acquired three-dimensional image are input into a preset defect detection and recognition network model, and the detection result corresponding to the solder joint to be detected is output. This method can efficiently and accurately detect solder joints on PCBs and can locate and classify defects.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and more particularly to a method, device, electronic device, and storage medium for detecting solder joints on a PCB. Background Art

[0002] Manufacturing is a core pillar of a country's economic system. In recent years, my country's manufacturing industry has grown rapidly, with its global share continuing to rise. As an essential component of electronic devices, printed circuit boards (PCBs) play a crucial role in the electronics industry. However, PCB solder joints are prone to defects, impacting device performance and safety. Therefore, defect detection for these solder joints has become an essential and critical step in electronic device manufacturing.

[0003] Currently, in the field of industrial nondestructive testing (NDT) both domestically and internationally, traditional methods, particularly for detecting solder joint defects on printed circuit boards (PCBs), primarily rely on manual visual inspection or automated detection techniques based on two-dimensional images. However, these methods often suffer from limited detection accuracy, difficulty obtaining abnormal samples, slow detection speeds, inability to accurately identify defect types, and unclear defect types when faced with complex three-dimensional solder joint structures. These issues not only impact the automation level and inspection efficiency of production lines, but also hinder further advancements in solder joint quality inspection technology. Therefore, a new method is urgently needed to efficiently and accurately detect and classify defects in three-dimensional solder joint images. This approach can overcome the limitations of existing technologies and promote innovation and development in PCB solder joint quality inspection technology. Summary of the Invention

[0004] In view of this, the present application provides a PCB solder joint detection method, device, electronic device and storage medium, which can efficiently and accurately realize PCB solder joint detection and locate and classify defects.

[0005] To solve the above technical problems, the technical solution of this application is implemented as follows:

[0006] In one embodiment, a method for detecting solder joints of a PCB is provided, the method comprising:

[0007] Acquire a three-dimensional image of the solder joint to be inspected on the target PCB;

[0008] Obtaining a detection result corresponding to the three-dimensional image of the welding point to be inspected based on a preset detection model; the detection result includes normal or abnormal; when the detection result is abnormal, the detection result also includes a defect image and a defect type;

[0009] Among them, the preset detection model performs forward noise addition on the acquired three-dimensional image through a preset diffusion model, and then subtracts the predicted noise from the noisy three-dimensional image to obtain a reconstructed three-dimensional image; and inputs the reconstructed three-dimensional image and the acquired three-dimensional image into a preset defect detection and recognition network model, and outputs the detection result corresponding to the welding point to be inspected; the predicted noise is predicted based on the noise added during the forward noise addition process and the abnormal voxels in the acquired three-dimensional image after being interfered by the added noise.

[0010] The preset detection model performs forward noise addition on the acquired three-dimensional image using a preset diffusion model, and then subtracts the predicted noise from the noisy three-dimensional image to obtain a reconstructed three-dimensional image, including:

[0011] Downsampling and encoding the acquired three-dimensional image into a latent space representation through an encoder;

[0012] Performing forward noise addition on the latent space representation to obtain latent variables;

[0013] Inputting the latent variables into a preset denoising network to obtain a latent space representation of the reconstructed three-dimensional image; wherein the preset denoising network subtracts a variable corresponding to the predicted noise from the latent variables to obtain the latent space representation of the reconstructed three-dimensional image;

[0014] A decoder is used to obtain a reconstructed three-dimensional image corresponding to the latent space representation of the reconstructed three-dimensional image.

[0015] The feature of the method is that the preset denoising network is an improved U-net;

[0016] The improved U-net is as follows: the nonlinear activation function after the convolution operation of each convolutional layer in the network residual structure of the original U-net is deleted, a TmeEmdeding is embedded between the two convolution operations to encode the time step t into the feature matrix; and after each convolution layer, the SimpleGate and CBAM attention layers are added.

[0017] Wherein, the method further comprises:

[0018] Generating the preset diffusion model includes:

[0019] Obtain a first training sample; the first training sample is a three-dimensional image of a first reference solder joint; the first reference solder joint is a normal solder joint; and the label of the first training sample is 0;

[0020] generating an initial diffusion model; the initial diffusion model including an initial denoising network;

[0021] When training the initial diffusion model using the first training sample, optimizing the predicted noise in the initial denoising network using a noise prediction loss function, and continuously optimizing the parameters of the initial denoising network;

[0022] When the value of the noise prediction loss function is less than a first preset threshold, obtaining a preset denoising network and a preset diffusion model;

[0023] The value of the noise prediction loss function is calculated based on the label of the first training sample, the prediction noise, and the noise used in the forward noisy process.

[0024] The predicted defect detection and identification network model includes: a preset 3D-Unet segmentation network and a classification decision layer;

[0025] The preset 3D-Unet segmentation network predicts voxel-level abnormality feature scores and a feature-level abnormality score map using the inconsistencies and commonalities between the three-dimensional image to be detected and the reconstructed three-dimensional image; and uses the feature-level abnormality score map to determine the defect location and shape, and determines the defect image in combination with the three-dimensional image to be detected, and outputs it; if the number of voxel-level abnormality feature scores is less than a preset number threshold, the detection result is determined to be normal;

[0026] The voxel-level abnormal feature scores are classified and decided by the classification decision layer to determine the defect type.

[0027] Wherein, the classification decision layer includes a Softmax layer and an Argmax operation;

[0028] Normalizing the voxel-level abnormal feature scores output by the predicted 3D-Unet segmentation network through a Softmax layer;

[0029] The Argmax operation is used to select the category label with the maximum value from the normalized results as the defect type.

[0030] Wherein, the method further comprises:

[0031] Generating the preset detection model includes:

[0032] Acquire a second training sample; the second training sample is a three-dimensional image of a reference solder joint; the reference solder joint includes a normal solder joint and an abnormal solder joint;

[0033] Generate an initial detection model; wherein the initial detection model includes: a preset diffusion model and an initial defect detection and identification network model;

[0034] The initial prediction model is trained using the second training sample; and after each training, the reference category label distribution and the predicted category label distribution are substituted into the cross entropy loss function to calculate the value of the cross quotient loss function;

[0035] When the calculated value of the cross quotient loss function is less than a second preset threshold, a preset defect detection and recognition network model and the preset detection model are obtained.

[0036] In another embodiment, a PCB solder joint detection device is provided, the device comprising:

[0037] An acquisition unit, configured to acquire a three-dimensional image of a solder joint to be inspected on a target PCB;

[0038] A detection unit is used to obtain a detection result corresponding to the three-dimensional image of the welding point to be detected based on a preset detection model; the detection result includes normal or abnormal; when the detection result is abnormal, the detection result also includes a defect image and a defect type; wherein, after the preset detection model performs forward noise addition on the acquired three-dimensional image through a preset diffusion model, the noisy three-dimensional image is subtracted from the predicted noise to obtain a reconstructed three-dimensional image; and the reconstructed three-dimensional image and the acquired three-dimensional image are input into a preset defect detection and recognition network model, and the detection result corresponding to the welding point to be detected is output; the predicted noise is predicted based on the noise added during the forward noise addition process and the abnormal voxels in the acquired three-dimensional image after being interfered by the added noise.

[0039] In another embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a method for detecting solder joints of a PCB when executing the program.

[0040] In another embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, a method for detecting solder joints of a PCB is implemented.

[0041] As can be seen from the above technical solution, in the above embodiment, by acquiring a 3D image of the solder joint to be inspected on the target PCB board and inputting this 3D image into a preset inspection model, a corresponding inspection result is obtained. The inspection result is either normal or abnormal. If abnormal, the inspection result also includes a defect image and defect type. The preset inspection model inputs the reconstructed 3D image output by the preset diffusion model and the acquired 3D image into a preset defect detection and recognition network model, outputting the inspection result corresponding to the solder joint to be inspected. The reconstructed 3D image is a normal image obtained by the diffusion model by using the difference between the acquired 3D image and the predicted noise. This solution can efficiently and accurately perform PCB solder joint inspection and can locate and classify defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0043] Figure 1 This is a schematic diagram of the preset detection model structure in the embodiment of this application;

[0044] Figure 2(a) is a schematic diagram of the structure corresponding to each convolutional layer in the network residual structure of the original U-net;

[0045] FIG2( b ) is a schematic diagram of the structure corresponding to each convolutional layer in the network residual structure of the improved U-net in an embodiment of the present application;

[0046] Figure 3 Schematic diagram of the process of processing a three-dimensional image using a preset diffusion model in an embodiment of the present application;

[0047] Figure 4 This is a schematic diagram of the classification decision layer structure in the embodiment of this application;

[0048] Figure 5 A schematic diagram of a process for generating a preset diffusion model in an embodiment of the present application;

[0049] Figure 6 Schematic diagram of the training process of the preset detection model in the embodiment of the present application;

[0050] Figure 7 This is a schematic diagram of the solder joint detection process of the PCB in the embodiment of the present application;

[0051] Figure 8 This is a schematic structural diagram of a PCB solder joint detection device according to an embodiment of the present application;

[0052] Figure 9 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0054] The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe the order or precedence of the objects. It should be understood that the numbers used in this way are interchangeable where appropriate, so that the implementation of the invention described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products or apparatus.

[0055] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0056] Traditional methods rely primarily on manual visual inspection or automated inspection technologies based on two-dimensional images. However, these methods often suffer from limited detection accuracy, difficulty obtaining abnormal samples, slow detection speeds, inability to accurately identify defect types, and unclear identification of defect types when faced with complex three-dimensional solder joint structures. These issues not only affect the automation level and inspection efficiency of production lines, but also hinder the further advancement of solder joint quality inspection technology.

[0057] To address the above-mentioned issues, an embodiment of the present application provides a PCB solder joint inspection method. By acquiring a three-dimensional image of the solder joint to be inspected on a target PCB board and inputting this 3D image into a preset inspection model, a corresponding inspection result is obtained. The inspection result is either normal or abnormal. If abnormal, the inspection result also includes a defect image and defect type. The preset inspection model inputs the reconstructed 3D image output by a preset diffusion model and the acquired 3D image into a preset defect detection and recognition network model, outputting the inspection result corresponding to the solder joint to be inspected. The reconstructed 3D image is a normal image obtained by the diffusion model by using the difference between the acquired 3D image and the predicted noise. This solution can efficiently and accurately implement PCB solder joint inspection and can locate and classify defects.

[0058] Before performing PCB solder joint inspection, you need to obtain a preset inspection model through model training. Figure 1 , Figure 1 This is a schematic diagram of the preset detection model structure in the embodiment of this application. Figure 1 The preset detection models include a preset diffusion model and a preset defect detection and identification network model.

[0059] Inputting the acquired three-dimensional image of the solder joint to be inspected of the target PCB into a preset diffusion model and a preset defect detection and recognition network model respectively; obtaining a reconstructed three-dimensional image through processing with the preset diffusion model, and inputting the reconstructed three-dimensional image into the preset defect detection and recognition network model;

[0060] The pre-set defect detection and recognition network model processes the input 3D image of the weld to be inspected and the reconstructed 3D image, outputting a detection result. This detection result includes normal and abnormal results. If the detection result is abnormal, the detection result also includes the defect image and defect type.

[0061] The preset diffusion model includes: an encoder, a noise adding network, a preset denoising network and a decoder;

[0062] The preset denoising network is an improved U-net convolutional neural network (U-net). The improved U-net is as follows: the nonlinear activation function after the convolution operation of each convolution layer in the network residual structure of the original U-net is deleted, and a noise timestamp (TmeEmdeding) is embedded between the two layers of convolution operations to encode the time step t into the feature matrix; and after each convolution layer, a sampling gate activation operation (SimpleGate) and a convolution block attention module (CBAM) attention layer are added.

[0063] See Figure 2(a). This diagram shows the structure of each convolutional layer in the original U-net residual network architecture. The original U-net residual network architecture includes multiple convolutional layers, and Figure 2(a) shows the structure of one of these convolutional layers. Each convolutional layer includes two convolution operations and two nonlinear activation functions (GELUs); each convolutional layer is followed by a pooling layer.

[0064] See Figure 2(b), which is a schematic diagram of the structure corresponding to each convolutional layer in the network residual structure of the improved U-net in the embodiment of the present application. The number of convolutional layers in the network residual structure of the improved U-net in the embodiment of the present application remains unchanged, and there is also a pooling layer after each convolutional layer, that is, the necessary convolution and normalization layers are retained. What is changed is that the nonlinear activation function after each convolution operation is deleted, and the SimpleGate and CBAM attention layers are added after the two convolution operations; and a TmeEmdeding is embedded between the two layers of convolution operations to encode the time step t into the feature matrix.

[0065] In the embodiment of the present application, SimpleGate is used to replace the complex nonlinear activation function GELU, which can improve data processing efficiency. SimpleGate can achieve the effect of nonlinear mapping through a single multiplication operation. The calculation formula of SimpleGate is as follows:

[0066] SimpleGate(X,Y)=X⊙Y;

[0067] SimpleGate directly divides the feature into two parts along the channel dimension and multiplies them; X and Y represent the division of the feature map with channel C, height H and width W along the channel dimension into two parts.

[0068] By combining channel attention and spatial attention, the CBAM attention mechanism can enhance the network's focus on key information and improve the quality of feature representation. The CBAM attention layer helps the network more accurately identify and recover key details and structures in the image, increasing the network's ability to capture key information. Introducing the CBAM attention mechanism into the U-Net network in the diffusion model can further improve its denoising performance and image restoration quality.

[0069] In the embodiments of the present application, SimpleGate and CBAM are introduced to achieve more accurate image denoising and reconstruction without significantly increasing additional algorithm overhead.

[0070] A TmeEmdeding is embedded between the two layers of convolution operations to encode the time step t into the feature matrix; that is, the current diffusion time step t is taken as input so that the overall system network can perceive the noise at different time steps t and better match the diffusion model.

[0071] See also Figure 3 , Figure 3 This is a schematic diagram of the process flow of a three-dimensional image using a preset diffusion model in an embodiment of the present application. The specific steps are:

[0072] Step 301 : Downsample and encode the acquired 3D image into a latent space representation through an encoder.

[0073] The encoder here can be a VQVAE encoder, which inputs the acquired three-dimensional image X0 into the VQVAE encoder ε for downsampling encoding, and the obtained latent space is expressed as Z0=ε(X0).

[0074] Step 302: Perform forward noise addition on the latent space representation to obtain latent variables.

[0075] In the process of forward noise addition, Gaussian noise is gradually added to obtain the latent variable Z T , the specific formula used is:

[0076]

[0077] Where t belongs to {1, 2, 3.......T}, representing the time step, ∈ t represents a Gaussian noise vector that obeys the standard normal distribution, It is a parameter that controls the size of the noise ratio and belongs to [0, 1].

[0078] Step 303: Input the latent variables into a preset denoising network to obtain a latent space representation of the reconstructed three-dimensional image; wherein the preset denoising network subtracts the variable corresponding to the predicted noise from the latent variable to obtain the latent space representation of the reconstructed three-dimensional image.

[0079] The predicted noise here is a prediction of the noise added during the noise addition process. If there are defects in the 3D image, the defects will also be considered as added noise during the prediction. Therefore, subtracting the variable corresponding to the predicted noise from the latent variable will produce a variable corresponding to a normal 3D image, which is then used as a reference image for comparison with the original input 3D image.

[0080] Step 304 : Use the decoder to obtain a reconstructed 3D image corresponding to the latent space representation of the reconstructed 3D image.

[0081] The preset defect detection and recognition network model includes: a preset three-dimensional U-shaped convolutional neural network (3D-Unet) segmentation network and a classification decision layer.

[0082] The preset 3D-Unet segmentation network uses the inconsistencies and commonalities between the 3D image of the weld to be inspected and the reconstructed 3D image to predict voxel-level abnormal feature scores and a feature-level abnormal score map. The feature-level abnormal score map is then used to determine the defect location and shape. The defect image is then determined and output in combination with the 3D image to be inspected. If the number of voxel-level abnormal feature scores is less than a preset threshold, the inspection result is determined to be normal.

[0083] The classification decision layer performs classification decision processing on the voxel-level abnormal feature scores output by the preset 3D-Unet segmentation network to determine the defect type.

[0084] See also Figure 4 , Figure 4 This is a schematic diagram of the classification decision layer structure in an embodiment of the present application. Figure 4 The classification decision layer in

[15] includes: the normalized exponential function (Softmax) layer and the maximum element index (Argmax) operation;

[0085] The voxel-level abnormal feature scores output by the preset 3D-Unet segmentation network are normalized through the Softmax layer;

[0086] The Argmax operation is used to select the category label with the maximum value from the normalized results as the defect type.

[0087] After the last output layer of the preset 3D-Unet segmentation network, a Softmax layer is added. The Softmax layer normalizes the output of the preset 3D-Unet segmentation network and converts it into a probability distribution so that the sum of the probabilities of each category is 1.

[0088] The mathematical expression of the Softmax function is:

[0089]

[0090] Among them, z i represents the original output value of the i-th node in the output layer, K is the total number of categories, and all p i The sum is 1. The above formula shows that the Softmax function converts the original output value of the convolution layer into a probability value by calculating the proportion of the exponential value of each category in the sum.

[0091] After obtaining the output of the Softmax function, the Argmax operation is used to select the category label with the maximum value from the probability distribution as the defect type, that is, the category label with the highest probability is selected as the defect label.

[0092] The preset detection model in the embodiment of the present application is obtained through training. It is necessary to first obtain the preset diffusion model by training the initial diffusion model, and then train the preset detection model including the preset diffusion model and the initial defect detection and recognition network model to obtain the preset defect detection and recognition network model, as well as the preset detection model. When training the preset defect detection and recognition network model, it is a process of obtaining the preset 3D-Unet segmentation network.

[0093] See also Figure 5 , Figure 5 This is a schematic diagram of the process of generating a preset diffusion model in the embodiment of this application. The specific steps are:

[0094] Step 501 , obtaining a first training sample; the first training sample is a three-dimensional image of a first reference solder joint; the first reference solder joint is a normal solder joint; and the label of the first training sample is 0.

[0095] Step 502: Generate an initial diffusion model; the initial diffusion model includes an initial denoising network.

[0096] The initial diffusion model includes an encoder, a denoising network, an initial denoising network and a decoder; wherein, the encoder, the denoising network and the decoder do not need to be trained and can be implemented according to relevant technologies. The initial denoising network in this application uses an improved U-net structure and needs to be trained. Therefore, the training of the initial diffusion model is the training of the initial denoising network, that is, the initial denoising network is trained to obtain a preset denoising network, and then the preset diffusion network is obtained.

[0097] Step 501 and step 502 are executed in no particular order.

[0098] Step 503 : When the initial diffusion model is trained using the first training sample, the noise prediction loss function is used to optimize the predicted noise in the initial denoising network, and the parameters of the initial denoising network are continuously optimized.

[0099] The value of the noise prediction loss function is calculated based on the label of the first training sample, the prediction noise, and the noise used in the forward noisy process.

[0100] During the training process, the noise prediction loss function is used to optimize the noise prediction target in the initial denoising network. The network is continuously trained and the network parameters are continuously optimized. The final preset denoising network can predict noise more accurately, thereby improving the image reconstruction ability of the preset diffusion model and reducing the error of later defect detection. Noise prediction loss function L d It is expressed as follows:

[0101] L d =(1-y)||∈ t -∈ θ (z t ,t)|| 2

[0102] Specifically, y represents the sample label of the first training sample input; ∈ t is the real noise of time step t, which is added to the noise during the forward diffusion process, ∈ θ (z t ,t) is the prediction of the noise at time step t, that is, the predicted noise, and ||·||^2 means finding the L2 norm.

[0103] In this embodiment, normal samples are used during training. In subsequent use, accurate noise prediction can be performed for both normal and abnormal three-dimensional images. For abnormal samples, after adding noise, the defect features are also regarded as noise during prediction. In this way, by subtracting the predicted noise from the input three-dimensional image, a normal image can be obtained for subsequent comparison, regardless of whether it is a normal image.

[0104] Step 504: When the value of the noise prediction loss function is less than a first preset threshold, a preset denoising network and a preset diffusion model are obtained.

[0105] This completes the training of the preset diffusion model.

[0106] See also Figure 6 , Figure 6 This is a schematic diagram of the training process of the preset detection model in the embodiment of this application. The specific steps are:

[0107] Step 601: Acquire a second training sample; the second training sample is a three-dimensional image of a reference solder joint; the reference solder joint includes a normal solder joint and an abnormal solder joint.

[0108] The second training sample here may be a training sample different from the first training sample, or may be a training sample that completely or partially includes the first training sample.

[0109] Step 602 : Generate an initial detection model; wherein the initial detection model includes: a preset diffusion model and an initial defect detection and identification network model.

[0110] Step 601 and step 602 are executed in no particular order.

[0111] Step 603: Use the second training sample to train the initial prediction model; and after each training, substitute the reference category label distribution and the predicted category label distribution into the cross entropy loss function to calculate the value of the cross quotient loss function.

[0112] When training the initial defect detection and recognition network, the cross entropy loss function is used to optimize the network. The real category label distribution and the predicted category distribution are substituted into the cross entropy loss function to calculate the loss value. This loss value is used to continuously train the network, ultimately achieving more accurate defect segmentation and recognition. The cross entropy loss function L seg It is expressed as follows:

[0113]

[0114] Among them, N is the number of samples, C is the number of defect types, and solder joint defects include voids, bridging, impurities, etc. i,c is the true label (0 or 1) of whether the i-th sample belongs to the c-th defect. is the probability that the i-th sample is predicted to be a class c defect.

[0115] Step 604 : When the calculated value of the cross quotient loss function is less than a second preset threshold, a preset defect detection and recognition network model and a preset detection model are obtained.

[0116] exist Figure 6 During the training process, if there are relatively few abnormal samples, a three-dimensional image can be generated by expanding the two-dimensional image and used as an abnormal training sample to train the preset detection model.

[0117] In practice, the expansion of 2D images to 3D images can be achieved through a pix2pix network. This network learns the mapping relationship between defect masks and 3D defect data, and can reconstruct 3D defect data from 2D images. This can solve the problem of a small number of 3D anomaly samples.

[0118] At this point, the training of the preset detection model is completed. This preset detection model can be used to detect solder joints on PCBs.

[0119] See also Figure 7 , Figure 7 This is a schematic diagram of the solder joint detection process of the PCB in the embodiment of this application. The specific steps are:

[0120] Step 701: Acquire a three-dimensional image of a soldering point to be inspected on a target PCB.

[0121] Step 702: Obtain a detection result corresponding to the solder joint to be detected based on a preset detection model; the detection result is normal or abnormal; when the detection result is abnormal, the detection result also includes a defect image and a defect type.

[0122] Among them, the preset detection model forward-noises the acquired three-dimensional image through a preset diffusion model, and then subtracts the predicted noise from the noisy three-dimensional image to obtain a reconstructed three-dimensional image; and inputs the reconstructed three-dimensional image and the acquired three-dimensional image into a preset defect detection and recognition network model, and outputs the detection results corresponding to the welding points to be detected; the predicted noise is predicted based on the noise added during the forward noisy process and the abnormal voxels in the acquired three-dimensional image after being interfered by the added noise.

[0123] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0124] Based on the same inventive concept, the present application also provides a PCB solder joint detection device. Figure 8 , Figure 8 This is a schematic diagram of the structure of a PCB solder joint detection device in an embodiment of the present application. The device includes:

[0125] An acquisition unit 801 is configured to acquire a three-dimensional image of a solder joint to be inspected on a target PCB;

[0126] The detection unit 802 is used to obtain a detection result corresponding to the three-dimensional image of the welding point to be detected based on a preset detection model; the detection result includes normal or abnormal; when the detection result is abnormal, the detection result also includes a defect image and a defect type; wherein, the preset detection model performs forward noise addition on the acquired three-dimensional image through a preset diffusion model, and then subtracts the predicted noise from the noisy three-dimensional image to obtain a reconstructed three-dimensional image; and inputs the reconstructed three-dimensional image and the acquired three-dimensional image into a preset defect detection and recognition network model, and outputs a detection result corresponding to the welding point to be detected; the predicted noise is predicted based on the noise added during the forward noise addition process and the abnormal voxels in the acquired three-dimensional image after being interfered with by the added noise.

[0127] In another embodiment,

[0128] The preset detection model performs forward noise addition on the acquired three-dimensional image using the preset diffusion model, and then subtracts the predicted noise from the noisy three-dimensional image to obtain a reconstructed three-dimensional image, including:

[0129] The acquired 3D image is downsampled and encoded into a latent space representation through an encoder;

[0130] Forward noise is added to the latent space representation to obtain the latent variables;

[0131] Inputting the latent variables into a preset denoising network to obtain a latent space representation of the reconstructed three-dimensional image; wherein the preset denoising network subtracts the variable corresponding to the predicted noise from the latent variables to obtain the latent space representation of the reconstructed three-dimensional image;

[0132] The decoder is used to obtain the reconstructed 3D image corresponding to the latent space representation of the reconstructed 3D image.

[0133] In another embodiment, the preset denoising network is an improved U-net;

[0134] The improved U-net is as follows: the nonlinear activation function after the convolution operation of each convolutional layer in the network residual structure of the original U-net is deleted, a TmeEmdeding is embedded between the two convolution operations to encode the time step t into the feature matrix; and after each convolution layer, the SimpleGate and CBAM attention layers are added.

[0135] In another embodiment, the apparatus further comprises:

[0136] The training unit 803 is used to generate a preset diffusion model, specifically:

[0137] Obtain a first training sample; the first training sample is a three-dimensional image of a first reference solder joint; the first reference solder joint is a normal solder joint; the label of the first training sample is 0;

[0138] Generate an initial diffusion model; the initial diffusion model includes an initial denoising network;

[0139] When the initial diffusion model is trained using the first training sample, the noise prediction loss function is used to optimize the predicted noise in the initial denoising network, and the parameters of the initial denoising network are continuously optimized;

[0140] When the value of the noise prediction loss function is less than a first preset threshold, obtaining a preset denoising network and a preset diffusion model;

[0141] The value of the noise prediction loss function is calculated based on the label of the first training sample, the prediction noise, and the noise used in the forward noisy process.

[0142] In another embodiment, the predictive defect detection and identification network model includes: a preset 3D-Unet segmentation network and a classification decision layer;

[0143] The 3D-Unet segmentation network uses the inconsistencies and commonalities between the 3D image to be inspected and the reconstructed 3D image to predict voxel-level abnormality feature scores and feature-level abnormality score maps. The feature abnormality score maps are then used to determine the location and shape of the defect. The defect image is then determined and output in combination with the 3D image to be inspected. If the number of voxel-level abnormality feature scores is less than a preset threshold, the inspection result is determined to be normal.

[0144] The classification decision layer performs classification decision processing on the voxel-level abnormal feature scores to determine the defect type.

[0145] In another embodiment, the classification decision layer includes a Softmax layer and an Argmax operation;

[0146] The voxel-level abnormal feature scores output by the 3D-Unet segmentation network are normalized through the Softmax layer;

[0147] The Argmax operation is used to select the category label with the maximum value from the normalized results as the defect type.

[0148] In another embodiment,

[0149] The training unit 803 is further used to generate a preset detection model, specifically:

[0150] Acquire a second training sample; the second training sample is a three-dimensional image of a reference solder joint; the reference solder joint includes a normal solder joint and an abnormal solder joint;

[0151] Generate an initial detection model; wherein the initial detection model includes: a preset diffusion model and an initial defect detection and identification network model;

[0152] The initial prediction model is trained using the second training sample; and after each training session, the reference category label distribution and the predicted category label distribution are substituted into the cross entropy loss function to calculate the value of the cross quotient loss function;

[0153] When the calculated value of the cross quotient loss function is less than the second preset threshold, a preset defect detection and recognition network model and a preset detection model are obtained.

[0154] The units in the above embodiments may be integrated into one body or deployed separately; they may be combined into one unit or further divided into multiple sub-units.

[0155] In another embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and a method for detecting solder joints of a PCB when the processor executes the program.

[0156] In another embodiment, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, a method for detecting solder joints of a PCB is implemented.

[0157] Figure 9 Schematic diagram of the physical structure of the electronic device provided by the embodiment of the present invention. Figure 9 As shown, the electronic device may include: a processor (Processor) 910, a communication interface (CommunicationsInterface) 920, a memory (Memory) 930 and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other via the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the following method:

[0158] Acquire a three-dimensional image of the solder joint to be inspected on the target PCB;

[0159] Obtaining a test result corresponding to the three-dimensional image of the weld to be inspected based on a preset inspection model; the test result includes normal or abnormal; when the test result is abnormal, the test result also includes a defect image and defect type;

[0160] Among them, the preset detection model forward-noises the acquired three-dimensional image through a preset diffusion model, and then subtracts the predicted noise from the noisy three-dimensional image to obtain a reconstructed three-dimensional image; and inputs the reconstructed three-dimensional image and the acquired three-dimensional image into a preset defect detection and recognition network model, and outputs the detection results corresponding to the welding points to be detected; the predicted noise is predicted based on the noise added during the forward noisy process and the abnormal voxels in the acquired three-dimensional image after being interfered by the added noise.

[0161] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0162] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0163] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiment.

[0164] The flowcharts and block diagrams in the accompanying drawings of the present application show the possible implementation architecture, functions and operations of the systems, methods and computer program products according to the various embodiments disclosed in the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in the order of the standards in different figures. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0165] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims disclosed in this application may be combined and / or coupled in various ways, even if such combinations and / or couplings are not explicitly disclosed in this application. In particular, without departing from the spirit and teachings of this application, the features described in the various embodiments and / or claims of this application may be combined and / or coupled in various ways, and all such combinations and / or couplings fall within the scope disclosed in this application.

[0166] The principles and implementation methods of the present invention are described herein using specific embodiments. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas, and is not intended to limit this application. For those skilled in the art, changes can be made in the specific implementation methods and application scope based on the ideas, spirit and principles of the present invention. Any modifications, equivalent replacements, improvements, etc. made therein should be included within the scope of protection of this application.

Claims

1. A method for detecting solder joints of a PCB, characterized in that: The method comprises: Acquire a three-dimensional image of the solder joint to be inspected on the target PCB; Obtaining a detection result corresponding to the three-dimensional image of the welding point to be inspected based on a preset detection model; the detection result includes normal or abnormal; when the detection result is abnormal, the detection result also includes a defect image and a defect type; The preset detection model performs forward noise addition on the acquired three-dimensional image using a preset diffusion model, and then subtracts the predicted noise from the noisy three-dimensional image to obtain a reconstructed three-dimensional image; the reconstructed three-dimensional image and the acquired three-dimensional image are input into a preset defect detection and recognition network model, and a detection result corresponding to the weld to be inspected is output; the predicted noise is predicted based on the noise added during the forward noise addition process and abnormal voxels in the acquired three-dimensional image after being interfered with by the added noise; Wherein, the preset defect detection and identification network model includes: a preset 3D-Unet segmentation network and a classification decision layer; The preset 3D-Unet segmentation network predicts voxel-level abnormality feature scores and a feature-level abnormality score map using the inconsistencies and commonalities between the three-dimensional image to be detected and the reconstructed three-dimensional image; and uses the feature-level abnormality score map to determine the defect location and shape, and determines the defect image in combination with the three-dimensional image to be detected, and outputs it; if the number of voxel-level abnormality feature scores is less than a preset number threshold, the detection result is determined to be normal; The voxel-level abnormal feature scores are classified and decided by the classification decision layer to determine the defect type.

2. The method according to claim 1, characterized in that After the preset detection model performs forward noise addition on the acquired three-dimensional image using a preset diffusion model, a reconstructed three-dimensional image is obtained by subtracting the predicted noise from the noisy three-dimensional image, including: Downsampling and encoding the acquired three-dimensional image into a latent space representation through an encoder; Performing forward noise addition on the latent space representation to obtain latent variables; Inputting the latent variables into a preset denoising network to obtain a latent space representation of the reconstructed three-dimensional image; wherein the preset denoising network subtracts a variable corresponding to the predicted noise from the latent variables to obtain the latent space representation of the reconstructed three-dimensional image; A decoder is used to obtain a reconstructed three-dimensional image corresponding to the latent space representation of the reconstructed three-dimensional image.

3. The method according to claim 2, characterized in that The preset denoising network is an improved U-net; The improved U-net is as follows: the nonlinear activation function after the convolution operation of each convolution layer in the network residual structure of the original U-net is deleted, and a TmeEmdeding is embedded between the two convolution operations to encode the time step t into the feature matrix; And after each convolutional layer, add SimpleGate and CBAM attention layers.

4. The method according to claim 1, wherein The method further comprises: Generating the preset diffusion model includes: Obtain a first training sample; the first training sample is a three-dimensional image of a first reference solder joint; the first reference solder joint is a normal solder joint; and the label of the first training sample is 0; generating an initial diffusion model; the initial diffusion model including an initial denoising network; When training the initial diffusion model using the first training sample, optimizing the predicted noise in the initial denoising network using a noise prediction loss function, and continuously optimizing the parameters of the initial denoising network; When the value of the noise prediction loss function is less than a first preset threshold, obtaining a preset denoising network and a preset diffusion model; The value of the noise prediction loss function is calculated based on the label of the first training sample, the prediction noise, and the noise used in the forward noisy process.

5. The method according to claim 1, wherein The classification decision layer includes a Softmax layer and an Argmax operation; Normalizing the voxel-level abnormal feature scores output by the preset 3D-Unet segmentation network through a Softmax layer; The Argmax operation is used to select the category label with the maximum value from the normalized results as the defect type.

6. The method according to claim 1, characterized in that The method further comprises: Generating the preset detection model includes: Acquire a second training sample; the second training sample is a three-dimensional image of a reference solder joint; the reference solder joint includes a normal solder joint and an abnormal solder joint; Generate an initial detection model; wherein the initial detection model includes: a preset diffusion model and an initial defect detection and identification network model; The initial detection model is trained using the second training sample; and after each training session, the reference category label distribution and the predicted category label distribution are substituted into the cross entropy loss function to calculate a value of the cross quotient loss function; When the calculated value of the cross quotient loss function is less than a second preset threshold, a preset defect detection and recognition network model and the preset detection model are obtained.

7. A PCB solder joint detection device, characterized in that: The device comprises: An acquisition unit, configured to acquire a three-dimensional image of a solder joint to be inspected on a target PCB; A detection unit, configured to obtain a detection result corresponding to the three-dimensional image of the weld to be detected based on a preset detection model; the detection result includes normal or abnormal; when the detection result is abnormal, the detection result also includes a defect image and a defect type; wherein, after the preset detection model performs forward noise addition on the acquired three-dimensional image using a preset diffusion model, the preset detection model subtracts predicted noise from the noisy three-dimensional image to obtain a reconstructed three-dimensional image; and inputs the reconstructed three-dimensional image and the acquired three-dimensional image into a preset defect detection and recognition network model, outputting a detection result corresponding to the weld to be detected; the predicted noise is predicted based on the noise added during the forward noise addition process and the abnormal voxels in the acquired three-dimensional image after being interfered with by the added noise; Wherein, the preset defect detection and identification network model includes: a preset 3D-Unet segmentation network and a classification decision layer; The preset 3D-Unet segmentation network predicts voxel-level abnormality feature scores and a feature-level abnormality score map using the inconsistencies and commonalities between the three-dimensional image to be detected and the reconstructed three-dimensional image; and uses the feature-level abnormality score map to determine the defect location and shape, and determines the defect image in combination with the three-dimensional image to be detected, and outputs it; if the number of voxel-level abnormality feature scores is less than a preset number threshold, the detection result is determined to be normal; The voxel-level abnormal feature scores are classified and decided by the classification decision layer to determine the defect type.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images