Circuit board welding spot detection method and system based on machine vision

Abstract

The invention relates to the technical field of circuit board manufacturing quality control, and discloses a circuit board welding spot detection method and system based on machine vision, and the method comprises the steps: constructing a welding spot space relation graph based on a circuit board design drawing and an actual image; processing the welding spot space relation graph by using a graph convolutional network, and detecting group anomalies which do not conform to normal distribution; constructing a causal graph network of welding spot defects and manufacturing parameters according to a group abnormal result; aiming at a defect mode observed in the causal graph network, constructing an anti-fact analysis model and calculating an anti-fact probability, and identifying necessary and sufficient reasons for defect formation; optimizing the causal graph structure by applying a differentiable causal discovery algorithm; a complete closed loop from problem discovery to problem explanation to problem prevention is realized, and the manufacturing quality and reliability of the circuit board are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of printed circuit board manufacturing quality control. More specifically, it relates to a method and system for detecting solder joints on a printed circuit board based on machine vision. Background Art

[0002] The quality of solder joints on a printed circuit board is a key factor in the reliability and service life of electronic products. Traditional solder joint detection methods mainly evaluate each solder joint independently. However, these methods have three problems: First, they ignore the spatial relationships between solder joints and group anomaly patterns, making it difficult to detect systematic defects; Second, when group anomalies are detected, they lack the ability to automatically infer the root causes of the anomalies, increasing the time and cost of troubleshooting; Third, traditional causal analysis usually relies on expert experience or simple correlations and is difficult to reveal the true causal mechanisms of solder joint defects in complex manufacturing environments.

[0003] In recent years, the development of graph neural networks and causal inference techniques has provided new ideas for solving these problems. Graph neural networks can effectively model the spatial relationships between nodes and are suitable for processing data with topological structures; causal inference techniques can discover causal relationships between variables from observational data and help understand the underlying mechanisms of system behavior. However, there is still a lack of methods for effectively combining these two technologies and applying them to the field of printed circuit board solder joint detection. In particular, a comprehensive solution for group anomaly detection and automatic root cause analysis has not yet emerged.

[0004] Therefore, there is an urgent need for a method for detecting solder joints on a printed circuit board that can simultaneously consider the group relationships of solder joints and automatically analyze the root causes of defects to improve the quality control level of printed circuit board manufacturing. Summary of the Invention

[0005] The present invention provides a method and system for detecting solder joints on a printed circuit board based on machine vision, which solves the technical problems of ignoring the group relationships of solder joints and lacking the ability of automatic root cause analysis in related technologies.

[0006] The present invention provides a method for detecting solder joints on a printed circuit board based on machine vision, including: constructing a solder joint spatial relationship graph based on the printed circuit board design drawing and the actual image; Applying a graph convolutional network to process the solder joint spatial relationship graph, learning the distribution characteristics of normal solder joint groups, and detecting group anomalies that do not conform to the normal distribution by comparing the actual distribution with the normal distribution model; According to the group anomaly results, constructing a causal graph network of solder joint defects and manufacturing parameters, where the nodes include the observed group anomaly characteristics and potential manufacturing parameters, and the edges represent the potential causal relationships between them; Construct a counterfactual analysis model and calculate the counterfactual probability for the observed defect patterns in the causal graph network to identify the necessary and sufficient causes of defect formation; Optimize the aforementioned causal graph structure by applying the differentiable causal discovery algorithm based on the counterfactual analysis results, the collected intervention data, and the observed data.

[0007] Further, the steps for constructing the solder joint spatial relationship graph include: Obtain the circuit board design diagram and the actual image data, and extract the solder joint position coordinates, types, and expected sizes; Extract features for each solder joint and construct a feature vector; Construct a relationship matrix between solder joints to represent the relationship strength between solder joints; Construct a complete solder joint spatial relationship graph based on the node features and the relationship matrix.

[0008] Further, in the step of constructing the relationship matrix between solder joints, the calculation formula for the relationship strength is: ; where represents the relationship strength between solder joint and solder joint , represents the Euclidean distance between solder joint and solder joint , is the distance attenuation parameter, represents whether there is an electrical connection relationship between solder joint and solder joint , with a connection being 1 and no connection being 0; and are the weight coefficients of the spatial relationship and the electrical relationship respectively, represents the exponential function.

[0009] Further, the graph convolutional network includes a three-layer structure of an input layer, a hidden layer, and an output layer, where: The input layer receives the solder joint node feature matrix and the adjacency matrix; The hidden layer contains 64 neurons; The output layer generates a solder joint representation vector with a dimension of 32; A ReLU activation function is used for non-linear transformation between each layer, and a Dropout layer is added after the hidden layer.

[0010] Further, the steps for detecting population anomalies that do not conform to the normal distribution include: Construct a normal distribution model, expressed as: ; where Denote the vector The probability density of is the representation vector of a single solder joint is the mean of the representation vectors of the normal solder joint population is the covariance matrix of the representation vectors of the normal solder joint population Denote the covariance matrix The determinant of Denote the covariance matrix The inverse matrix of is the dimension of the representation vector Denote the transpose operator Denote the exponential function Denote of power; Calculate the Mahalanobis distance between the representation vector of the actual solder joint population and the normal distribution model. When the Mahalanobis distance is greater than the preset threshold, determine that the solder joint or solder joint population is abnormal.

[0011] Furthermore, the steps of constructing the causal graph network of solder joint defects and manufacturing parameters include: Define the set of causal graph network nodes, including the observed population anomaly feature nodes and potential manufacturing parameter nodes; Collect parameter anomaly data to form a parameter anomaly data set; Initialize the causal graph network structure based on domain knowledge; Use the collected parameter anomaly data to learn the edge weights of the causal graph network.

[0012] Furthermore, the steps of constructing the counterfactual analysis model include: Sample multiple noise samples from the posterior distribution; For each noise sample, calculate the result under the intervention condition; Statistically analyze the system responses under different intervention conditions and estimate the counterfactual probability.

[0013] Furthermore, the loss function of the differentiable causal discovery algorithm is defined as: ; Where Denote the total loss function Denote the data fitting error Denote the graph structure sparsity constraint Denote the intervention consistency constraint And Denote the weight coefficients of the sparsity constraint and the intervention consistency constraint respectively.

[0014] Furthermore, the differentiable causal discovery algorithm uses the Adam optimizer for gradient update, with the initial learning rate set to 0.001, and a learning rate decay strategy is adopted. After every 50 iteration cycles, the learning rate is reduced to 0.9 times the original value.

[0015] The present invention provides a circuit board solder joint detection system based on machine vision for implementing the above-mentioned circuit board solder joint detection method based on machine vision, including: A graph construction module for constructing a solder joint spatial relationship graph; An anomaly detection module for detecting solder joint population anomalies; A causal analysis module for constructing a causal graph network between solder joint defects and manufacturing parameters; A counterfactual inference module for identifying necessary and sufficient causes for defect formation; A causal optimization module for optimizing the causal graph structure.

[0016] The beneficial effects of the present invention are as follows: It realizes a complete closed-loop from "discovering problems" to "explaining problems" and then to "preventing problems", effectively improving the manufacturing quality and reliability of circuit boards, reducing material waste and rework costs, and providing technical support for the manufacture of high-quality electronic products; By constructing a solder joint spatial relationship graph and applying a graph convolutional network for processing, the system can simultaneously consider the spatial relationship and electrical connection relationship of solder joints, thereby realizing effective detection of systematic defects, improving the population anomaly detection rate compared with traditional methods, and enhancing the system's ability to identify systematic defects; Through the causal graph network and counterfactual inference method, the system can automatically analyze the potential manufacturing causes of population defects, shorten the troubleshooting time, greatly improve production efficiency, reduce the dependence on expert experience, and make the defect analysis process more scientific and objective; By applying the differentiable causal structure optimization algorithm, the system can continuously optimize the causal relationship model, with the root cause analysis accuracy rate improved compared with traditional methods, providing reliable decision-making support for manufacturing process optimization, and effectively reducing misjudgment and missed judgment situations. Description of the Drawings

[0017] Figure 1 is a flowchart of a circuit board solder joint detection method based on machine vision of the present invention; Figure 2 is a flowchart of the steps for constructing a solder joint spatial relationship graph of the present invention; Figure 3 is a flowchart of the steps for detecting population anomalies in a graph convolutional network of the present invention; Figure 4 is a flowchart of the steps for constructing a causal graph network of the present invention; Figure 5It is a flowchart of the counterfactual causal inference step of the present invention; Figure 6 It is a flowchart of the differentiable causal structure optimization step of the present invention. Detailed implementation manners

[0018] Now, the subject matter described herein will be discussed with reference to exemplary embodiments. It should be understood that discussing these embodiments is only to enable those skilled in the art to better understand and thus implement the subject matter described herein. Without departing from the scope of protection of the content of this specification, changes can be made to the functions and arrangements of the elements discussed. Each example can omit, substitute, or add various processes or components as needed. Additionally, the features described in some examples can also be combined in other examples.

[0019] In at least one embodiment of the present invention, a method for detecting solder joints on a circuit board based on machine vision is disclosed. As Figures 1 to 6 shown, it includes: Step 1: Based on the circuit board design drawing and the actual image, construct a solder joint spatial relationship graph; Construct a solder joint spatial relationship graph to represent the solder joints and their relationships on the circuit board as a graph structure. Specifically, it includes the following sub-steps: Step 1.1: Obtain the circuit board design drawing and actual image data; Based on the computer-aided design (CAD) file of the circuit board and the actual circuit board image captured by a high-resolution camera, extract information such as the solder joint position coordinates, types, and expected sizes.

[0020] Step 1.2: Extract solder joint node features; Extract features for each solder joint to construct a feature vector , which contains the following information: ; Where represents the feature vector of the th solder joint, and represent the coordinate positions of the solder joint on the circuit board, represents the solder joint type (such as spherical solder joint, columnar solder joint, etc.), represents the solder joint size, represents the solder joint gray value feature, represents the solder joint contour feature, represents the index number of the solder joint.

[0021] Step 1.3: Model the solder joint relationships; Construct a relationship matrix between solder joints: ; Among them is the total number of solder joints, represents the set of real numbers, represents the adjacency matrix of solder joint relationships; represents solder joint and solder joint The relationship strength between them is calculated by the formula: ; Among them represents the Euclidean distance between solder joint and solder joint ; is the distance attenuation parameter, represents whether there is an electrical connection relationship between solder joint and solder joint : 1 for connection and 0 for no connection; and are the weight coefficients of spatial relationship and electrical relationship respectively, represents the exponential function.

[0022] Step 1.4, construct the spatial relationship graph of solder joints; Based on the node features and relationship matrix, construct the spatial relationship graph of solder joints: ; Among them represents the spatial relationship graph of solder joints, represents the set of solder joint nodes, represents the set of edges between solder joints, represents the node feature matrix.

[0023] The adjacency matrix of the graph is the relationship matrix calculated in the previous step .

[0024] This step outputs the spatial relationship graph of solder joints and the relationship adjacency matrix, which contain the feature information such as the position, type, and size of solder joints, as well as the spatial and electrical connection relationships between solder joints. These data structures precisely express the topological structure and attribute characteristics of solder joints on the circuit board, providing a data basis for subsequent anomaly detection and enabling the system to analyze from the perspective of group relationships rather than just individual solder joints.

[0025] Step 2, apply the graph convolutional network to process the spatial relationship graph of solder joints, learn the distribution characteristics of the normal solder joint group, and detect group anomalies that do not conform to the normal distribution model by comparing the difference between the actual distribution and the normal distribution model; In this step, use the spatial relationship graph of solder joints constructed in Step 1 and the relationship adjacency matrix Take [[INPUT]] as the input, process this data through a graph convolutional network, learn the representation vector of the solder joints, and establish a normal distribution model to achieve anomaly detection. The solder joint topological structure and feature information captured in Step 1 are the key basis for the population anomaly analysis in this step. The specific steps are as follows: Step 2.1, construct a graph convolutional network model; Construct a multi-layer graph convolutional network model for learning the distribution characteristics of the solder joint population. Each layer of graph convolution operation is defined as: ; where represents the node feature matrix of the th layer, represents the node feature matrix of the th layer, is the degree matrix (diagonal matrix, where the rd diagonal element is the sum of the th row elements of ), is the weight matrix of the th layer, is the activation function (such as the ReLU function), and the superscript represents the th layer of the network; represents the adjacency matrix with self-connections added, expressed as: ; where represents the solder joint relationship adjacency matrix, is the identity matrix, represents the total number of nodes.

[0026] In this embodiment, the graph convolutional network model specifically includes three-layer structure: an input layer, a hidden layer, and an output layer. The input layer receives the solder joint node feature matrix and the adjacency matrix . The hidden layer contains 64 neurons, and the output layer generates a solder joint representation vector with a dimension of 32. A ReLU activation function is used for non-linear transformation between each layer, and a Dropout layer (dropout rate of 0.3) is added after the hidden layer to prevent overfitting. This model can effectively capture the spatial relationship characteristics between solder joints and provide a basis for population anomaly detection.

[0027] Step 2.2, solder joint population representation learning; Learn the representation vector of the solder joint population through the graph convolutional network, and the calculation formula is: ; where Denote the learned representation vector of the solder joint population, Denote is a matrix, where is the number of solder joints, is the dimension of the representation vector; Denote the result of processing the graph by the graph convolutional network, Denote the graph convolutional network function, Denote the adjacency matrix of solder joint relationships, is the node feature matrix, is the network parameter.

[0028] Step 2.3, construction of the normal distribution model; Based on the representation vector of the normal solder joint population, construct a normal distribution model: ; where Denote the probability density of the representation vector , is the representation vector of a single solder joint, is the mean of the representation vector of the normal solder joint population, is the covariance matrix of the representation vector of the normal solder joint population, Denote the determinant of the covariance matrix , Denote the inverse matrix of the covariance matrix , is the dimension of the representation vector, Denote the transpose operator, Denote the exponential function, Denote of power.

[0029] Step 2.4, population anomaly detection; Calculate the Mahalanobis distance between the representation vector of the actual solder joint population and the normal distribution model to identify the abnormal solder joint population: Abnormal solder joint population: ; where Denote the Mahalanobis distance of the representation vector , Denote the square root operation, is the representation vector of a single solder joint, is the mean of the representation vector of the normal solder joint population; When the Mahalanobis distance is greater than the preset threshold , determine that the solder joint or solder joint population is abnormal. At the same time, introduce the population anomaly score To evaluate the abnormality degree of the entire area: ; Among them represents the group abnormality score, represents the solder joint group sub - graph to be evaluated, represents the sub - graph the number of solder joints in it, represents for the sub - graph all the solder joint representation vectors in perform summation, represents the representation vector of the i - th solder joint, represents that the vector belongs to the sub - represents the covariance matrix the inverse matrix of, represents the transpose operator graph , represents a preset abnormality threshold, represents the index number of the solder joint, represents the Mahalanobis distance of; When , it is determined that this solder joint group is an abnormal group and the cause of the abnormality needs to be further analyzed; When , it is determined that this solder joint group is a normal group. This scoring mechanism can effectively identify the solder joint group abnormalities in the local area and provide a basis for subsequent causal analysis.

[0030] This step outputs the set of abnormal solder joints, the group abnormality score, and the representation vector.

[0031] These data quantitatively describe the degree and distribution characteristics of solder joint abnormalities. The anomaly detection results clearly identify the solder joint areas that need further analysis, providing a target for subsequent causal analysis. The learned representation vectors also provide a low - dimensional and effective representation for the extraction of abnormal features.

[0032] Step 3: According to the group abnormality results, construct a causal graph network between solder joint defects and manufacturing parameters. The nodes include the observed group abnormality features and potential manufacturing parameters, and the edges represent the potential causal relationships between them; This step directly uses the causal relationship model of the abnormal solder joint set and the group abnormality score detected in Step 2. The results of Step 2 determine the types and areas of solder joint defects that need attention, providing a basis for defining the nodes of the causal graph network. Specifically, it includes the following sub - steps: Step 3.1: Define the causal graph nodes; Define the set of nodes of the causal graph network , including the following two types of nodes: Observed group anomaly feature nodes: ; where represents the set of observed group anomaly feature nodes, , , respectively represent the 1st, 2nd, th solder joint anomaly feature, is the total number of anomaly features; Potential manufacturing parameter nodes: ; where represents the set of potential manufacturing parameter nodes, , , respectively represent the 1st, 2nd, th manufacturing parameter, is the total number of manufacturing parameters; Therefore, the complete node set is: ; where represents the complete node set of the causal graph network, represents the union operation of the sets.

[0033] Step 3.2, Collect parameter - anomaly data; Collect historical data from the manufacturing system to form a parameter - anomaly data set: ; where represents the parameter - anomaly data set, and respectively represent the manufacturing parameter and the observed anomaly feature of the th sample, is the total number of samples, represents the set of all sample pairs from to .

[0034] Step 3.3, Initialize the causal graph structure; Initialize the causal graph network based on domain knowledge: ; where represents the causal graph network, represents the node set, represents the set of directed edges between nodes; Initial edge weight matrix: ; Represents the causal strength from node to node . Represents the size of the node set . Represents the set of real numbers.

[0035] Step 3.4, Edge weight learning; Using the collected parameter anomaly data, learn the edge weights of the causal graph network. The maximum likelihood estimation method is adopted to optimize the objective function: ; where represents the loss function with respect to the weight matrix , represents the conditional probability of observing the anomaly feature under the given manufacturing parameters and the weight matrix , is the regularization coefficient, is 's norm (the sum of the absolute values of all elements), which is used to promote the sparsity of the weight matrix, represents the summation over all samples from to , represents the natural logarithm function.

[0036] This step outputs the causal graph network of solder joint defects and manufacturing parameters and its edge weight matrix, establishing a preliminary causal relationship model between abnormal solder joint features and manufacturing parameters. This causal graph network expresses the relationship mapping of "which manufacturing parameters may cause which types of solder joint defects", providing a structural basis for subsequent counterfactual analysis. At the same time, the parameter anomaly data set also provides training data for further model optimization.

[0037] Step 4, For the defect patterns observed in the causal graph network, construct a counterfactual analysis model and calculate the counterfactual probability to identify the necessary and sufficient causes of defect formation; This step is based on the causal graph network and its edge weight matrix constructed in Step 3, and further applies the counterfactual causal inference method to deeply analyze the defect formation mechanism. The preliminary causal relationship structure provided by the causal graph network enables the system to precisely construct and analyze counterfactual problems for specific defect patterns, thereby revealing the necessary and sufficient causal relationships between manufacturing parameters and solder joint defects. Specifically, it includes the following sub-steps: Step 4.1, Construct counterfactual problems; For the observed solder joint defect and the current manufacturing parameters , construct a counterfactual analysis model: "If the value of parameter is changed to , will the solder joint defect change?"; It is formally expressed as calculating the counterfactual probability: ; where represents the solder joint defect variable, represents the observed value of the defect, represents the manufacturing parameter variable, represents the current parameter value, represents the hypothesized parameter intervention value, represents the defect variable under the intervention , represents the conditional probability distribution of and being observed, and if the intervention is performed, then .

[0038] Step 4.2, construction of the structural equation model; Construct a structural equation model (SEM) to represent the quantitative relationship between variables in the causal graph network: ; where represents the -th variable, represents the set of parent nodes of (the set of variables that directly affect ), represents the external noise (representing unobserved influencing factors), represents the mapping function from parent nodes to child nodes (describing how parent nodes affect child nodes), represents that the variable index ranges from 1 to , represents the total number of variables.

[0039] Step 4.3, calculation of counterfactual intervention; Based on the structural equation model, calculate the result of counterfactual intervention: ; where represents the set of all external noise variables, represents the posterior distribution of the noise variable and being observed, , represents the conditional probability distribution of given the noiseand the observed values Under the condition, the variable after intervention The conditional probability of represents integrating over all possible noise values to perform integration.

[0040] In actual implementation, due to the complexity of the integration calculation, the Monte Carlo sampling method is used to approximately calculate the counterfactual probability: Sample from the posterior distribution to obtain noise samples: ; where , , respectively represent the 1st, 2nd, and the th noise samples, is the total number of noise samples; For each noise sample, calculate the result under the intervention condition: ; where represents the value of under the condition of the intervention and the th noise sample ; represents the set obtained by excluding from the set of the parent nodes of represents the set difference operation, represents the structural equation function of represents the noise related to in the th noise sample, Estimate the counterfactual probability as: ; where represents summing over all noise samples from to represents the indicator function, which is 1 when and 0 otherwise, represents approximately equal to.

[0041] This method can quantitatively evaluate the causal influence degree of each manufacturing parameter on the solder joint defects by statistically analyzing the system responses under different intervention conditions, providing a scientific basis for root cause analysis.

[0042] Step 4.4, defect root cause identification; Calculate the causal effects of each manufacturing parameter on solder joint defects and define the parameters on the defect The causal effect intensity is as follows: ; where represents the causal effect intensity of parameter on the defect ; represents the intervention operation, setting to the value , represents the probability of observing the defect under the intervention ; The probability of observing the defect under the intervention ; represents the absolute value operation.

[0043] Based on the causal effect intensity, identify the main manufacturing parameters that cause solder joint defects, sort them according to the effect intensity, and form the root cause analysis results.

[0044] This step outputs the causal effect intensity of each manufacturing parameter on solder joint defects and its sorting results, determines the most likely root cause and its impact degree for each observed defect mode. These results precisely quantify the relationship of "which manufacturing parameter change can most effectively reduce a specific type of defect", provide data support for the precise adjustment and optimization of the manufacturing process, and also provide a verification basis for the further optimization of the causal graph structure.

[0045] Step 5: Based on the intervention data and observation data, apply the differentiable causal discovery algorithm to optimize the causal graph structure and improve the accuracy of root cause analysis.

[0046] This step comprehensively utilizes the parameter anomaly data set collected in Step 3, the causal effect intensity calculated in Step 4, and additional intervention experiment data, and applies the differentiable causal discovery algorithm to optimize the previously constructed causal graph network . The counterfactual analysis results of Step 4 provide the causal relationship intensity for preliminary verification in this step, while the data set in Step 3 provides the basic data for model training. The combination of such multi-source data makes the causal structure optimization more accurate and reliable. Specifically, it includes the following sub-steps: Step 5.1: Construct the differentiable causal structure learning objective; Define the loss function for differentiable causal structure learning: ; where represents the total loss function, represents the data fitting error, Represents the sparsity constraint of the graph structure, represents the intervention consistency constraint, and respectively represent the weight coefficients of the sparsity constraint and the intervention consistency constraint.

[0047] Step 5.2, calculation of data fitting error; To approximately calculate the intervention distribution , the Monte Carlo sampling method is used: Sampling from the noise distribution: , ; where represents the prior distribution of the noise variable , represents the th noise sample, represents "obeys", represents that the sample index ranges from 1 to , represents that the Monte Carlo sampling parameter abnormal data is calculated under the th noise sample value; Calculate: ; where represents the value calculated by the structural equation model under the given intervention , noise sample and causal graph , value, structural equation model; Estimate through , where represents the probability distribution of under the intervention , represents all the calculated sample sets.

[0048] Step 5.3, sparsity constraint of the graph structure; To promote the sparsity of the causal graph structure, introduce regularization constraint: ; where represents the sparsity loss, represents the weight matrix 's norm, Represents the source node to the node edge weight, represents all node pairs for summation, represents the absolute value of the weight.

[0049] Step 5.4, Intervention consistency constraint; To ensure that the learned causal structure is consistent with the intervention experiment results, an intervention consistency constraint is introduced: ; where represents the intervention consistency loss, represents the th intervention experiment, for the variable the intervention is the value input, represents the th intervention experiment, for the variable the intervention value is observed output, represents the output predicted by the model under the given input and the causal graph condition, represents the square of the norm (square of the Euclidean distance), represents the total number of intervention experiments, represents the sum over all intervention experiments from to .

[0050] Step 5.5, Gradient descent optimization; Use the gradient descent method to optimize the causal graph structure: ; where represents the weight update rule, represents the learning rate (controls the step size of each update), represents the loss function with respect to the weight matrix gradient (directional derivative), represents the assignment operation. Through iterative optimization, a more accurate causal graph structure is obtained.

[0051] This method uses the Adam optimizer for gradient update. The initial learning rate is set to 0.001, and a learning rate decay strategy is adopted. After every 50 epochs, the learning rate is reduced to 0.9 times the original value. An early stopping strategy is set. When the validation set loss does not improve for 10 consecutive epochs, the training process is terminated early. The entire optimization process is executed for a maximum of 500 epochs, and the batch size for each epoch is 64. This optimization algorithm configuration can improve the efficiency and accuracy of causal structure discovery while ensuring convergence.

[0052] This step outputs the optimized causal graph network structure and its updated edge weight matrix , and these data structures represent a more accurate causal relationship model between solder joint defects and manufacturing parameters after data validation and optimization. The optimized causal graph network not only improves the accuracy of root cause analysis but also enables the system to adapt to new data and changing manufacturing environments, forming a continuously self-improving closed-loop system. These optimization results provide a reliable theoretical basis for subsequent visual analysis and manufacturing parameter adjustment and are the key technical support for realizing intelligent manufacturing and quality control.

[0053] Step 6, Visualization of solder joint detection results based on augmented reality; The present invention provides a method for visualizing solder joint detection results based on augmented reality technology to help engineers intuitively understand the defect locations and the causal relationships between them and manufacturing parameters, and improve the defect analysis efficiency. It specifically includes the following sub-steps: Step 6.1, 3D reconstruction of solder joints; Based on the 2D image and depth information of the circuit board, a 3D model of the solder joint is constructed: ; where represents the 3D point cloud model of the solder joint, represents the 2D image, represents the depth information, represents the 3D reconstruction function (the algorithm for converting the 2D image and depth information into a 3D model).

[0054] For 3D reconstruction, the structured light projection method is specifically used to obtain depth information, and then combined with RGB image information for point cloud generation. For each solder joint area, multi-angle images (at least 3 different angles) are collected, and the 3D coordinates are calculated through feature point matching and triangulation principle, and finally a high-precision 3D model is generated.

[0055] Step 6.2, Highlight the abnormal area; Highlight the detected abnormal solder joints in the 3D model: ; Among them represents the color of the visualized point . represents a point in the 3D model represents the highlight color represents the abnormal area (i.e., the set of solder joints that satisfy , where represents the Mahalanobis distance represents the threshold value), represents the original color represents the membership relationship ( represents the point belongs to the abnormal area ).

[0056] The system supports multiple highlight display modes, including the heat map mode (indicating the degree of abnormality with a gradient of red, yellow, and blue), the contour mode (only showing the boundary of the abnormal area), and the blinking mode (timed blinking prompts for the abnormal area); users can switch different modes according to actual needs to improve the recognition efficiency of the abnormal area.

[0057] Step 6.3, visualization of causal relationships Visualize the causal relationship between solder joint defects and manufacturing parameters in a graphical way ; Among them represents the visualized causal graph represents the causal graph ( is the set of nodes is the set of edges is the edge weight matrix), represents the 3D model of the solder joint represents the visualization function, which converts the abstract causal graph structure into an intuitive visual representation form

[0058] The visualization of causal relationships adopts a directed graph form, where nodes represent various manufacturing parameters and defect characteristics, and edges represent causal relationships. The thickness of the edge represents the causal strength , and the width of the edge is usually set to ; Among them represents the width of the edge is the scaling factor represents the absolute value of the edge weight, and the color represents positive and negative correlations (red represents positive correlation , blue represents negative correlation ).

[0059] Users can select a specific defect through the interactive interface, and the system automatically highlights all causal paths related to this defect

[0060] Step 6.4, Augmented Reality Interaction Interface; Develop an augmented reality interaction interface to support real-time interaction between the user and the detection results: ; where represents the rendering result of the augmented reality interface, represents the three-dimensional model of the solder joint, represents the visualized causal graph, represents the real-time camera input image, represents the coordinate transformation matrix (mapping the three-dimensional model coordinates to the camera coordinate system), represents the augmented reality rendering function (the algorithm for superimposing the three-dimensional model and visualization information onto the real-time camera image).

[0061] Coordinate transformation matrix is calculated from the camera calibration parameters and the pose estimation algorithm: ; where represents the coordinate transformation matrix, represents the extrinsic parameter matrix (describing the position and orientation of the camera in the world coordinate system), represents the intrinsic parameter matrix (describing the focal length, principal point, and distortion parameters of the camera, etc.).

[0062] The augmented reality interface supports the following interaction functions: gesture zooming, where the user can adjust the model size through a two-finger pinch gesture; touch rotation, where the model view can be rotated by swiping a single finger; voice commands, supporting basic voice instructions such as "zoom in", "zoom out", "rotate"; focus query, where after the user clicks on a specific solder joint, the system automatically displays the detailed information and related manufacturing parameters of that solder joint.

[0063] Step 6.5, Generation of Abnormal Root Cause Analysis Report; Based on the detection and causal analysis results, automatically generate an abnormal root cause analysis report: ; where represents the abnormal root cause analysis report, represents the causal graph, represents the abnormal detection data set, represents the manufacturing parameter set, represents the report generation function.

[0064] Abnormal severity score is calculated based on the following formula: ; where Indicates the abnormal severity score, Indicates the number of abnormal solder joints, Indicates the total number of solder joints, Indicates the sum of the Mahalanobis distances of all abnormal solder joints, and respectively represent the weight coefficients of the number of abnormal solder joints and the average abnormal degree, Indicates the proportion of abnormal solder joints, Indicates the average abnormal degree.

[0065] The report includes the following content: defect statistics, including the quantity, distribution, and severity of various types of defects; critical causal path analysis, listing the manufacturing parameters with the greatest impact and their causal paths; parameter optimization suggestions, providing optimization adjustment suggestions for manufacturing parameters based on the results of causal analysis; historical comparison analysis, comparing the current detection results with historical data to analyze the trend changes of defects.

[0066] A printed circuit board solder joint detection system based on machine vision, used to execute the above-mentioned printed circuit board solder joint detection method based on machine vision, includes: A graph construction module, used to construct a spatial relationship graph of solder joints; An anomaly detection module, used to detect group anomalies of solder joints; A causal analysis module, used to construct a causal graph network of solder joint defects and manufacturing parameters; A counterfactual inference module, used to identify the necessary and sufficient causes of defect formation; A causal optimization module, used to optimize the causal graph structure.

[0067] In an embodiment of the present invention, an application example of the above-mentioned printed circuit board solder joint detection method based on machine vision is provided: It has been actually applied to the printed circuit board solder joint detection production line of an electronic manufacturing enterprise. This production line produces approximately 1000 printed circuit boards per day, and each printed circuit board contains 300 - 500 solder joints. The printed circuit boards are mainly used for smartphone motherboards and communication devices, and have extremely high requirements for the quality of solder joints. The traditional solder joint detection method uses single - point detection technology, which cannot effectively detect group anomaly patterns. Moreover, after an anomaly is detected, it is necessary to manually analyze and investigate possible causes, resulting in low production efficiency and high rework rates.

[0068] First, obtain the printed circuit board design diagram from the CAD system of the electronic manufacturing enterprise, and at the same time use a high - resolution industrial camera to collect images of the actually produced printed circuit boards. Based on this data, extract the solder joint position coordinates and feature information, as shown in Table 1 below: Table 1: Example of solder joint position and feature information

[0069] Based on the solder joint positions and electrical connection information, a relationship matrix between solder joints is constructed, and part of the data is shown in Table 2: Table 2: Relationship Matrix of Solder Joints (Partial)

[0070] Using the circuit board samples marked as normal in the production line historical data, a graph convolutional network model is trained. After the model training is completed, the solder joint spatial relationship graph of the new circuit board is input into the model to obtain the solder joint group representation vector, and then the Mahalanobis distance is calculated to determine whether there is an abnormality.

[0071] The detection results on the circuit boards produced in a certain batch are shown in Table 3: Table 3: Example of Solder Joint Group Abnormality Detection Results

[0072] Based on the historical data collected in the manufacturing system, the causal graph network of solder joint defects and manufacturing parameters is initialized and optimized. For the abnormality of "inconsistent solder joint height" in area B, relevant manufacturing parameter data are collected, and through counterfactual analysis, the causal effect intensity of each manufacturing parameter on the "inconsistent solder joint height" abnormality is calculated, and the results are shown in Table 4: Table 4: Calculation Results of Causal Effect Intensity of Manufacturing Parameters

[0073] Applying the differentiable causal discovery algorithm, through the intervention data of multiple production tests, the causal graph structure is continuously optimized. The comparison of the causal discovery accuracy before and after optimization is shown in Table 5: Table 5: Comparison of Accuracy Before and After Causal Structure Optimization

[0074] After implementing this method, the performance of the circuit board solder joint detection system of this electronic manufacturing enterprise has been improved. It is mainly reflected in the following two aspects: The comparison results between the traditional single-point detection method and this method in terms of group abnormality detection are shown in Table 6: Table 6: Comparison of Group Abnormality Detection Capabilities

[0075] After applying this method, the efficiency and quality of the manufacturing system have also been improved, as shown in Table 7: Table 7: Improvement of Production Efficiency and Quality

[0076] As can be seen from the above application examples, the present method can effectively improve the accuracy and efficiency of circuit board solder joint detection in the actual production environment, especially has advantages in group anomaly detection and root cause analysis, providing reliable quality assurance and decision-making support for manufacturing enterprises.

[0077] The embodiments of the present invention have been described above, but the embodiments are not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative rather than restrictive. Under the inspiration of the present embodiments, those of ordinary skill in the art can also make more equivalent embodiments in various forms, all of which fall within the protection scope of the present embodiments.

Claims

1. A method for detecting solder joints on a circuit board based on machine vision, characterized in that, Including: Based on the circuit board design diagram and the actual image, construct a solder joint spatial relationship diagram; Apply a graph convolutional network to process the solder joint spatial relationship diagram, learn the distribution characteristics of the normal solder joint group, and detect group anomalies that do not conform to the normal distribution by comparing the difference between the actual distribution and the normal distribution model; According to the group anomaly results, construct a causal graph network of solder joint defects and manufacturing parameters, where the nodes include the observed group anomaly characteristics and potential manufacturing parameters, and the edges represent the potential causal relationships between them; For the defect patterns observed in the causal graph network, construct a counterfactual analysis model and calculate the counterfactual probability to identify the necessary and sufficient causes of defect formation; Based on the counterfactual analysis results, the collected intervention data and the observed data, apply the differentiable causal discovery algorithm to optimize the aforementioned causal graph structure.

2. The method for detecting solder joints of a circuit board based on machine vision according to claim 1, wherein The steps of constructing the solder joint spatial relationship diagram include: Obtain the circuit board design diagram and actual image data, and extract the solder joint position coordinates, types, and expected sizes; Extract features for each solder joint to construct a feature vector; Construct a relationship matrix between solder joints to represent the relationship strength between solder joints; Based on the node features and the relationship matrix, construct a complete solder joint spatial relationship diagram.

3. The method for detecting circuit board solder joints based on machine vision according to claim 2, wherein, In the step of constructing the relationship matrix between solder joints, the calculation formula for the relationship strength is: ； Among them represents the strength of the relationship between solder joints and solder joints ; the relationship strength represents the Euclidean distance between solder joints and solder joints ; the Euclidean distance is the distance attenuation parameter represents whether there is an electrical connection relationship between solder joints and solder joints ; if there is a connection, it is 1, and if there is no connection, it is 0 and are the weight coefficients of the spatial relationship and the electrical relationship respectively represents the exponential function 4. A method for detecting solder joints on a circuit board based on machine vision according to claim 1, characterized in that, The graph convolutional network includes a three-layer structure of an input layer, a hidden layer, and an output layer, where: The input layer receives the solder joint node feature matrix and the adjacency matrix; The hidden layer contains 64 neurons; The output layer generates a solder joint representation vector with a dimension of 32; A ReLU activation function is used for non-linear transformation between each layer, and a Dropout layer is added after the hidden layer.

5. A method for detecting solder joints on a circuit board based on machine vision according to claim 1, characterized in that, The steps of detecting group anomalies that do not conform to the normal distribution include: Construct a normal distribution model, expressed as: ； where represents the probability density of the vector . is the representation vector of a single solder joint, is the mean of the representation vectors of the normal solder joint population, is the covariance matrix of the representation vectors of the normal solder joint population, represents the covariance matrix . represents the covariance matrix . is the dimension of the representation vector, represents the transpose operator, represents the exponential function, represents to the power; Calculate the Mahalanobis distance between the actual solder joint group representation vector and the normal distribution model. When the Mahalanobis distance is greater than the preset threshold, determine that the solder joint or solder joint group is abnormal.

6. The method for detecting solder joints on a circuit board based on machine vision according to claim 1, characterized in that, The steps of constructing the causal graph network of solder joint defects and manufacturing parameters include: Define the node set of the causal graph network, including the observed group anomaly characteristic nodes and potential manufacturing parameter nodes; Collect parameter anomaly data to form a parameter anomaly data set; Initialize the causal graph network structure based on domain knowledge; Use the collected parameter anomaly data to learn the edge weights of the causal graph network.

7. A method for detecting solder joints on a circuit board based on machine vision according to claim 1, characterized in that, The steps of constructing the counterfactual analysis model include: Sample multiple noise samples from the posterior distribution; For each noise sample, calculate the result under the intervention condition; Statistically analyze the system responses under different intervention conditions and estimate the counterfactual probability.

8. A method for detecting solder joints on a circuit board based on machine vision according to claim 1, characterized in that, The loss function of the differentiable causal discovery algorithm is defined as: ； Among them represents the total loss function represents the data fitting error represents the graph structure sparsity constraint represents the intervention consistency constraint and represent the weight coefficients of the sparsity constraint and the intervention consistency constraint respectively 9. A method for detecting solder joints on a circuit board based on machine vision according to claim 1, characterized in that, The differentiable causal discovery algorithm uses an Adam optimizer for gradient update, the initial learning rate is set to 0.001, and a learning rate decay strategy is adopted. Every 50 iteration cycles, the learning rate is reduced to 0.9 times the original.

10. A circuit board solder joint detection system based on machine vision, characterized in that, A method for detecting circuit board solder joints based on machine vision for implementing any one of claims 1-9 includes: A graph construction module for constructing a solder joint spatial relationship diagram; An anomaly detection module for detecting solder joint group anomalies; A causal analysis module for constructing a causal graph network of solder joint defects and manufacturing parameters; A counterfactual inference module for identifying necessary and sufficient causes of defect formation; A causal optimization module for optimizing the causal graph structure.

Images