Circuit board welding fault recognition method and system based on machine vision

Abstract

The present application relates to the technical field of electronic manufacturing visual detection, in particular to a circuit board welding fault identification method and system based on machine vision, the method comprising: receiving a welding area image and completing gray scale conversion, Gaussian filtering and adaptive threshold segmentation, and extracting a solder joint binary mask. The geometric shape parameters of the solder joint are calculated, the ideal state category is judged through a welding state inference model, the deviation vector is obtained by comparing the ideal state and the actual parameters, and the state inversion calculation is started. The process calls a solder flowability physical simulation engine, simulates the solder spreading behavior based on temperature field and pad layout data, iteratively adjusts the surface tension coefficient and wetting angle parameters, and matches the simulation morphology and the actual morphology until the matching parameters are mapped as the welding fault type. The present application improves the attribution accuracy of circuit board welding fault identification through physical cause inversion and simulation parameter iterative matching.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology in electronic manufacturing, and in particular to a method and system for identifying circuit board soldering faults based on machine vision. Background Technology

[0002] Conventional circuit board soldering fault identification relies on machine vision for image preprocessing and feature extraction. It obtains solder joint contour information through grayscale conversion, filtering and noise reduction, and threshold segmentation. Then, classification models or decision rules are used to directly map the geometric features of the solder joints to the corresponding fault categories. This approach uses surface feature comparison as the core method for fault determination. Some existing technologies only optimize the accuracy of image segmentation and feature extraction, without conducting in-depth analysis of the underlying causes of solder joint morphological deviations. The identification process remains at the level of a direct correlation between features and faults.

[0003] Existing feature-based direct classification schemes cannot distinguish the physical causes corresponding to deviations in weld joint geometry. The classification results only reflect differences in image features and do not directly correspond to changes in physical parameters during the welding process, resulting in ambiguous physical orientation of the fault identification results. Current technologies do not incorporate simulation mechanisms of the welding physical process, making it impossible to reconstruct the true physical process of weld joint formation through parameter iteration. This makes it difficult to establish a precise correspondence between welding physical parameters and fault types, compromising the accuracy of fault attribution.

[0004] It is necessary to construct a deviation vector between the ideal state category of the solder joint and the actual geometric parameters, and to carry out the reverse deduction of the physical cause based on this deviation vector. At the same time, it is necessary to use a solder flow physical simulation engine to achieve the matching between the simulated form and the actual form through parameter iterative adjustment, thereby completing the mapping between the fault type and the physical parameters. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method and system for identifying circuit board soldering faults based on machine vision.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a machine vision-based method for identifying circuit board soldering faults, comprising: The system receives images of the welding area and performs grayscale conversion and Gaussian filtering to remove noise interference. It then uses adaptive threshold segmentation technology to extract the binarized mask of the weld point. Based on the extracted binary mask of the weld point, the geometric morphology parameters of each weld point are calculated, and a welding state inference model is constructed. The welding state inference model uses the geometric morphology parameters as input features and determines the ideal state category of the weld point through preset classification decision rules. The determined ideal state category is compared with the actual geometric shape parameters, and the deviation vector between the ideal state category and the actual geometric shape parameters is calculated. Based on the deviation vector, the state inversion calculation process is started, and the state inversion calculation process reversely derives the ideal physical cause of the deviation of the current geometric shape. The state inversion calculation process calls the solder flow physical simulation engine, which simulates the spreading behavior of solder under different surface tensions based on the current solder joint temperature field data and pad layout data. By iteratively adjusting the surface tension coefficient and wetting angle parameters in the solder flow physical simulation engine, the geometry of the solder joint output by the simulation is matched with the actual geometry extracted from the image of the soldering area. The surface tension coefficient and wetting angle parameter when a match is successful are mapped back to the standard welding fault type to generate the final identification result.

[0007] As a further aspect of the present invention, the step of receiving the welding area image and performing grayscale conversion and Gaussian filtering to remove noise interference, and extracting the binarized mask of the weld point using adaptive threshold segmentation technology includes: The image of the welding area was captured by an industrial camera and contains the complete outline of the weld point. The color space of the welding area image is converted to grayscale space to obtain a single-channel grayscale image; A Gaussian convolution kernel of a specific size is applied to the grayscale image. The size of the Gaussian convolution kernel is dynamically set according to the average pixel diameter of the solder joint in order to eliminate random noise generated during image acquisition. The global threshold of the grayscale image is calculated using the Otsu method, and the grayscale image is then coarsely segmented. For regions that still stick together after coarse segmentation, a local adaptive thresholding algorithm is introduced to dynamically adjust the segmentation threshold based on the mean and variance of grayscale values ​​in the neighborhood of each pixel. The image after adaptive thresholding is subjected to morphological opening operation to eliminate isolated small noise points, and then morphological closing operation is performed to fill the small holes inside the solder joint contour. The resulting connected components are the binary masks of the solder joints, and the coordinates of the minimum bounding rectangle of each connected component are recorded.

[0008] As a further aspect of the present invention, the construction of a welding state reasoning model, wherein the welding state reasoning model uses the geometric morphology parameters as input features and determines the ideal state category of the weld joint through preset classification decision rules, includes: The geometric parameters include at least the area, perimeter, and roundness of the weld point; Define a multidimensional feature vector, which is composed of the normalized ratio of the area value to the standard area of ​​the pad design and the ratio of the perimeter to the equivalent circle circumference in the geometric morphology parameters. The multidimensional feature vector is input into a multilayer perceptron classifier, and the activation function of the hidden layer of the multilayer perceptron classifier adopts the hyperbolic tangent function; The output layer of the multilayer perceptron classifier is equipped with five neuron nodes, which correspond to five ideal state categories: full, lacking tin, having more tin, offset, and bridging, respectively. Load the pre-trained weight matrix into the multilayer perceptron classifier and perform forward propagation calculation on the multidimensional feature vector; The category label corresponding to the neuron with the highest activation value in the output layer is selected as the ideal state category of the solder joint.

[0009] As a further aspect of the present invention, the determined ideal state category is compared with the actual geometric shape parameters, and a deviation vector between the ideal state category and the actual geometric shape parameters is calculated, including: The deviation vector reflects the degree of deviation of the solder joint in size and shape; Retrieve the ideal geometric template corresponding to the standard area of ​​the solder pad design. The ideal geometric template defines the theoretical area and perimeter of the solder joint in a full state. Read the measured area and measured perimeter of the current weld point, and calculate the absolute difference between the measured area and the theoretical area, as well as the absolute difference between the measured perimeter and the theoretical perimeter. Divide the absolute difference by their respective theoretical values ​​to obtain an area deviation rate and a perimeter deviation rate. The area deviation rate and the perimeter deviation rate are combined into a two-dimensional numerical sequence, which is the deviation vector.

[0010] As a further aspect of the present invention, the step of initiating a state inversion calculation process based on the deviation vector, wherein the state inversion calculation process reversely derives the ideal physical cause of the deviation in the current geometric shape, including: Analyze the sign characteristics of the deviation vector. If the area deviation rate and the perimeter deviation rate are both negative, the physical cause is determined to be insufficient solder supply. If both the area deviation rate and the perimeter deviation rate are positive, the physical cause is likely due to excessive solder buildup. If the area deviation rate fluctuates between positive and negative but the perimeter deviation rate increases, the physical cause is likely to be the solder spike phenomenon caused by excessive solder creep. Based on the determined physical cause tendency, a specific set of boundary condition constraints is locked in the solder flow physical simulation engine. The normalized deviation vector is used as an input constraint and passed to the solver interface of the solder flow physical simulation engine to trigger the iterative calculation of the solver.

[0011] As a further aspect of the present invention, the state inversion calculation process invokes a solder flow physical simulation engine. This engine, based on current solder joint temperature field data and pad layout data, simulates the solder spreading behavior under different surface tensions, including: The temperature curve of the reflow oven temperature zone corresponding to the shooting time is extracted from the metadata of the welding area image and used as the initial temperature field data of the weld point. The length and width dimensions of the pads and the opening shape of the solder mask layer are extracted using an image recognition algorithm to construct the pad layout data; The initial temperature field data and the pad layout data are imported into the internal mesh model of the solder flow physical simulation engine; A volume force load, calculated from the deviation vector, is applied at the boundary of the internal mesh model to simulate gravity and capillary force. Set the physical material properties of the solder flow physical simulation engine, including the density and melting point of the solder alloy, and the surface roughness coefficient of the copper pad; The solder flow physical simulation engine is started to perform transient dynamic simulation, simulating the entire process of solder flow and solidification on the substrate after melting.

[0012] As a further aspect of the present invention, the step of iteratively adjusting the surface tension coefficient and wetting angle parameters in the solder flow physical simulation engine until the geometry of the solder joint output in the simulation matches the actual geometry extracted from the soldering area image includes: Initialize a set of default surface tension coefficients and wetting angle parameters in the solder flow physical simulation engine; Run a simulation process once, export the three-dimensional morphological data of the solder joints after the simulation is completed, and project the three-dimensional morphological data of the solder joints into a two-dimensional simulation contour. The Hausdorff distance between the simulated contour and the actual weld point contour extracted from the weld area image is calculated as a morphological difference metric. Determine whether the morphological difference metric is less than the set convergence threshold. If so, determine that the simulation result matches the actual situation. If not, then the surface tension coefficient and wetting angle parameters are fine-tuned using the steepest descent method based on the gradient direction of the morphological difference measurement value. Repeat the simulation process and parameter fine-tuning steps until the change in the morphological difference metric value between two consecutive iterations is lower than the set tolerance range.

[0013] As a further aspect of the present invention, the step of mapping the surface tension coefficient and wetting angle parameter when the match is successful back to the standard welding fault type to generate the final identification result includes: The final identification result includes the fault location and fault type; Establish a parameter-fault type mapping lookup table, which records the typical fault names corresponding to different abnormal surface tension coefficient ranges and wetting angle ranges; Retrieve the mapping lookup table to find the fault name entry that falls within the same range as the surface tension coefficient and wetting angle parameter when a successful match is achieved; The found fault name entries are identified as the specific fault type of the current solder joint; The coordinates of the smallest bounding rectangle recorded in the binary mask of the solder joint are read as the pixel location information of the fault location; The identified fault type is paired with the pixel location information and encapsulated into a structured identification result data packet.

[0014] As a further aspect of the present invention, after generating the final identification result, the method further includes: integrating the identification results of all solder joints and outputting a complete list of circuit board soldering fault diagnoses, specifically including: Iterate through the identification result data packet corresponding to each solder joint processed in the circuit board soldering fault identification method; Extract the pixel positioning information from each data packet and convert it into the horizontal and vertical coordinates in the physical coordinate system of the circuit board. All converted coordinate information and corresponding fault types are summarized into a single table structure; The faults were classified and statistically analyzed, and the frequency of each fault type on the entire circuit board was calculated. The table structure and statistical data are written together into a standardized diagnostic checklist file, along with a screenshot of the circuit board with fault markers overlaid on it.

[0015] As a further aspect of the present invention, the present invention also includes a circuit board soldering fault identification system based on machine vision, the system including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, it implements the steps of the circuit board soldering fault identification method based on machine vision described above.

[0016] Compared with the prior art, the advantages and positive effects of the present invention are as follows: By determining the ideal state category of the weld joint through a welding state reasoning model, and comparing this ideal state category with the actual geometric morphology parameters to obtain a deviation vector, the state inversion calculation process is initiated based on the deviation vector. The physical causes of the current geometric morphology deviating from the ideal state are deduced in reverse. This transforms the weld joint fault judgment logic from direct classification of surface image features to tracing and locating the underlying physical causes. It establishes a direct correspondence between the fault identification results and the physical causes in the welding process, distinguishing the same geometric morphology deviation phenomenon caused by different physical causes. It eliminates the ambiguity of judgment based solely on surface visual features, strengthens the fit between the fault judgment results and the actual welding physical process, refines the directionality of fault attribution, avoids misjudgment of fault type due to feature similarity, and achieves accurate tracing of the physical causes behind abnormal weld joint morphology.

[0017] During the state inversion calculation, a solder flow physical simulation engine is invoked. Based on solder joint temperature field data and pad layout data, the solder spreading behavior under different surface tensions is simulated. By iteratively adjusting the surface tension coefficient and wetting angle parameters in the simulation engine, the geometric shape of the solder joint output by the simulation is matched with the actual geometric shape extracted from the welding area image. The surface tension coefficient and wetting angle parameters corresponding to the matched state are mapped to the standard welding fault type system. Relying on the simulation iterative matching mechanism of the welding physical process, a quantitative correspondence between welding physical parameters and welding fault types is constructed. This eliminates the fuzzy correspondence between visual features and fault types in traditional identification schemes, enabling the fault identification results to have clear physical parameter representations. This improves the consistency between fault type determination and changes in welding physical parameters, ensures the accuracy and reliability of fault identification attribution, and allows the fault identification results to directly reflect the abnormal state of welding physical parameters, providing direct parameter support for the accurate definition of fault types. Attached Figure Description

[0018] Figure 1 This is a flowchart of the circuit board soldering fault identification method based on machine vision according to the present invention; Figure 2 A flowchart of the solder joint binarization mask extraction method; Figure 3 A flowchart illustrating the construction and classification method for welding state reasoning models; Figure 4 Iterative adjustment curves for solder simulation parameters; Figure 5 Box plot showing the distribution of morphological differences in different welding fault types. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0021] See Figure 1 The system receives images of the welding area acquired by an image acquisition device. Grayscale conversion and Gaussian filtering are performed on the received welding area images to remove noise interference. Then, an adaptive threshold segmentation technique is used to extract a binary mask of the solder joints. Based on the extracted binary mask, the geometric morphology parameters of each solder joint are calculated, and a welding state inference model is constructed accordingly. This inference model uses the aforementioned geometric morphology parameters as input features and determines the ideal state category of the solder joint using preset classification decision rules. The determined ideal state category is compared with the actual geometric morphology parameters, and the deviation vector between the two is calculated. Based on this deviation vector, a state inversion calculation process is initiated. This process aims to reverse-engineer the ideal physical cause of the deviation in the current geometric shape. The state inversion calculation process calls a solder flow physics simulation engine, which simulates the solder spreading behavior under different surface tensions based on the current solder joint temperature field data and pad layout data. The surface tension coefficient and wetting angle parameters in the solder flow physics simulation engine are iteratively adjusted until the simulated solder joint geometry matches the actual geometry extracted from the welding area image. The surface tension coefficient and wetting angle parameter when a match is successful are mapped back to the standard welding fault type to generate the final identification result.

[0022] In one embodiment of the invention, the welding area image is acquired by a 5-megapixel industrial area array camera, with an image resolution set to 2592 x 1944 pixels. The coverage diameter of each weld point in the image ranges from 50 to 80 pixels. The welding area image acquired by the industrial camera contains the complete outline of the weld points, and there is a significant grayscale difference between the weld point area and the green solder resist background. For further details, please refer to [reference needed]. Figure 2The RGB color space of the soldering area image is converted to grayscale space. The conversion process is calculated according to the standard brightness formula Gray=0.299R+0.587G+0.114B, resulting in a single-channel grayscale image with pixel values ​​ranging from 0 to 255. A Gaussian convolution kernel of a specific size is applied to the grayscale image for filtering. In some embodiments, the size of the Gaussian convolution kernel is dynamically set according to the average pixel diameter of the solder joint. The average pixel diameter is obtained by pre-analysis of all candidate solder joint regions in the image. The standard deviation of the Gaussian convolution kernel is proportional to the kernel size. The Gaussian filtering operation eliminates random noise introduced by reflections from the circuit board surface and the camera sensor during image acquisition, while maintaining the sharpness of the solder joint edges.

[0023] In some embodiments, the Otsu method is used to calculate the global threshold of the grayscale image. The Otsu method iterates through all possible thresholds from 0 to 255, calculating the inter-class variance of foreground and background pixels under each threshold segmentation. The threshold that maximizes the inter-class variance is selected as the optimal global segmentation threshold. This optimal global threshold is used to perform a coarse segmentation of the grayscale image, generating a preliminary binary image. In the coarse segmentation result, most solder joint areas are separated from the background. For areas that still exhibit adhesion after coarse segmentation, a local adaptive thresholding algorithm is introduced for fine segmentation. The local adaptive thresholding algorithm dynamically adjusts the segmentation threshold based on the grayscale mean and variance within the neighborhood of each pixel. Optionally, the local neighborhood window size is set to 31 pixels multiplied by 31 pixels. The formula for calculating the local adaptive threshold is expressed as:

[0024] in: Indicates the location in the image coordinates The local threshold used for the pixel at that location, This represents the arithmetic mean of the gray values ​​of all pixels within a local neighborhood window centered on that pixel. This represents the standard deviation of all pixel grayscale values ​​within the local neighborhood window. `k` is a weighting coefficient used to adjust the contribution of the standard deviation to the threshold calculation, and `c` is a constant bias term used to fine-tune the threshold. It can be understood that the local adaptive thresholding algorithm can effectively handle the problem of some areas of the same solder joint being too bright or too dark due to uneven lighting.

[0025] In the specific implementation, the binary image processed by the local adaptive thresholding algorithm undergoes morphological opening. This morphological opening operation uses a circular structuring element with a radius of 3 pixels, performing erosion followed by dilation to eliminate isolated small noise points and burrs in the image. Next, a morphological closing operation is performed on the image, using the same circular structuring element, performing dilation followed by erosion to fill in small holes inside the solder joint contour caused by impurities or uneven coloring, making the solder joint area more complete and smooth. Finally, a connected component analysis algorithm identifies and marks all white connected regions in the image; each independent connected component serves as a binary mask for a solder joint. It can be understood that while generating the binary mask for the solder joint, the coordinates of the minimum bounding rectangle for each connected component are calculated and recorded. These coordinates, with the top-left corner of the image as the origin and in pixels, record the top-left x-coordinate, top-left y-coordinate, width, and height of the rectangle. This coordinate information is used for solder joint localization and geometric parameter calculation in subsequent steps.

[0026] In one embodiment of the present invention, see [reference] Figure 3 The geometric parameters of the solder joint are calculated, including at least the area, perimeter, and roundness. The area is obtained by counting all pixels with a value of 1 in the binary mask of the solder joint; the count result is the area value in pixels. The perimeter is calculated by chain code tracing the outline of the binary mask of the solder joint; the sum of the center distances of all adjacent pixels on the outline is the perimeter value. In some embodiments, roundness is calculated as an indicator describing the near-perfect circularity of the solder joint shape; the roundness calculation formula is expressed as:

[0027] in: This represents the calculated roundness value. Indicates the area of ​​the solder joint. Indicates the perimeter of the solder joint. Pi is a constant. For an ideal circular solder joint, the roundness value equals 1; when the solder joint shape deviates from a circle, the roundness value will be less than 1. In practical implementation, a multidimensional feature vector is defined to characterize the shape of the solder joint. The multidimensional feature vector is composed of the normalized ratio of the area value to the standard area of ​​the solder pad design, and the ratio of the perimeter to the equivalent circle circumference in the geometric morphology parameters. The standard area of ​​the solder pad design is read from the circuit board design drawings or bill of materials and is a fixed value in square millimeters. Before calculation, it needs to be converted to pixel square units according to the image scale. The equivalent circle circumference is the theoretical circumference calculated based on the standard area of ​​the solder pad design, assuming the solder joint is an ideal circle. The calculation formula is... ,in This refers to the standard area of ​​the solder pad design. The area normalization ratio is the measured area of ​​the solder joint divided by the standard area of ​​the solder pad design, and the perimeter ratio is the measured perimeter of the solder joint divided by the equivalent circle circumference. It can be understood that by calculating these ratios instead of directly using the original pixel values, the scale effect caused by imaging at different magnifications can be eliminated, making the multidimensional feature vectors scale-invariant.

[0028] In some embodiments, a multidimensional feature vector is input into a multilayer perceptron classifier. The multilayer perceptron classifier has an input layer, at least one hidden layer, and an output layer. The number of neurons in the input layer is consistent with the dimension of the multidimensional feature vector. The activation function of the hidden layer of the multilayer perceptron classifier is the hyperbolic tangent function, which maps the input values ​​of the neurons to outputs between -1 and 1. Optionally, the hidden layer can be a single layer containing 128 neurons. The output layer of the multilayer perceptron classifier has five neuron nodes, corresponding to five ideal state categories: saturated, insufficient solder, excessive solder, offset, and bridging. The output layer uses the Softmax activation function to convert the raw output values ​​of the five neuron nodes into a probability distribution. A pre-trained weight matrix is ​​loaded into the multilayer perceptron classifier. The pre-trained weight matrix is ​​obtained through training on a large number of labeled solder joint image samples and stored in a configuration file. Forward propagation is performed on the input multidimensional feature vector. The forward propagation calculation includes a linear transformation from the input layer to the hidden layer and activation with the hyperbolic tangent function, and a linear transformation from the hidden layer to the output layer and activation with the Softmax function.

[0029] In one embodiment of the present invention, the determined ideal state category is compared with the actual geometric morphology parameters, and a deviation vector between the ideal state category and the actual geometric morphology parameters is calculated. The deviation vector reflects the degree of deviation of the solder joint in size and shape. An ideal geometric template corresponding to the standard area of ​​the solder pad design is retrieved. The ideal geometric template defines the theoretical area and theoretical perimeter of the solder joint in a full state. The theoretical area and theoretical perimeter are calculated based on standard welding process parameters and solder pad dimensions. The measured area and measured perimeter of the current solder joint are read. The measured area and measured perimeter are calculated from the binary mask of the solder joint, and the absolute difference between the measured area and the theoretical area, and the absolute difference between the measured perimeter and the theoretical perimeter are calculated. In some embodiments, the absolute differences are divided by their respective theoretical values ​​to obtain an area deviation rate and a perimeter deviation rate. The area deviation rate and perimeter deviation rate together characterize the relative deviation of the solder joint shape. The calculation formulas for the area deviation rate and the perimeter deviation rate are expressed as follows:

[0030] in: This represents the calculated area deviation rate or perimeter deviation rate, where M represents the measured area or perimeter of the weld joint, and T represents the theoretical area or perimeter read from the ideal geometric template. It can be understood that when the measured value is greater than the theoretical value, the deviation rate... It is a positive value; when the measured value is less than the theoretical value, the deviation rate is... It is a negative value. In practical implementation, the area deviation rate and the perimeter deviation rate are combined into a two-dimensional numerical sequence. This two-dimensional numerical sequence is the deviation vector, which can be represented as... ,in Represents the area deviation rate. This represents the perimeter deviation rate.

[0031] In some embodiments, based on the deviation vector, a state inversion calculation process is initiated. This process reverse-engineers the ideal physical causes leading to the deviation in the current geometric shape. The sign characteristics of the deviation vector are analyzed, and the tendency of the physical causes is inferred by judging the positive and negative combinations of the area deviation rate and the perimeter deviation rate. If the area deviation rate... Negative value and perimeter deviation rate If the value is negative, the physical cause is likely insufficient solder supply. If the area deviation rate... It is a positive value and the perimeter deviation rate If the value is positive, the physical cause is likely due to excessive solder buildup. If the area deviation rate... Fluctuating positively and negatively within a small range near zero, but with a perimeter deviation rate A significant increase indicates that the physical cause is likely a solder spike phenomenon caused by excessive solder creep. In practice, based on the determined physical cause tendency, a set of boundary condition constraints is locked in the solder flow physical simulation engine. For example, for a tendency of insufficient solder supply, a smaller initial solder volume boundary condition is locked; for a tendency of excessive solder accumulation, a larger initial solder volume boundary condition is locked. Optionally, the normalized deviation vector is used as an input constraint and passed to the solver interface of the solder flow physical simulation engine. The normalization process maps each component of the deviation vector to a standard range of 0 to 1 or -1 to 1. It can be understood that after receiving the input constraint, the solver interface triggers iterative calculations. The iterative calculations use the current geometric shape as the convergence target and adjust the simulation parameters in reverse to match the shape.

[0032] In one embodiment of the present invention, the state inversion calculation process calls a solder flow physical simulation engine. This engine simulates the spreading behavior of solder under different surface tensions based on the current solder joint temperature field data and pad layout data. The initial temperature field data of the solder joint is obtained by extracting the reflow oven temperature curve corresponding to the shooting time from the metadata of the soldering area image. The metadata stores the set temperature and conveyor belt speed of each temperature zone at the time of image acquisition, and the temperature curve is represented in the form of time-temperature data pairs. The length and width dimensions of the pads and the opening shape of the solder resist layer are extracted using an image recognition algorithm to construct the pad layout data. The image recognition algorithm first locates the pad area, then calculates the actual pixel size of the pad through edge detection and contour fitting, and then converts the pixel size into physical dimensions based on the known circuit board size and scale. The opening shape of the solder resist layer is also obtained through contour analysis. In some embodiments, the initial temperature field data and pad layout data are imported into the internal mesh model of the solder flow physical simulation engine. The internal mesh model uses unstructured tetrahedral elements to discretize the pads, solder, and surrounding air domain. Volumetric force loads, derived from the deviation vector, are applied at the boundaries of the internal mesh model to simulate gravity and capillary forces. The formula for calculating the volumetric force load is as follows:

[0033] in: This represents the volume force vector applied to the fluid element. This indicates the density of the solder alloy. Represents the gravitational acceleration vector. This represents the surface tension coefficient of solder. Indicates the curvature of the fluid surface. This represents the unit normal vector of the fluid surface. The physical material properties of the solder flow physics simulation engine are set, including the density and melting point of the solder alloy, and the surface roughness coefficient of the copper pads. These properties are input into the simulation engine as material model parameters. The solder flow physics simulation engine is then started to perform transient dynamics simulation. This transient dynamics simulation uses the finite volume method to solve the coupled system of the Navier-Stokes equations and the heat transfer equations, simulating the entire process of solder heating and melting, spreading and flowing on the pads, and subsequently cooling and solidifying. The simulation time step is adaptively adjusted according to the Courant number condition.

[0034] In practical implementation, the typical values ​​of the physical material properties of the solder flow physical simulation engine can be set with reference to Table 1 below: Table 1: Example Table of Physical Material Properties in Solder Flow Physics Simulation Engine

[0035] In some embodiments, the surface tension coefficient and wetting angle parameters in the solder flow physics simulation engine are iteratively adjusted until the geometry of the solder joint output in the simulation matches the actual geometry extracted from the soldering area image. A set of default surface tension coefficients and wetting angle parameters is initialized in the solder flow physics simulation engine; the default values ​​are based on the typical performance settings of the standard Sn63Pb37 solder alloy on a copper substrate. A complete transient dynamics simulation is run, and the 3D morphology data of the solder joint is exported after the simulation. The 3D morphology data describes the shape of the solidified solder joint in the form of a set of node coordinates. The 3D morphology data of the solder joint is projected along the observation direction as a 2D simulation contour, which consists of a series of ordered 2D coordinate points. It can be understood that the Hausdorff distance between the simulation contour and the actual solder joint contour extracted from the soldering area image is calculated as a morphological difference metric, which measures the maximum and minimum distance between two sets of points. It is determined whether the morphological difference metric is less than a set convergence threshold, which is preset according to the image resolution and solder joint size. If the Hausdorff distance is less than the convergence threshold, the simulation result is considered to match the actual situation. Optionally, if the morphological difference metric is not less than the convergence threshold, the surface tension coefficient and wetting angle parameters are fine-tuned using the steepest descent method based on the gradient direction of the morphological difference metric with respect to the gradient direction. The steepest descent method updates the parameters along the negative gradient direction to reduce the morphological difference metric. The complete simulation process and parameter fine-tuning steps are repeated, with each iteration re-simulating and calculating the difference based on the updated parameters. The iteration process continues until the change in the morphological difference metric calculated in two consecutive iterations is lower than the set tolerance range. At this point, the parameter adjustment is considered to have converged, and the simulated profile fully matches the real profile.

[0036] See Figure 4In the parameter iterative optimization process of the solder flow physical simulation engine, the coordinated adjustment of surface tension coefficient and wetting angle is the core link to achieve solder joint morphology matching. The figure uses the iteration number as the horizontal axis and the left and right vertical axes to represent the dynamic trends of surface tension coefficient (N / m) and wetting angle (°): the solid dotted line represents the surface tension coefficient, which shows a continuous decreasing convergence trend. The initial value is approximately 0.685 N / m, and after 20 iterations, it gradually decreases to approximately 0.488 N / m, reflecting the continuous correction of the solder capillary action intensity during the simulation process; the solid square line represents the wetting angle, which fluctuates and increases within the range of 32° to 43°, reflecting the iterative calibration process of the solder's spreading ability on the copper substrate. From the evolutionary perspective, the decrease in surface tension coefficient and the increase in wetting angle show a significant negative correlation: when the surface tension coefficient decreases rapidly, the wetting angle increases accordingly, indicating a physical coupling constraint relationship between the two—the smaller the surface tension, the easier the solder spreads, and the smaller the corresponding wetting angle. This physical intuition is reversed in the later stages of iteration because the simulation engine simultaneously considers the coupling effects of multiple physical fields such as temperature field and pad morphology. In the later stages of iteration, the fluctuation range of surface tension coefficient narrows, and although the wetting angle still fluctuates, it tends to stabilize at a high level overall. This indicates that the parameter combination has gradually approached the optimal solution that matches the simulated morphology with the actual solder joint contour, providing a reliable parameter basis for subsequent mapping of soldering fault types.

[0037] In one embodiment of the present invention, the surface tension coefficient and wetting angle parameter when a match is successfully established are mapped back to standard soldering fault types to generate a final identification result, which includes the fault location and fault type. A mapping lookup table between parameters and fault types is established. This table records typical fault names corresponding to different abnormal surface tension coefficient ranges and wetting angle ranges. The surface tension coefficient range is based on the variation range of surface tension of the solder alloy under common contamination or oxidation conditions, and the wetting angle range is based on the variation range of wetting characteristics caused by poor solderability of the pad or insufficient temperature. The mapping lookup table is searched to find fault name entries whose surface tension coefficient and wetting angle parameters fall into the same range as those when a match is successfully established. The surface tension coefficient and wetting angle parameters when a match is successfully established are obtained from the output of the solder flow physics simulation engine after iterative convergence. In some embodiments, the found fault name entries are determined as the specific fault type of the current solder joint. Fault name entries include, but are not limited to, "cold solder joint," "virtual solder joint," "solder ball," "tombstone," and "poor wetting." The coordinates of the minimum bounding rectangle recorded in the binary mask of the solder joint are read. The coordinates of the minimum bounding rectangle are represented in the image pixel coordinate system and used as the pixel location information of the fault location. In practice, the determined fault type is paired with pixel positioning information and encapsulated into a structured identification result data packet. The identification result data packet adopts JSON or XML format and contains two main fields: "fault type" and "location coordinates".

[0038] In some embodiments, after generating the final recognition result, the recognition results of all solder joints are integrated to output a complete list of circuit board soldering fault diagnoses. The recognition result data package corresponding to each solder joint processed in the machine vision-based circuit board soldering fault recognition method is traversed. Pixel positioning information is extracted from each recognition result data package. Pixel positioning information is the pixel coordinates of the center point of the solder joint's minimum bounding rectangle, and it is converted into the horizontal and vertical coordinates in the circuit board physical coordinate system. The conversion process is calculated based on a preset scale and coordinate origin offset. The conversion formula is expressed as:

[0039] Where: X and Y represent the horizontal and vertical coordinates in the transformed physical coordinate system of the circuit board, usually in millimeters; x and y represent the horizontal and vertical coordinates of the pixel extracted from the pixel positioning information; S represents the scaling factor from pixel to physical size, the value of which is obtained by camera calibration, representing the actual physical length represented by each pixel. and This represents the offset of the physical coordinate system origin relative to the image pixel coordinate system origin. All transformed coordinate information and corresponding fault types are summarized in a table structure. Each row of the table represents a record of a faulty solder joint, and the columns include "Physical Coordinate X," "Physical Coordinate Y," and "Fault Type." In practice, fault types are categorized and statistically analyzed, and the frequency of each fault type on the entire circuit board is calculated. The calculation process iterates through the summary table, counting the occurrences in the "Fault Type" column. Optionally, the table structure and statistical data are written together into a standardized diagnostic checklist file, which can be in plain text, CSV, or PDF format. Essentially, the file includes a screenshot of the circuit board with overlaid fault markers. This screenshot is generated by drawing a numbered, prominent graphic marker at each identified faulty solder joint location on the original soldering area image and saving it as a new image file.

[0040] See Figure 5In the analysis of morphological differences in circuit board soldering faults, the degree of morphological deviation of different fault types can be intuitively quantified through box plots. The vertical axis of the plot represents the deviation of the actual solder joint geometry from the ideal fullness, calculated from the Hausdorff distance between the simulated and actual contours. The horizontal axis represents five typical soldering faults: cold solder joint, poor solder joint, solder ball, tombstoning, and poor wetting. From the distribution characteristics, cold solder joint faults have the highest overall morphological difference value, with a median of approximately 0.175 and a data distribution range of 0.10–0.25, indicating that its geometric shape deviates most significantly from the ideal state. This is highly consistent with the physical causes of cold solder joints, such as irregular solidification morphology and rough contours due to insufficient solder melting. The morphological difference metrics for cold solder joints and solder ball faults are relatively low, with medians of approximately 0.065 and 0.05, respectively, and distribution ranges concentrated in the ranges of 0.01–0.12 and 0.01–0.14, indicating that the morphological deviations of these two faults are relatively weak. Solder ball faults exhibit slightly higher morphological dispersion than cold solder joints, reflecting the morphological fluctuations caused by the random attachment of solder balls. The morphological difference metrics for tombstoning and poor wetting faults are at a moderate level, with medians of approximately 0.085 for both, and distribution ranges of 0.01–0.14 and 0.01–0.14, respectively. The box plots for both faults have similar box heights and whisker ranges, reflecting similar degrees of morphological deviation caused by component misalignment and insufficient wetting. Overall, the box plots clearly reveal the quantitative differences in morphological deviation among different soldering faults: cold solder joints exhibit the strongest morphological heterogeneity, cold solder joints and solder ball faults exhibit the weakest morphological heterogeneity, while tombstoning and poor wetting are at a moderate level. This provides an intuitive statistical basis for fault type identification based on morphological difference metrics.

[0041] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for identifying circuit board soldering faults based on machine vision, characterized in that, include: The system receives images of the welding area and performs grayscale conversion and Gaussian filtering to remove noise interference. It then uses adaptive threshold segmentation technology to extract the binarized mask of the weld point. Based on the extracted binary mask of the weld point, the geometric morphology parameters of each weld point are calculated, and a welding state inference model is constructed. The welding state inference model uses the geometric morphology parameters as input features and determines the ideal state category of the weld point through preset classification decision rules. The determined ideal state category is compared with the actual geometric shape parameters, and the deviation vector between the ideal state category and the actual geometric shape parameters is calculated. Based on the deviation vector, the state inversion calculation process is started, and the state inversion calculation process reversely derives the ideal physical cause of the deviation of the current geometric shape. The state inversion calculation process calls the solder flow physical simulation engine, which simulates the spreading behavior of solder under different surface tensions based on the current solder joint temperature field data and pad layout data. By iteratively adjusting the surface tension coefficient and wetting angle parameters in the solder flow physical simulation engine, the geometry of the solder joint output by the simulation is matched with the actual geometry extracted from the image of the soldering area. The surface tension coefficient and wetting angle parameter when a match is successful are mapped back to the standard welding fault type to generate the final identification result.

2. The circuit board soldering fault identification method based on machine vision as described in claim 1, characterized in that, The process of receiving the welding area image and performing grayscale conversion and Gaussian filtering to remove noise interference, and extracting the binarized mask of the weld point using adaptive threshold segmentation technology, includes: The image of the welding area was captured by an industrial camera and contains the complete outline of the weld point. The color space of the welding area image is converted to grayscale space to obtain a single-channel grayscale image; A Gaussian convolution kernel of a specific size is applied to the grayscale image. The size of the Gaussian convolution kernel is dynamically set according to the average pixel diameter of the solder joint in order to eliminate random noise generated during image acquisition. The global threshold of the grayscale image is calculated using the Otsu method, and the grayscale image is then coarsely segmented. For regions that still stick together after coarse segmentation, a local adaptive thresholding algorithm is introduced to dynamically adjust the segmentation threshold based on the mean and variance of grayscale values ​​in the neighborhood of each pixel. The image after adaptive thresholding is subjected to morphological opening operation to eliminate isolated small noise points, and then morphological closing operation is performed to fill the small holes inside the solder joint contour. The resulting connected components are the binary masks of the solder joints, and the coordinates of the minimum bounding rectangle of each connected component are recorded.

3. The circuit board soldering fault identification method based on machine vision as described in claim 1, characterized in that, The construction of the welding state inference model, which uses the geometric morphology parameters as input features and determines the ideal state category of the weld joint through preset classification decision rules, includes: The geometric parameters include at least the area, perimeter, and roundness of the weld point; Define a multidimensional feature vector, which is composed of the normalized ratio of the area value to the standard area of ​​the pad design and the ratio of the perimeter to the equivalent circle circumference in the geometric morphology parameters. The multidimensional feature vector is input into a multilayer perceptron classifier, and the activation function of the hidden layer of the multilayer perceptron classifier adopts the hyperbolic tangent function; The output layer of the multilayer perceptron classifier is equipped with five neuron nodes, which correspond to five ideal state categories: full, lacking tin, having more tin, offset, and bridging, respectively. Load the pre-trained weight matrix into the multilayer perceptron classifier and perform forward propagation calculation on the multidimensional feature vector; The category label corresponding to the neuron with the highest activation value in the output layer is selected as the ideal state category of the solder joint.

4. The circuit board soldering fault identification method based on machine vision as described in claim 3, characterized in that, The determined ideal state category is compared with the actual geometric shape parameters, and the deviation vector between the ideal state category and the actual geometric shape parameters is calculated, including: The deviation vector reflects the degree of deviation of the solder joint in size and shape; Retrieve the ideal geometric template corresponding to the standard area of ​​the solder pad design. The ideal geometric template defines the theoretical area and perimeter of the solder joint in a full state. Read the measured area and measured perimeter of the current weld point, and calculate the absolute difference between the measured area and the theoretical area, as well as the absolute difference between the measured perimeter and the theoretical perimeter. Divide the absolute difference by their respective theoretical values ​​to obtain an area deviation rate and a perimeter deviation rate. The area deviation rate and the perimeter deviation rate are combined into a two-dimensional numerical sequence, which is the deviation vector.

5. The circuit board soldering fault identification method based on machine vision as described in claim 4, characterized in that, Based on the deviation vector, the state inversion calculation process is initiated. This process reversely derives the ideal physical cause of the current geometric deviation, including: Analyze the sign characteristics of the deviation vector. If the area deviation rate and the perimeter deviation rate are both negative, the physical cause is determined to be insufficient solder supply. If both the area deviation rate and the perimeter deviation rate are positive, the physical cause is likely due to excessive solder buildup. If the area deviation rate fluctuates between positive and negative but the perimeter deviation rate increases, the physical cause is likely to be the solder spike phenomenon caused by excessive solder creep. Based on the determined physical cause tendency, a specific set of boundary condition constraints is locked in the solder flow physical simulation engine. The normalized deviation vector is used as an input constraint and passed to the solver interface of the solder flow physical simulation engine to trigger the iterative calculation of the solver.

6. The circuit board soldering fault identification method based on machine vision as described in claim 5, characterized in that, The state inversion calculation process calls the solder flow physical simulation engine. Based on the current solder joint temperature field data and pad layout data, the solder flow physical simulation engine simulates the spreading behavior of solder under different surface tensions, including: The temperature curve of the reflow oven temperature zone corresponding to the shooting time is extracted from the metadata of the welding area image and used as the initial temperature field data of the weld point. The length and width dimensions of the pads and the opening shape of the solder mask layer are extracted using an image recognition algorithm to construct the pad layout data; The initial temperature field data and the pad layout data are imported into the internal mesh model of the solder flow physical simulation engine; A volume force load, calculated from the deviation vector, is applied at the boundary of the internal mesh model to simulate gravity and capillary force. Set the physical material properties of the solder flow physical simulation engine, including the density and melting point of the solder alloy, and the surface roughness coefficient of the copper pad; The solder flow physical simulation engine is started to perform transient dynamic simulation, simulating the entire process of solder flow and solidification on the substrate after melting.

7. The circuit board soldering fault identification method based on machine vision as described in claim 6, characterized in that, The step of iteratively adjusting the surface tension coefficient and wetting angle parameters in the solder flow physical simulation engine until the geometry of the solder joint output in the simulation matches the actual geometry extracted from the soldering area image includes: Initialize a set of default surface tension coefficients and wetting angle parameters in the solder flow physical simulation engine; Run a simulation process once, export the three-dimensional morphological data of the solder joints after the simulation is completed, and project the three-dimensional morphological data of the solder joints into a two-dimensional simulation contour. The Hausdorff distance between the simulated contour and the actual weld point contour extracted from the weld area image is calculated as a morphological difference metric. Determine whether the morphological difference metric is less than the set convergence threshold. If so, determine that the simulation result matches the actual situation. If not, then the surface tension coefficient and wetting angle parameters are fine-tuned using the steepest descent method based on the gradient direction of the morphological difference measurement value. Repeat the simulation process and parameter fine-tuning steps until the change in the morphological difference metric value between two consecutive iterations is lower than the set tolerance range.

8. The circuit board soldering fault identification method based on machine vision as described in claim 7, characterized in that, The process of mapping the surface tension coefficient and wetting angle parameter when a match is successful back to the standard welding fault type to generate the final identification result includes: The final identification result includes the fault location and fault type; Establish a parameter-fault type mapping lookup table, which records the typical fault names corresponding to different abnormal surface tension coefficient ranges and wetting angle ranges; Retrieve the mapping lookup table to find the fault name entry that falls within the same range as the surface tension coefficient and wetting angle parameter when a successful match is achieved; The found fault name entries are identified as the specific fault type of the current solder joint; The coordinates of the smallest bounding rectangle recorded in the binary mask of the solder joint are read as the pixel location information of the fault location; The identified fault type is paired with the pixel location information and encapsulated into a structured identification result data packet.

9. The circuit board soldering fault identification method based on machine vision as described in claim 8, characterized in that, After generating the final identification result, the process also includes: integrating the identification results of all solder joints and outputting a complete list of circuit board soldering fault diagnoses, specifically including: Iterate through the identification result data packet corresponding to each solder joint processed in the circuit board soldering fault identification method; Extract the pixel positioning information from each data packet and convert it into the horizontal and vertical coordinates in the physical coordinate system of the circuit board. All converted coordinate information and corresponding fault types are summarized into a single table structure; The faults were classified and statistically analyzed, and the frequency of each fault type on the entire circuit board was calculated. The table structure and statistical data are written together into a standardized diagnostic checklist file, along with a screenshot of the circuit board with fault markers overlaid on it.

10. A machine vision-based circuit board soldering fault identification system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the machine vision-based circuit board soldering fault identification method according to any one of claims 1 to 9.

Images