PCB circuit board solder joint detection method and system based on image recognition

Abstract

This invention discloses a method and system for detecting solder joints on PCB circuit boards based on image recognition, belonging to the field of image processing technology. The method first acquires and preprocesses 3D point clouds of PCB solder joints, and calculates the local geometric probability density. Non-uniform downsampling is performed using density inverse weighting, prioritizing the retention of sparse key features such as solder joint pins and removing redundant substrate data, outputting a set of physical feature points. This set is then input into a feature extraction network, and strong correlations in feature dimensions are eliminated through local and global forging mapping technology to construct isotropic hypersphere spatial feature vectors. Finally, a multi-coverage discrimination model containing defective hyperellipsoids and normal hyperspheres is constructed based on local density and spatial features. The solder joint quality is comprehensively judged using an integrated voting strategy. This invention effectively solves the problems of false detection and missed detection caused by sparse feature loss and sample imbalance, and significantly improves the detection accuracy of minute defects.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for detecting solder joints on PCB circuit boards based on image recognition. Background Technology

[0002] Printed circuit boards (PCBs), as the support for electronic components and the carrier of electrical connections, are the core components of the modern electronics industry. In the PCB assembly process, solder joints, as the physical and electrical hubs connecting components to the circuit board, directly determine the reliability and lifespan of electronic products. As electronic products develop towards miniaturization and high density, the size of solder joints is decreasing, their structure is becoming more complex, and they are prone to various defects such as insufficient solder, excessive solder, bridging, tombstoning, and cold solder joints. Traditional manual visual inspection is not only inefficient but also easily affected by human eye fatigue and subjective factors, leading to missed inspections or misjudgments. Therefore, developing efficient and accurate automated inspection technology is of great practical significance and necessity for ensuring the quality of electronic products and improving production efficiency.

[0003] In existing technical solutions, to address the problems of traditional two-dimensional detection's inability to acquire height information and its susceptibility to ambient light interference, several schemes for metal weld joint morphology detection have been proposed. Chinese patent document CN119131495B discloses a method for metal weld joint morphology detection and a post-weld quality inspection and identification system. This scheme mainly involves constructing a multi-angle progressive sampling system to collect weld joint point clouds, designing a point cloud fusion scheme to obtain a complete weld joint morphology, and finally projecting the fused 3D point cloud data onto the XY plane to generate a grayscale image, which is then input into a convolutional neural network for morphological description and quality classification. The aforementioned prior art projects 3D point cloud data into a 2D grayscale image as neural network input. While this dimensionality reduction process is compatible with general networks, it inevitably loses... For judging minute defects such as cold solder joints or trace amounts of tin, the three-dimensional topological structure and geometric deformation information are crucial. In PCB solder joint scenarios, the substrate area is dense and the pin area is sparse. The statistical filtering based on Gaussian distribution used in the current technology is difficult to adaptively distinguish effective sparse feature points from noise. This can easily lead to the accidental deletion of key sparse boundary features or the retention of too much redundant substrate data. In addition, directly relying on deep networks such as ResNet for feature extraction and Softmax classification is often limited by the strong correlation of the original feature dimensions. Moreover, when facing extremely imbalanced defect samples in industrial scenarios, it is difficult to construct clear geometric boundaries to accurately define the distribution range of normal and different types of defects in the feature space, thus limiting the coverage and discrimination accuracy of a small number of defect types. Summary of the Invention

[0004] The technical problem solved by this invention is that existing technologies project 3D point cloud data into 2D grayscale images as input to neural networks. While this dimensionality reduction process is compatible with general networks, it inevitably loses the three-dimensional topological structure and geometric deformation information that is crucial for judging minor defects such as cold solder joints or trace amounts of tin. In PCB solder joint scenarios, the substrate area is dense while the highly reflective pin area is sparse. The statistical filtering based on Gaussian distribution used in existing technologies is difficult to adaptively distinguish between effective sparse feature points and noise, which can easily lead to the accidental deletion of key sparse boundary features or the retention of too much redundant substrate data. In addition, directly relying on deep networks such as ResNet for feature extraction and Softmax classification is often limited by the strong correlation of the original feature dimensions. Moreover, when facing extremely imbalanced defect samples in industrial scenarios, it is difficult to construct clear geometric boundaries to accurately define the distribution range of normal and different types of defects in the feature space, thus limiting the coverage and discrimination accuracy of a small number of defect types.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a PCB circuit board solder joint detection method based on image recognition, comprising the following steps:

[0006] Step S1: Obtain 3D point cloud data of PCB solder joints and perform preprocessing;

[0007] Step S2: Calculate the local geometric probability density of each point in the preprocessed 3D point cloud data, and perform non-uniform downsampling on the preprocessed 3D point cloud data based on the local geometric probability density to output a set of physical feature points.

[0008] Step S3: Input the physical feature point cloud into the feature extraction network to obtain the original feature vector, and perform local forging and global forging on the original feature vector to generate local feature vector and hypersphere space feature vector;

[0009] Step S4: Construct a multi-coverage discrimination model based on the local geometric probability density, local feature vector, and hypersphere space feature vector; obtain the coverage status of the sample to be tested in the multi-coverage discrimination model; and use an integrated voting strategy to comprehensively determine the solder joint quality and defect type, and output the detection results.

[0010] Step S1 includes:

[0011] Acquire 3D point cloud data of PCB solder joints and perform preprocessing;

[0012] The 3D point cloud data is divided into a training sample set and a test sample set according to its purpose. The training sample set includes 3D point cloud data of normal samples and 3D point cloud data of defective samples, and both have category labels. The test sample set includes 3D point cloud data of PCB circuit boards to be inspected.

[0013] The defect types in the defect samples include insufficient solder, excessive solder, bridging, tombstoning, missing leads, encased solder, and cold solder joints;

[0014] The preprocessing includes attitude correction and data segmentation.

[0015] In step S2, the process of calculating the local geometric probability density of each point in the preprocessed 3D point cloud data specifically includes:

[0016] The points in the preprocessed 3D point cloud data to be calculated are used as the target points;

[0017] Obtain the Euclidean distance between the target point and the remaining points in a single point cloud block, sort the Euclidean distances in ascending order, and select the points that rank first in the Euclidean distance sort as the neighbor points of the target point.

[0018] The local geometric probability density of the target point is obtained based on the inverse relationship of distance. The mathematical expression for the local geometric probability density is:

[0019] ;

[0020] in, Points in point cloud data Its first Euclidean distance between neighboring points This represents the total number of points in the current point cloud block. For any point in a single point cloud block Its first Euclidean distance between neighboring points.

[0021] In step S2, the non-uniform downsampling includes a first sampling and a second sampling;

[0022] The specific process of the first sampling includes:

[0023] Statistically analyze the frequency distribution of the Z-coordinate of points in a single point cloud block, divide the height intervals, and identify the height interval with the highest frequency as the PCB substrate plane;

[0024] Set a frequency segmentation threshold, search for the height range upward from the PCB substrate plane, and define the lower limit of the height range that is first lower than the frequency segmentation threshold as the dividing height, and extend upward from there to a preset length range as the joint height range;

[0025] The percentage of points within the height range of the junction is counted, multiplied by the total number of samples and the gain coefficient, to calculate the target number of samples within the height range of the junction.

[0026] Within the joint area, points ranked first, equal to the number of target slots, are selected according to their local geometric probability density from smallest to largest to form the feature point set at the root of the weld joint.

[0027] The specific process of the second sampling includes:

[0028] Randomly select one point from a single point cloud block to construct the first set, and the remaining points form the second set;

[0029] For any point in the second set, calculate the shortest Euclidean distance to all points in the first set, and calculate the weighted distance between any two points. The mathematical expression for the weighted distance is:

[0030] The mathematical expression for the weighted distance is:

[0031] ;

[0032] in, This is the weighting adjustment coefficient. For weighted distance, Let be the shortest Euclidean distance from any point in the second set to all points in the first set. For any point in the second set, ;

[0033] Select the point that maximizes the weighted distance, move it from the second set to the first set, and repeat the weighted distance calculation until the number of points in the first set reaches a preset value. Output the first set as the sparse defect feature point set.

[0034] The set of feature points at the root of the weld joint and the set of feature points of sparse defects are combined to obtain a cloud of physical feature points.

[0035] In step S2, the physical feature point cloud is input into the feature extraction network to obtain the original feature vector;

[0036] The process of local forging mapping specifically includes:

[0037] For each dimension of the original feature vector, the cumulative probability value is obtained through mapping rules and linear interpolation.

[0038] The cumulative probability value is matched with a preset set of discrete mapping points, and the value of the corresponding dimension of the original feature vector is replaced with the uniform distribution value in the set of the closest discrete uniform distribution points.

[0039] Replace each dimension in the original feature vector to obtain a local feature vector;

[0040] The specific process of constructing the mapping rules includes:

[0041] For each dimension of the original feature vector of the training sample set, calculate its training probability density and the cumulative distribution function;

[0042] The process of obtaining the training probability density includes:

[0043] For each dimension of the original feature vector of the training sample set, find the maximum and minimum values ​​of the dimension. The maximum and minimum values ​​constitute a numerical range. Divide the numerical range evenly into multiple bucket intervals. Traverse all training sample sets and count the number of training samples falling into each bucket interval. Divide the number of training samples falling into each bucket interval by the total number of training sample sets as the training probability density. Obtain the training probability density of each bucket interval.

[0044] ;

[0045] in, Indicates the first The cumulative probability value of each bucket interval. The value of is a natural number between 1 and g. Indicates the first Training probability density of each bucket interval;

[0046] The mapping rule is a set of parameters consisting of the minimum and maximum values ​​of the numerical range, the number of bucket intervals, and the cumulative probability value corresponding to each bucket interval.

[0047] In step S3, the specific process of global forging includes:

[0048] The local feature vectors of the training sample set are concatenated into a training matrix, and the covariance matrix of the training matrix is ​​obtained. The inverse of the covariance matrix is ​​used as the global metric matrix. The global metric matrix is ​​decomposed by Cholesky to obtain the transformation matrix. The transformation matrix is ​​multiplied by the local feature vectors to obtain the hypersphere feature vectors.

[0049] In step S4, the multi-coverage discrimination model includes a first coverage discrimination model, a second coverage discrimination model, and a third coverage discrimination model;

[0050] Constructing the first coverage discrimination model specifically includes:

[0051] The defect samples are divided according to defect type, and a hyperellipsoid model is constructed for each defect type. The hyperellipsoid models of all defect types together constitute the first coverage discrimination model.

[0052] Based on the defect samples of each defect type, local feature vectors are obtained, and the mean vector of all local feature vectors is calculated as the geometric center of the hyperellipsoid model.

[0053] Calculate the covariance matrix of the defect sample relative to the geometric center, and perform eigenvalue decomposition on the covariance matrix to obtain several eigenvalues ​​and corresponding eigenvectors;

[0054] The direction of the feature vector is defined as the principal axis direction of the hyperellipsoid, and the feature value is defined as the degree of dispersion of the data in the corresponding principal axis direction;

[0055] Based on the preset coverage coefficient, the semi-axis length of the hyperellipsoid in each principal axis direction is calculated using the eigenvalues.

[0056] Based on the geometric center, the rotation matrix composed of eigenvectors, and the semi-axis lengths of each principal axis, a hyperellipsoid model is constructed.

[0057] The specific process of constructing the second coverage discrimination model includes:

[0058] The second coverage discrimination model is a hypersphere model constructed from normal samples;

[0059] Based on the iterative self-organizing analysis algorithm, the hypersphere spatial feature vectors of normal samples are dynamically clustered, and several cluster centers are adaptively determined according to the preset splitting and merging thresholds.

[0060] For each cluster center, calculate the maximum distance from all normal samples within its cluster to the cluster center, and determine the final coverage radius by combining the fine-tuning coefficient;

[0061] Multiple hypersphere models are constructed with each cluster center as the center and its corresponding coverage radius as the size. These multiple hypersphere models together constitute the second coverage discrimination model.

[0062] Constructing the third coverage discrimination model specifically includes:

[0063] Obtain the local geometric probability density of all normal samples in the training samples, calculate the mean and standard deviation of the local geometric probability density of all normal samples, and construct a third coverage discrimination model;

[0064] For normal samples in the training sample set, the local geometric probability density of each point in the normal sample is obtained, the global mean and global standard deviation of the local geometric probability density of all normal samples are calculated, and the effective interval of physical density of normal solder joints is constructed. The effective interval of physical density is the third coverage discrimination model.

[0065] Step S4, the specific process of obtaining the detection results includes:

[0066] The sample to be tested is processed in steps S1 and S2 to obtain the local geometric probability density, local feature vector, and hypersphere space feature vector of a single point cloud block of the sample to be tested.

[0067] Calculate the average local geometric probability density of the sample to be tested, determine whether the average local geometric probability density falls within the effective range of the third coverage discrimination model, and generate a physical discrimination result.

[0068] Construct hyperellipsoidal discrimination equations for each defect type in the first coverage discrimination model, substitute the local feature vector of the sample to be tested into the hyperellipsoidal discrimination equations for each defect type in the first coverage discrimination model, determine whether it falls into the coverage area of ​​any defect type, and generate defect discrimination results;

[0069] Substitute the hypersphere spatial feature vector of the sample to be tested into the second coverage discrimination model, calculate its distance to each normal cluster center, determine whether it falls into the coverage area of ​​any normal hypersphere, and generate normal discrimination results;

[0070] The physical discrimination results, defect discrimination results, and normal discrimination results are converted into voting intentions on whether the weld joint is qualified or not. The majority voting strategy is used to determine the quality of the weld joint. If it is determined to be a defect, the defect type is determined based on the defect discrimination results.

[0071] The PCB circuit board solder joint detection system based on image recognition includes an acquisition module, a sampling module, a feature extraction module, and a detection module.

[0072] The acquisition module is used to acquire 3D point cloud data of PCB solder joints and perform preprocessing.

[0073] The sampling module is used to calculate the local geometric probability density of each point in the preprocessed 3D point cloud data, and to perform non-uniform downsampling on the preprocessed 3D point cloud data based on the local geometric probability density, and output a set of physical feature points.

[0074] The feature extraction module is used to input the physical feature point cloud into the feature extraction network to obtain the original feature vector, and to perform local forging and global forging on the original feature vector to generate local feature vector and hypersphere space feature vector;

[0075] The detection module is used to construct a multi-coverage discrimination model based on the local geometric probability density, local feature vector, and hypersphere space feature vector, obtain the coverage status of the sample to be tested in the multi-coverage discrimination model, and use an integrated voting strategy to comprehensively determine the solder joint quality and defect type, and output the detection results.

[0076] The beneficial effects of the present invention are as follows: The present invention effectively solves the problem of loss of key sparse features caused by statistical filtering or dimensional projection in the prior art by adopting a non-uniform downsampling strategy based on local geometric probability density. The present invention uses the inverse relationship between the distance between a point and its neighbor to accurately distinguish between high-density substrate redundant areas and low-density solder joint key feature areas. While significantly reducing the data processing scale, it prioritizes the retention of key physical feature points that characterize small deformations and edge topology, thereby significantly improving the sensitivity to capture micro-defects such as cold solder joints and missing pins.

[0077] Building upon this foundation, this invention introduces a unique feature forging mapping mechanism. Through nonlinear interpolation and global metric matrix decomposition, the original feature vectors are forcibly mapped to an isotropic, uniform distribution. This process eliminates the strong correlation and distribution bias between the original physical feature dimensions, constructing a regular hypersphere feature vector. This avoids the redundant coupling problem inherent in features directly extracted by traditional deep learning, providing a solid mathematical foundation for subsequent accurate measurements based on geometric distance. Addressing the extreme imbalance problem caused by the scarcity of defect samples in industrial scenarios, this solution abandons the traditional classifier's tendency to fit the majority class with a single discrimination logic. Instead, it constructs a multi-coverage discrimination model encompassing physical density intervals, defect hyperellipsoids, and normal hyperspheres. This model can accurately enclose normal and defect data clusters with different distribution patterns using clear geometric boundaries. Combined with an integrated voting strategy for physical discrimination, defect discrimination, and normal discrimination, it achieves robust judgment of various complex defects under limited sample conditions, significantly reducing the false negative and false positive rates. Attached Figure Description

[0078] Figure 1 The flowchart illustrates the steps of a PCB circuit board solder joint detection method based on image recognition, as provided in one embodiment of the present invention. Detailed Implementation

[0079] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0080] Example 1, referring to Figure 1 This paper provides a PCB circuit board solder joint detection method based on image recognition, which includes the following steps:

[0081] Step S1: Obtain 3D point cloud data of PCB solder joints and perform preprocessing;

[0082] Step S2: Calculate the local geometric probability density of each point in the preprocessed 3D point cloud data, and perform non-uniform downsampling on the preprocessed 3D point cloud data based on the local geometric probability density to output the physical feature point cloud set.

[0083] Step S3: Input the physical feature point cloud into the feature extraction network to obtain the original feature vector, and perform local forging and global forging on the original feature vector to generate local feature vector and hypersphere space feature vector;

[0084] Step S4: Construct a multi-coverage discrimination model based on local geometric probability density, local feature vectors, and hypersphere space feature vectors; obtain the coverage status of the sample to be tested in the multi-coverage discrimination model; and use an integrated voting strategy to comprehensively determine the solder joint quality and defect type, and output the detection results.

[0085] The proposed detection method constructs a complete closed loop from 3D point cloud data acquisition to multi-dimensional coverage determination. By calculating the local geometric probability density and performing non-uniform downsampling, it can effectively overcome the omission problem of sparse key features (such as weld tip and micro-crack) in traditional uniform sampling. Combining local and global feature forging technology, the original features are mapped to an orthogonal hypersphere space, eliminating redundant correlations between features. Finally, by using a multi-coverage discrimination model and ensemble voting strategy, the classification boundary between normal and defective features is clearly defined at the mathematical and geometric level, significantly improving the detection accuracy and robustness in industrial scenarios with imbalanced samples.

[0086] Step S1 includes:

[0087] Acquire 3D point cloud data of PCB solder joints and perform preprocessing;

[0088] 3D point cloud data is divided into training sample set and test sample set according to its purpose. The training sample set contains 3D point cloud data of normal samples and 3D point cloud data of defective samples, and all of them have category labels; the test sample set contains 3D point cloud data of PCB circuit boards to be inspected.

[0089] The defect types in the defect samples include insufficient solder, excessive solder, bridging, tombstoning, missing leads, encased solder, and cold solder joints;

[0090] Preprocessing includes attitude correction and data segmentation.

[0091] In one specific embodiment of the present invention, 3D point cloud data is obtained from Gerber files of normal and defective PCB circuit boards. The coordinates of each solder joint in the Gerber file are traversed, and based on these coordinates as the center, the data is processed according to the window size (in this embodiment, ). In the 3D point cloud data, spatial segmentation is performed, and each segmented 3D data fragment is a local point cloud block (in this embodiment, the number of points in a local point cloud block is 100,000), and each local point cloud block contains a solder joint.

[0092] The point cloud blocks corresponding to the solder joints of a normal PCB circuit board are normal samples, and the point cloud blocks corresponding to the solder joints of a defective PCB circuit board are defective samples. The defective samples cover the defect types of the solder joints, and the normal samples and defective samples constitute the training sample set.

[0093] Each training sample is stored as coordinate matrix ( The number of points within this point cloud block is 100,000 in this embodiment, and 3 represents the x, y, z coordinates.

[0094] The local point cloud blocks of normal samples and defective samples also include category labels. The category labels are used to indicate that normal samples are normal solder joints, and for defective samples, the category labels indicate the defect type of the solder joint.

[0095] In this embodiment, the category label is set as an integer index. ,in The integer represents a normal, acceptable solder joint, while the other integers correspond to different defect types.

[0096] A 3D laser profilometer is used to collect 3D point cloud data to be tested, and the sample set to be tested is subjected to attitude correction and data block preprocessing.

[0097] Attitude correction utilizes a random sampling consensus algorithm to fit the plane equation of the PCB circuit board in the original 3D point cloud data, calculates the angle between the plane's normal vector and the standard Z-axis and the rotation axis, and rotates the point cloud data to obtain the 3D point cloud data with attitude correction, so that the PCB substrate plane is parallel to the horizontal plane (XY plane), effectively correcting the tilt error caused by mechanical mounting or board warping.

[0098] Rotational transformation involves using a random sample consensus algorithm to randomly select points in the original point cloud data and fit the equation of the plane containing the PCB substrate. The normal vector of the plane is Calculate the normal vector With standard Z-axis vector

[0099] The angle between Given a rotation axis, normalize the rotation axis to obtain a unit rotation axis. Based on the unit rotation axis and the included angle, construct a... The rotation matrix is ​​multiplied by the coordinates of each point in the original 3D point cloud data to obtain the pose-corrected 3D point cloud data.

[0100] Data partitioning includes: setting an upper limit on the number of data blocks in a single point cloud. (In this embodiment, the value is set to 100,000) and the lower limit. (In this embodiment, the value is set to 1,000). The pose-corrected 3D point cloud data is segmented in spatial order. During the segmentation process, when the number of remaining point clouds is less than [a certain value], [the remaining number of point clouds is determined]. At this point, the remaining point cloud is merged into the previous point cloud block to ensure that each data block used for calculation has a statistically significant number of samples. Then, for each segmented point cloud block, the actual number of points is calculated. Dynamically calculate nearest neighbor parameters The mathematical expression for the nearest neighbor parameter is:

[0101] ;

[0102] in, For nearest neighbor parameters, This represents the actual number of point clouds. This represents the floor function, which rounds down to the nearest integer. The largest integer, this step ensures the nearest neighbor parameter It is always a valid integer and can adaptively adjust according to changes in the amount of data, thus ensuring the statistical significance of the local geometric probability density calculation.

[0103] After obtaining the nearest neighbor parameters, the 3D point cloud data of the PCB circuit board after rotational changes is segmented into... The next step is to process the data for each individual point cloud block.

[0104] By introducing attitude correction based on spatial cutting and random sampling consistency algorithm using Gerber files, this invention can accurately locate the physical coordinates of each weld point and adaptively correct tilt errors caused by mechanical installation or board warping, ensuring the consistency of point cloud data on the spatial reference. The mechanism of dynamically calculating nearest neighbor parameters based on the actual number of point clouds ensures that the calculation of local geometric probability density is always statistically significant under different data densities, avoiding calculation distortion caused by fixed parameters when data fluctuates.

[0105] Step S2, the process of calculating the local geometric probability density of each point in the preprocessed 3D point cloud data, specifically includes:

[0106] The points in the preprocessed 3D point cloud data to be calculated are used as the target points;

[0107] Obtain the Euclidean distance between the target point and the remaining points in a single point cloud block, sort the Euclidean distances in ascending order, and select the points that rank first in the Euclidean distance sort as the neighbor points of the target point.

[0108] The local geometric probability density of the target point is obtained based on the inverse relationship of distance. The mathematical expression for the local geometric probability density is:

[0109] ;

[0110] in, Points in point cloud data Its first Euclidean distance between neighboring points This represents the total number of points in the current point cloud block.

[0111] .

[0112] In a specific embodiment of the present invention, the process of calculating the local geometric probability density of each point in the 3D point cloud data specifically includes:

[0113] If each point cloud block of the preprocessed 3D point cloud data contains Let there be a set of points. For each of these points Perform the following operations:

[0114] Set the number of nearest neighbors to search , ;

[0115] Distance points are obtained using Euclidean distance. Recent Each neighboring point, calculate the point. Its first The Euclidean distance between n neighboring points is denoted as . The local geometric probability density is calculated based on the inverse relationship of distance and then normalized. The calculation formula is as follows:

[0116] ;

[0117] in, Points in point cloud data Its first Euclidean distance between neighboring points This represents the total number of points in the current point cloud block. For set any point in Its first The Euclidean distance between neighboring points, numerator Representative point Unnormalized local density intensity, denominator This represents the sum of density intensities of all points in the current data block, used to map local density intensities to a unified probability space to obtain local geometric probability density.

[0118] If the local geometric probability density is greater than the probability density threshold (in this embodiment, the probability density threshold is set to 0.8 times the average local geometric probability density of all points in the current point cloud block), it is classified as a high-density region; if the local geometric probability density is less than or equal to the probability density value, it is classified as a low-density region.

[0119] Specifically, in the substrate area of ​​the PCB circuit board, the point cloud features are dense, and the calculated first... Distance between neighboring points The value will be too small, resulting in a large reciprocal, thus affecting the calculated local geometric probability density. It tends to be larger, corresponding to high-density areas;

[0120] At the tips of solder joints and solder ramps on the PCB board, due to abrupt shape changes, drastic height variations, or laser reflections, the collected point clouds are relatively sparse and discrete, resulting in the calculated first... Distance between neighboring points The value will be too large, resulting in a smaller reciprocal, thus the calculated local geometric probability density set will be too large. It will be on the smaller side, corresponding to low-density areas;

[0121] Through the above steps, the coordinate data of unordered 3D point cloud data can be transformed into feature data with density semantics, and subsequent sampling can then be based on this. Values ​​should be retained preferentially. Low-density critical solder joint feature points with smaller values ​​are excluded. High-density redundant substrate points with larger values.

[0122] In step S2, non-uniform downsampling includes first sampling and second sampling;

[0123] The specific process of the first sampling includes:

[0124] Statistically analyze the frequency distribution of the Z-coordinate of points in a single point cloud block, divide the height intervals, and identify the height interval with the highest frequency as the PCB substrate plane;

[0125] Set a frequency segmentation threshold, search for the height range upward from the PCB substrate plane, and define the lower limit of the height range that is first lower than the frequency segmentation threshold as the dividing height, and extend upward from there to a preset length range as the joint height range;

[0126] The percentage of points within the height range of the junction is counted, multiplied by the total number of samples and the gain coefficient, to calculate the target number of samples within the height range of the junction.

[0127] Within the joint area, points ranked first, equal to the number of target slots, are selected according to their local geometric probability density from smallest to largest to form the feature point set at the root of the weld joint.

[0128] In a specific embodiment of the present invention, the Z coordinates of the current point cloud block are counted, and the distribution range of the Z coordinates in the Z-axis direction is obtained (the distribution range is the interval range formed by the minimum and maximum Z coordinates). The distribution range is divided into several height intervals (25 in this embodiment) with a statistical step size (0.2 mm). All points in the current point cloud block are traversed, the number of points falling into each height interval is counted, and a frequency distribution histogram is constructed.

[0129] Obtain the maximum frequency value and its corresponding Z coordinate value from the frequency distribution histogram. Since the PCB substrate has the largest area and a uniform height, the interval with the highest frequency in the frequency distribution histogram represents the physical plane height of the PCB substrate.

[0130] Take 20% (an empirical value) of the maximum frequency value as the frequency segmentation threshold. Starting from the Z-coordinate value corresponding to the maximum frequency value, search upwards along the Z-axis to find the height interval where the Z-coordinate of the point cloud frequency first falls below the frequency segmentation threshold. Mark the value corresponding to the lower limit of the interval as the boundary height. ;

[0131] Dividing height Extend upwards by h statistical steps (in this embodiment) The range of the joint height is defined as the extension of 0.04mm.

[0132] Count the number of point clouds falling within the height range of the junction, and calculate their proportion of the total number of points in the current point cloud block. ;

[0133] Set the gain factor (In this embodiment) In this embodiment, a total of A points need to be sampled (in this embodiment, A is 1024, that is, the target sampling points set in the physical feature point cloud are 1024). Therefore, the number of samples allocated to the height range of the joint is:

[0134] ;

[0135] in, The number of samples allocated to the height range of the joint. This represents the percentage of point clouds falling within the height range of the junction region in the total number of point clouds. This is the gain coefficient, and a safety upper limit is set for the number of samples allocated to the height range of the joint (in this embodiment, the safety upper limit is 0.5). If The value is greater than 0.5 (And can only get 0.5A).

[0136] Based on the calculated number of samples Among all points falling within the height range of the junction, those ranked first are selected based on their local geometric probability density from smallest to largest. These points will be used to filter out the selected points. Each point is defined as the root feature point set of the solder joint. The root feature point set of the solder joint is used to characterize the minute deformation features at the interface between the solder joint and the substrate.

[0137] This invention designs a sampling strategy for joints based on vertical height frequency distribution. This strategy utilizes the high consistency of the substrate plane in spatial distribution and, by statistically analyzing the frequency histogram of point clouds in the vertical direction, can automatically and accurately locate the physical plane of the PCB substrate. Based on this, it extends upward to lock the critical joint area at the root of the solder joint. Unlike conventional sampling algorithms that treat all areas equally, this scheme introduces a gain coefficient within this narrow but crucial physical connection zone, forcibly increasing the sampling weight of this area. By prioritizing the selection of points with lower local density, it can effectively capture the tiny edge contours and potential crack features at the root of the solder joint. This design ensures that even under the interference of a large amount of substrate data, the micro-deformation features at the root can still be preserved with high fidelity, thereby significantly improving the detection capability of hidden defects such as missing pins and root fractures.

[0138] The specific process of the second sampling includes:

[0139] Randomly select one point from a single point cloud block to construct the first set, and the remaining points form the second set;

[0140] For any point in the second set, calculate the shortest Euclidean distance to all points in the first set, and calculate the weighted distance between any two points. The mathematical expression for the weighted distance is:

[0141] The mathematical expression for weighted distance is:

[0142] ;

[0143] in, This is the weighting adjustment coefficient. For weighted distance, Let be the shortest Euclidean distance from any point in the second set to all points in the first set. For any point in the second set, ;

[0144] Select the point that maximizes the weighted distance, move it from the second set to the first set, and repeat the weighted distance calculation until the number of points in the first set reaches a preset value. Output the first set as the sparse defect feature point set.

[0145] The set of feature points at the root of the weld joint and the set of feature points of sparse defects are combined to obtain the set of physical feature points.

[0146] In a specific embodiment of the present invention, a point is randomly selected from the current point cloud block of the preprocessed 3D point cloud data and placed into the first set (the original first set is an empty set, which is now a set containing one point). The remaining points form a second set. The Euclidean distance from the points in the first set to each point in the second set is calculated. The point in the second set that maximizes the Euclidean distance is selected and moved to the first set (the first set now contains two points).

[0147] For each remaining point in the second set The weighted distance calculation steps are performed, including:

[0148] Calculate points separately The Euclidean distance to all points in the first set (since the first set contains two points, there are two Euclidean distances) is taken as the minimum value among these Euclidean distances, and this minimum value is selected as the point. The shortest physical distance to the first set is denoted as , junction point Its own local geometric probability density The weighted distance is calculated using the following formula. The mathematical expression for the weighted distance is:

[0149] ;

[0150] in, The weight adjustment coefficient is 1 in this embodiment. This formula represents that if point density The smaller the value (i.e., located in a sparse defect or edge region), the larger its weighted distance will be;

[0151] After traversing all points in the second set, select the weighted distance. The largest point is moved from the second set to the first set, thereby updating the first set;

[0152] Repeat the above steps of calculating the weighted distance and updating the first set until the number of points in the first set reaches the preset sampling number, and output the first set as the sparse defect feature point set;

[0153] In this embodiment, the target sampling number set in the physical feature point cloud is 1024, and the number of points in the feature point set at the root of the weld joint is [number missing]. The preset sampling number for the sparse defect feature point set is 1024. The difference.

[0154] This invention proposes a weighted iterative sampling algorithm that integrates local geometric probability density, improving upon traditional spatial farthest point sampling. When calculating the distance between points, this algorithm introduces an inverse density weight. Under this mechanism, sparse point clouds located at the tips of solder joint pins, the edges of solder ramps, and around defects are assigned a greater weighted distance due to their lower local density. Therefore, during the iteration process, they are judged as targets further away and selected preferentially. This strategy overcomes the defect of traditional uniform sampling that easily ignores sparse regions, shifting the sampling focus from simple spatial coverage to feature preservation. This allows the final output feature point set to automatically focus on low-density regions containing rich geometric information, effectively solving the problem of key feature point loss caused by illumination reflection or abrupt shape changes, and providing highly recognizable physical feature inputs for subsequent defect identification.

[0155] In step S2, the physical feature point cloud is input into the feature extraction network to obtain the original feature vector;

[0156] The process of local forging mapping specifically includes:

[0157] For each dimension of the original feature vector, the cumulative probability value is obtained through mapping rules and linear interpolation.

[0158] The cumulative probability value is matched with a preset set of discrete mapping points, and the value of the corresponding dimension of the original feature vector is replaced with the uniform distribution value in the set of the closest discrete uniform distribution points.

[0159] Replace each dimension in the original feature vector to obtain a local feature vector;

[0160] The specific process of constructing mapping rules includes:

[0161] For each dimension of the original feature vector of the training sample set, calculate its training probability density and the cumulative distribution function;

[0162] The process of obtaining the training probability density includes:

[0163] For each dimension of the original feature vector of the training sample set, find the maximum and minimum values ​​of the dimension. The maximum and minimum values ​​constitute the numerical range. Divide the numerical range evenly into multiple bucket intervals. Traverse all training sample sets and count the number of training samples falling into each bucket interval. Divide the number of training samples falling into each bucket interval by the total number of training sample sets as the training probability density. Obtain the training probability density of each bucket interval.

[0164] ;

[0165] in, Indicates the first The cumulative probability value of each bucket interval. The value of is a natural number between 1 and g. Indicates the first Training probability density of each bucket interval;

[0166] The mapping rule is a set of parameters consisting of the minimum and maximum values ​​of the numerical range, the number of bucket intervals, and the cumulative probability value corresponding to each bucket interval.

[0167] In a specific embodiment of the present invention, the PointNet++ network is used as the feature extraction network, which includes a three-layer cascaded set abstraction layer and a one-layer global max pooling layer.

[0168] The input is the set of physical feature points output from step S1, and the data format is as follows: The coordinate matrix, the first layer SA extracts local geometric features, and the output is... The feature map is aggregated by the second layer SA, which is a multi-scale feature map, and the output is... Feature maps, the third layer SA generates a global feature description, output The feature map, after global max pooling, outputs a dimension of... The original feature vector ;

[0169] The process of obtaining the discrete mapping point set specifically includes:

[0170] Set the range of the target uniformly distributed values ​​as Construct within the value range A set of discrete mapping points with equal spacing values (In this embodiment) The mathematical expression for the discrete mapping point set is:

[0171] ;

[0172] in, For discrete mapping point set, For the set of scattered mapping points The first in Each element in the discrete mapping point set is a target uniform distribution value.

[0173] The process of local forging specifically includes:

[0174] The training sample set is subjected to operations such as calculating local geometric probability density, non-uniform downsampling, and inputting into the feature extraction network to obtain the original feature vector of the training sample set;

[0175] For the first feature vector of the training sample set Find the maximum value in each dimension. and minimum value The maximum and minimum values ​​constitute the range. , to this range of values Cut into Each bucket interval, Given a positive integer, iterate through all training sample sets, count the number of training samples falling into each bucket interval, divide the number of training samples falling into each bucket interval by the total number of training sample sets, and use the result as the training probability density to obtain the training probability density of each bucket interval.

[0176] The cumulative probability value is obtained by summing the training probability density from the first bucket interval to the current bucket interval. The mathematical expression for the cumulative probability value is:

[0177] ;

[0178] in, Indicates the first The cumulative probability value of each bucket interval. The value range is 1~ Natural numbers between;

[0179] For each element in the original feature vector of a single point cloud patch of the test sample (No. (dimensional), based on the mapping parameter set constructed during the training phase (the mapping parameters include the minimum value) Maximum value Number of bucket intervals and cumulative probability value Perform the following linear interpolation steps:

[0180] calculate exist relative position The mathematical expression for relative position is:

[0181] ;

[0182] in, for exist The relative position in Let c be the value of the original feature vector of the sample to be tested in the c-th dimension. The maximum value of this dimension in the training sample set. The minimum value of this dimension in the training sample set. This represents the total number of bucket intervals.

[0183] make The integer index after rounding down represents... The index of the bucket interval it is in, let The decimal part represents The offset percentage within this range;

[0184] Get s and The two boundary values ​​(i.e.) and The cumulative probability value of the test sample is calculated using a linear interpolation formula. The mathematical expression for the cumulative probability value of the sample to be tested is:

[0185] ;

[0186] in, This represents the cumulative probability value of the sample to be tested. This represents the cumulative probability value of the s-th bucket interval (i.e., the lower boundary probability of the current interval, corresponding to...). ), This is the cumulative probability value of the (s+1)th bucket interval (i.e., the probability of the upper boundary of the current interval, corresponding to...). ), This represents the relative offset ratio of the measured value within the s-th bucket interval;

[0187] like If the value exceeds the boundary, simply take 0 or 1. Out of bounds representation:

[0188] when (Right now When ), it means the value is less than the minimum value in the training set. Take 0 directly, when (Right now When ), it means the value is greater than the maximum value in the training set. Take 1;

[0189] In the discrete mapping point set In the middle, obtain and The element that is numerically closest is denoted as , The mathematical expression is:

[0190] ;

[0191] in, For the selected target uniform distribution characteristic value, This represents the cumulative probability value of the sample to be tested. For discrete mapping point set Any element in the traversal, "Variable" here means taking the variable that minimizes the absolute value expression. , The mathematical expression represents finding the point in the preset uniformly distributed discrete scale that is closest to the currently calculated cumulative probability value in Euclidean distance, and using it as the target uniformly distributed value for the final mapping output.

[0192] The value of the element corresponding to the c-th dimension in the original feature vector Replace with the uniform distribution value of the target. Replace all the values ​​of all dimensions in the original feature vector according to the above scheme to form the hypersphere feature vector.

[0193] By using local forging mapping based on the cumulative distribution function, this invention can force the unevenly distributed and scaled data in the original feature vector to be mapped to a standard uniform distribution. This process is equivalent to performing a non-linear standardization reshaping of the features, eliminating the weight bias caused by the difference in the value range of features of different dimensions, so that the subsequent feature analysis is no longer constrained by the original dimensions of the data, and laying the foundation for constructing an isotropic feature space.

[0194] In step S3, the specific process of global forging includes:

[0195] The local feature vectors of the training sample set are concatenated into a training matrix, and the covariance matrix of the training matrix is ​​obtained. The inverse of the covariance matrix is ​​used as the global metric matrix. The global metric matrix is ​​decomposed by Cholesky to obtain the transformation matrix. The transformation matrix is ​​multiplied by the local feature vectors to obtain the hypersphere feature vectors.

[0196] In one specific embodiment of the present invention, the local feature vectors of each training sample set are concatenated into a training matrix, where each row of the training matrix is ​​a local feature vector of the training sample set. The covariance matrix of the training matrix is ​​then obtained, and its mathematical expression is as follows:

[0197] ;

[0198] in, The covariance matrix of the training matrix, The total number of training samples in the training sample set. The training matrix is ​​formed by concatenating the local feature vectors of all training samples row by row. The mean vector of all row vectors in the training matrix (i.e., the local feature vectors of all training samples) (given by the matrix) (Calculate the arithmetic mean of each column separately) The transpose of the matrix;

[0199] The inverse of the covariance matrix is ​​used as the global metric matrix. For the global metric matrix Perform Cholesky decomposition ( ) to obtain the transformation matrix Transformation matrix It is a lower triangular matrix, and the transformation matrix will be... Multiplying by each local eigenvector yields the hypersphere space eigenvector.

[0200] This invention utilizes a global forging technique, employing the inverse matrix of the training set's covariance matrix and Cholesky decomposition, to whiten local features. This step not only achieves feature decorrelation and decouples the dependencies between dimensions, but also projects the feature vectors onto an orthogonal hypersphere. In this space, the Euclidean distance between feature points directly corresponds to their similarity, enabling subsequent geometric distance-based classification algorithms to achieve optimal performance and avoiding the interference of feature redundancy on discrimination accuracy.

[0201] In step S4, the multi-coverage discrimination model includes a first coverage discrimination model, a second coverage discrimination model, and a third coverage discrimination model;

[0202] Constructing the first coverage discrimination model specifically includes:

[0203] The defect samples are divided according to defect type, and a hyperellipsoid model is constructed for each defect type. The hyperellipsoid models of all defect types together constitute the first coverage discrimination model.

[0204] Based on the defect samples of each defect type, local feature vectors are obtained, and the mean vector of all local feature vectors is calculated as the geometric center of the hyperellipsoid model.

[0205] Calculate the covariance matrix of the defect sample relative to the geometric center, and perform eigenvalue decomposition on the covariance matrix to obtain several eigenvalues ​​and corresponding eigenvectors;

[0206] The direction of the eigenvector is defined as the principal axis direction of the hyperellipsoid, and the eigenvalue is defined as the degree of dispersion of the data along the corresponding principal axis direction;

[0207] Based on the preset coverage coefficient, the semi-axis length of the hyperellipsoid in each principal axis direction is calculated using eigenvalues;

[0208] A hyperellipsoid model is constructed based on the geometric center, the rotation matrix composed of eigenvectors, and the semi-axis lengths of each principal axis.

[0209] In one specific embodiment of the present invention, for a certain type of defect sample, the local feature vector of the defect sample is obtained, and the mean vector of the local feature vectors of all defect samples is used as the mean center. The mean center is a vector with the same dimension as the local feature vector.

[0210] The training sample set is decentered by subtracting the mean center from the local feature vectors. The decentered training sample set is then concatenated as row vectors to obtain the decentered training matrix. The covariance matrix of the decentered training matrix is ​​obtained, and eigenvalue decomposition is performed on the covariance matrix to obtain eigenvalues ​​and eigenvectors. The eigenvectors are then arranged as row vectors to form a rotation matrix. ;

[0211] The expression for the covariance matrix is:

[0212] ;

[0213] in, The covariance matrix of the centered training matrix. This represents the total number of defect samples for the current specific defect type. It is a matrix formed by concatenating the local feature vectors of defect samples of the current specific defect type row by row. For matrix The mean vector of all row vectors in the vector;

[0214] Introducing coverage factor (In this embodiment, we take 3.0). Calculate the semi-axis length in each principal axis direction. The mathematical expression for the semi-axis length is:

[0215] ;

[0216] in, For the hyperellipsoid in the th The length of the semi-axis in the direction of the principal axis The preset coverage factor (used to adjust the envelope of the hyperellipsoid, based on...) In principle, a value of 3.0 can cover approximately 99.7% of the samples. The first covariance matrix is ​​the first... The eigenvalue (representing the sample data in the th eigenvalue) The degree of dispersion or variance along each principal axis direction), the number of principal axes of the hyperellipsoid model is consistent with the dimension of the local feature vectors.

[0217] In response to the diverse shapes and discrete distribution of defect samples, this invention utilizes eigenvalue decomposition technology to construct a unique hyperellipsoidal model for each defect type. This model, through the calculation of the principal axis direction and semi-axis length, can accurately enclose the specific distribution range of this type of defect in the feature space. Compared with the traditional linear classification boundary, this closed geometric envelope can more strictly define the defect region, effectively prevent interference from unknown patterns, and improve the specificity of identifying specific types of defects.

[0218] The specific process of constructing the second coverage discrimination model includes:

[0219] The second coverage discrimination model is a hypersphere model constructed from normal samples;

[0220] Based on the iterative self-organizing analysis algorithm, the hypersphere spatial feature vectors of normal samples are dynamically clustered, and several cluster centers are adaptively determined according to the preset splitting and merging thresholds.

[0221] For each cluster center, calculate the maximum distance from all normal samples within its cluster to the cluster center, and determine the final coverage radius by combining the fine-tuning coefficient;

[0222] Multiple hypersphere models are constructed with each cluster center as the center and its corresponding coverage radius as the size. These multiple hypersphere models together constitute the second coverage discrimination model.

[0223] Obtain the local geometric probability density of all normal samples in the training samples, calculate the mean and standard deviation of the local geometric probability density of all normal samples, and construct a third coverage discrimination model;

[0224] The construction of the third coverage discrimination model specifically includes:

[0225] For normal samples in the training sample set, the local geometric probability density of each point in the normal sample is obtained, the global mean and global standard deviation of the local geometric probability density of all normal samples are calculated, and the effective interval of physical density of normal solder joints is constructed. The effective interval of physical density is the third coverage discrimination model.

[0226] In one specific embodiment of the present invention, the hypersphere space feature vectors of normal samples in the training sample set are obtained to form a set. Set the number of clusters Minimum number of samples Splitting threshold Merging threshold ;

[0227] In this embodiment ;

[0228] Random selection Using the hypersphere feature vectors of the 10 normal samples as initial cluster centers, all remaining normal samples are assigned to the nearest initial cluster centers, generating... In the iterative self-organizing analysis, if the number of samples in a classification cluster is less than a certain value... If the sample variance of a cluster exceeds a certain threshold, then that cluster is removed. If the initial cluster centers of the two new clusters are less than 1, then the cluster will split into two new clusters. If so, then the two clusters will be merged into one cluster;

[0229] After multiple iterations, the iteration stops after obtaining four classification clusters (i.e., the four main subtypes exhibited by normal samples), namely the first category, the second category, the third category, and the fourth category, and the geometric centers of these four categories are output. ;

[0230] For each category, a hypersphere model is constructed separately;

[0231] Iterate through all samples in the first category and calculate the center of each sample. Find the Euclidean distance and obtain the maximum value of the Euclidean distance. Set the radius fine-tuning coefficient (In this embodiment, we take 1.1) Calculate the coverage radius of the first hypersphere. ;

[0232] Similarly, using the centers of the second, third, and fourth categories as the sphere centers, the maximum Euclidean distance within each cluster is multiplied by the fine-tuning coefficient. Construct a second hypersphere model with radius ( ). ), the third hypersphere model ( and the fourth hypersphere model ( );

[0233] The construction of the third coverage discrimination model specifically includes:

[0234] For normal samples in the training sample set, obtain the local geometric probability density of each point in the normal samples;

[0235] Calculate the global mean and global standard deviation of the local geometric probability density of all normal samples, and construct the effective interval of the physical density of normal solder joints. The mathematical expression for the effective interval of physical density is:

[0236] ;

[0237] in, For the third coverage discrimination model, This represents the mean local geometric probability density of the normal sample. 3 represents the local geometric probability density standard deviation of the normal sample, and 3 is the statistical tolerance coefficient.

[0238] Considering that normal solder joints also have certain morphological fluctuations in actual production, this invention uses an iterative self-organizing analysis algorithm to dynamically cluster normal samples and constructs multiple hypersphere models with adaptive radii to cover the subtype distribution of normal samples. At the same time, combined with the effective range of physical density based on statistical tolerance coefficient, a dual constraint of geometric features and physical density is formed. This design not only accommodates the reasonable deviation of qualified solder joints, but also eliminates pseudo-qualified points that are similar in appearance but have abnormal solder content through the hard threshold of physical density.

[0239] Step S4, the specific process of obtaining the detection results includes:

[0240] The sample to be tested is processed in steps S1 and S2 to obtain the local geometric probability density, local feature vector, and hypersphere space feature vector of a single point cloud block of the sample to be tested.

[0241] Calculate the average local geometric probability density of the sample to be tested, determine whether the average local geometric probability density falls within the effective range of the third coverage discrimination model, and generate the physical discrimination result.

[0242] Construct hyperellipsoidal discrimination equations for each defect type in the first coverage discrimination model, substitute the local feature vector of the sample to be tested into the hyperellipsoidal discrimination equations for each defect type in the first coverage discrimination model, determine whether it falls into the coverage area of ​​any defect type, and generate defect discrimination results;

[0243] Substitute the hypersphere spatial feature vector of the sample to be tested into the second coverage discrimination model, calculate its distance to each normal cluster center, determine whether it falls into the coverage area of ​​any normal hypersphere, and generate normal discrimination results;

[0244] The physical discrimination results, defect discrimination results, and normal discrimination results are converted into voting intentions on whether the weld is qualified or not. The majority voting strategy is used to determine the quality of the weld joint. If it is determined to be a defect, the defect type is determined based on the defect discrimination results.

[0245] Calculate the mean of the local geometric probability density of all M sampling points of the current point cloud block in the test sample, and determine whether the mean of the local geometric probability density of the point cloud block of the test sample belongs to the valid range of physical density. If it belongs to the valid range of physical density, the physical discrimination result is marked as qualified; if it does not belong to the valid range of physical density, the physical discrimination result is marked as unqualified.

[0246] For the m-th defect type, calculate the normalized Mahalanobis distance between the local feature vector of the sample to be tested and the center of the m-th defect type. The mathematical expression for the distance, the normalized Mahalanobis distance, is:

[0247] ;

[0248] in, The normalized distance of the sample to be tested relative to the center of the m-th defect type is denoted as . For feature dimension, Let be the semi-axis length of the hyperellipsoidal model of the m-th defect in the j-th principal axis direction;

[0249] The mean center of the m-th defect is subtracted from the local characteristic vector of the sample under test and then rotated by a rotation matrix. The projected coordinates along the j-th principal axis after rotation, i.e., the local characteristic vector minus the mean center, are... Multiplication yields a product, which is also a vector. The first in Elements (i.e., numerical values) ), that is, the sample in the th ... Projected coordinates along each principal axis;

[0250] For the m-th defect type, the mathematical expression of the hyperellipsoidal discrimination equation is:

[0251] ;

[0252] If there exists any defect type m such that If the condition is met, it means the sample falls within the defect coverage area. The defect determination result is then marked as unqualified, and the type of defect hit is recorded. If none of the defect types m can satisfy the condition... If the defect is found to be valid, the defect determination result is marked as qualified.

[0253] Substitute the hypersphere feature vector of the sample to be tested into the second coverage discrimination model, that is, calculate the Euclidean distance from the hypersphere feature vector of the sample to the four cluster centers. The mathematical expression for Euclidean distance is:

[0254] ;

[0255] in, For the sample to be tested up to the first Euclidean distance between cluster centers For the first The coordinate vectors of the cluster centers The component in the d-th dimension, for The component in the d-th dimension, For dimensions;

[0256] If any cluster center exists , making ( If the coverage radius of the k-th hypersphere model is given, then the normal discrimination result is marked as qualified. If all cluster centers None of them can make ( For the first If the coverage radius of a hypersphere model is less than or equal to the normal discrimination result, then the result is marked as unqualified.

[0257] Count the number of qualified votes from the above physical discrimination results, defect discrimination results, and normal discrimination results;

[0258] If the number of qualified votes is greater than or equal to 2, then the point cloud blocks in the sample to be tested are finally judged to be normal and qualified.

[0259] If the number of qualified votes is less than 2, the point cloud block is ultimately determined to be unqualified, and the defect type of the unqualified point cloud block is further determined.

[0260] If the defect identification result records the hit defect type, then the defect type of the unqualified point cloud block is the hit defect type recorded in that record.

[0261] If the unrecorded defect type (i.e. the defect judgment result is qualified) but the physical judgment result is unqualified, and the mean of the local geometric probability density of the sample to be tested is less than the global mean of the local geometric probability density of the normal sample, then the defect type of the unqualified point cloud block is determined to be insufficient solder and empty solder.

[0262] If the physical discrimination result is unqualified, and the mean of the local geometric probability density of the sample to be tested is greater than or equal to the global mean of the local geometric probability density of the normal sample, then the defect type of the unqualified point cloud block is determined to be foreign object occlusion and excess tin.

[0263] All point cloud blocks of the sample to be tested are processed in sequence using steps S2, S3 and S4. The judgment results of all point cloud blocks of the sample to be tested are obtained and summarized. If the judgment results of all point cloud blocks are qualified, the test result of the sample to be tested (i.e. PCB circuit board solder joint) is qualified.

[0264] If the judgment result of any point cloud block is unqualified, the test result of the sample to be tested (i.e., the solder joint of the PCB circuit board) is unqualified, and the defect type of the unqualified point cloud block is output.

[0265] This solution ultimately adopts an integrated voting strategy that combines the results from three dimensions: physical density discrimination, defect geometric coverage discrimination, and normal geometric coverage discrimination. This majority voting mechanism greatly reduces the risk of misjudgment by a single model. In particular, when there is a discrepancy in geometric feature discrimination, the system can use the comparison between the average local geometric probability density of the samples and the global mean to make logical inferences about defects such as insufficient solder, empty solder or excessive solder, and foreign object obstruction from the perspective of physical solder content, providing a physically interpretable fallback guarantee for the final quality judgment.

[0266] The PCB circuit board solder joint detection system based on image recognition includes an acquisition module, a sampling module, a feature extraction module, and a detection module.

[0267] The acquisition module is used to acquire 3D point cloud data of PCB solder joints and perform preprocessing.

[0268] The sampling module is used to calculate the local geometric probability density of each point in the preprocessed 3D point cloud data, and to perform non-uniform downsampling on the preprocessed 3D point cloud data based on the local geometric probability density, and output the physical feature point cloud set.

[0269] The feature extraction module is used to input the physical feature point cloud into the feature extraction network to obtain the original feature vector, and to perform local forging and global forging on the original feature vector to generate local feature vector and hypersphere space feature vector;

[0270] The detection module is used to construct a multi-coverage discrimination model based on local geometric probability density, local feature vectors and hypersphere space feature vectors, obtain the coverage status of the sample under test in the multi-coverage discrimination model, and use an integrated voting strategy to comprehensively determine the solder joint quality and defect type, and output the detection results.

[0271] This invention effectively solves the problem of loss of key sparse features caused by statistical filtering or dimensional projection in the prior art by adopting a non-uniform downsampling strategy based on local geometric probability density. This solution uses the inverse relationship between the distance between a point and its neighbors to accurately distinguish between high-density substrate redundant areas and low-density solder joint key feature areas. While significantly reducing the data processing scale, it prioritizes the retention of key physical feature points that characterize small deformations and edge topology, thereby significantly improving the sensitivity to capture micro-defects such as cold solder joints and missing pins.

[0272] Building upon this foundation, the present invention introduces a unique feature forging mapping mechanism. Through nonlinear interpolation and global metric matrix decomposition techniques, it forces the original feature vectors to be mapped into an isotropic, uniform distribution.

[0273] Specifically, the original extracted physical features often have strong correlations between dimensions, and the numerical scales of different feature dimensions may vary greatly. This causes the data to present a flat, tilted or irregular anisotropic form in geometric space. In this state, directly calculating the distance is inaccurate because features with large numerical ranges will mask key small features, and the coupling between features will cause information redundancy.

[0274] This invention calculates the inverse of the covariance matrix and performs Cholesky decomposition to obtain a transformation matrix that linearly transforms the feature vectors. Mathematically, this process forces the originally entangled and stretched feature distribution to be shaped into a standard sphere with identical properties in all directions (i.e., isotropic). This process eliminates the strong correlation and distribution bias between the original physical feature dimensions, constructing a regular hyperspherical feature vector. This avoids the problem of redundant coupling in features directly extracted by traditional deep learning, providing a solid mathematical foundation for subsequent accurate measurements based on geometric distance. Addressing the extreme imbalance problem caused by the scarcity of defect samples in industrial scenarios, this solution abandons the single discrimination logic of traditional classifiers that tend to fit the majority class. Instead, it constructs a multi-coverage discrimination model that includes physical density intervals, defect hyperellipsoids, and normal hyperspheres. This model can accurately enclose normal and defect data clusters with different distribution patterns using clear geometric boundaries. Combined with an integrated voting strategy of physical discrimination, defect discrimination, and normal discrimination, it achieves robust judgment of various complex defects under conditions of few samples, significantly reducing the false negative and false positive rates.

[0275] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0276] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the present invention.

Claims

1. A PCB solder joint detection method based on image recognition, characterized in that, Includes the following steps: Step S1: Obtain 3D point cloud data of PCB solder joints and perform preprocessing; Step S2: Calculate the local geometric probability density of each point in the preprocessed 3D point cloud data, and perform non-uniform downsampling on the preprocessed 3D point cloud data based on the local geometric probability density to output a set of physical feature points. Step S3: Input the physical feature point cloud into the feature extraction network to obtain the original feature vector, and perform local forging and global forging on the original feature vector to generate local feature vector and hypersphere space feature vector; Step S4: Construct a multi-coverage discrimination model based on the local geometric probability density, local feature vector, and hypersphere space feature vector; obtain the coverage status of the sample to be tested in the multi-coverage discrimination model; and use an integrated voting strategy to comprehensively determine the solder joint quality and defect type, and output the detection results.

2. The image recognition-based PCB circuit board solder joint detection method of claim 1, wherein, Step S1 includes: Acquire 3D point cloud data of PCB solder joints and perform preprocessing; The 3D point cloud data is divided into a training sample set and a test sample set according to its purpose. The training sample set includes 3D point cloud data of normal samples and 3D point cloud data of defective samples, and both have category labels. The test sample set includes 3D point cloud data of PCB circuit boards to be inspected. The defect types in the defect samples include insufficient solder, excessive solder, bridging, tombstoning, missing leads, encased solder, and cold solder joints; The preprocessing includes attitude correction and data segmentation.

3. The image recognition-based PCB circuit board solder joint inspection method of claim 2, wherein, In step S2, the process of calculating the local geometric probability density of each point in the preprocessed 3D point cloud data specifically includes: The points in the preprocessed 3D point cloud data to be calculated are used as the target points; Obtain the Euclidean distance between the target point and the remaining points in a single point cloud block, sort the Euclidean distances in ascending order, and select the points that rank first in the Euclidean distance sort as the neighbor points of the target point. The local geometric probability density of the target point is obtained based on the inverse relationship of distance. The mathematical expression for the local geometric probability density is: ； wherein, is the local geometric probability density, is a point in the point cloud data the Euclidean distance between the point and its first neighbor point, is the total number of points in the current point cloud block, is an arbitrary point in a single point cloud block Its first Euclidean distance between neighboring points.

4. The PCB circuit board solder joint detection method based on image recognition as described in claim 3, characterized in that, In step S2, the non-uniform downsampling includes a first sampling and a second sampling; The specific process of the first sampling includes: Statistically analyze the frequency distribution of the Z-coordinate of points in a single point cloud block, divide the height intervals, and identify the height interval with the highest frequency as the PCB substrate plane; Set a frequency segmentation threshold, search for the height range upward from the PCB substrate plane, and define the lower limit of the height range that is first lower than the frequency segmentation threshold as the dividing height, and extend upward from there to a preset length range as the joint height range; The percentage of points within the height range of the junction is counted, multiplied by the total number of samples and the gain coefficient, to calculate the target number of samples within the height range of the junction. Within the joint area, points ranked first, equal to the number of target slots, are selected according to their local geometric probability density from smallest to largest to form the feature point set at the root of the weld joint. The specific process of the second sampling includes: Randomly select one point from a single point cloud block to construct the first set, and the remaining points form the second set; For any point in the second set, calculate the shortest Euclidean distance to all points in the first set, and calculate the weighted distance between any two points. The mathematical expression for the weighted distance is: The mathematical expression for the weighted distance is: ； in, This is the weighting adjustment coefficient. For weighted distance, Let be the shortest Euclidean distance from any point in the second set to all points in the first set. For any point in the second set, ; Select the point that maximizes the weighted distance, move it from the second set to the first set, and repeat the weighted distance calculation until the number of points in the first set reaches a preset value. Output the first set as the sparse defect feature point set. The set of feature points at the root of the weld joint and the set of feature points of sparse defects are combined to obtain a cloud of physical feature points.

5. The PCB circuit board solder joint detection method based on image recognition as described in claim 4, characterized in that, In step S2, the physical feature point cloud is input into the feature extraction network to obtain the original feature vector; The process of local forging mapping specifically includes: For each dimension of the original feature vector, the cumulative probability value is obtained through mapping rules and linear interpolation. The cumulative probability value is matched with a preset set of discrete mapping points, and the value of the corresponding dimension of the original feature vector is replaced with the uniform distribution value in the set of the closest discrete uniform distribution points. Replace each dimension in the original feature vector to obtain a local feature vector; The specific process of constructing the mapping rules includes: For each dimension of the original feature vector of the training sample set, calculate its training probability density and the cumulative distribution function; The process of obtaining the training probability density includes: For each dimension of the original feature vector of the training sample set, find the maximum and minimum values ​​of the dimension. The maximum and minimum values ​​constitute a numerical range. Divide the numerical range evenly into multiple bucket intervals. Traverse all training sample sets and count the number of training samples falling into each bucket interval. Divide the number of training samples falling into each bucket interval by the total number of training sample sets as the training probability density. Obtain the training probability density of each bucket interval. ； in, Indicates the first The cumulative probability value of each bucket interval. The value of is a natural number between 1 and g. Indicates the first Training probability density of each bucket interval; The mapping rule is a set of parameters consisting of the minimum and maximum values ​​of the numerical range, the number of bucket intervals, and the cumulative probability value corresponding to each bucket interval.

6. The PCB circuit board solder joint detection method based on image recognition as described in claim 5, characterized in that, In step S3, the specific process of global forging includes: The local feature vectors of the training sample set are concatenated into a training matrix, and the covariance matrix of the training matrix is ​​obtained. The inverse of the covariance matrix is ​​used as the global metric matrix. The global metric matrix is ​​decomposed by Cholesky to obtain the transformation matrix. The transformation matrix is ​​multiplied by the local feature vectors to obtain the hypersphere feature vectors.

7. The PCB circuit board solder joint detection method based on image recognition as described in claim 6, characterized in that, In step S4, the multi-coverage discrimination model includes a first coverage discrimination model, a second coverage discrimination model, and a third coverage discrimination model; Constructing the first coverage discrimination model specifically includes: The defect samples are divided according to defect type, and a hyperellipsoid model is constructed for each defect type. The hyperellipsoid models of all defect types together constitute the first coverage discrimination model. Based on the defect samples of each defect type, local feature vectors are obtained, and the mean vector of all local feature vectors is calculated as the geometric center of the hyperellipsoid model. Calculate the covariance matrix of the defect sample relative to the geometric center, and perform eigenvalue decomposition on the covariance matrix to obtain several eigenvalues ​​and corresponding eigenvectors; The direction of the feature vector is defined as the principal axis direction of the hyperellipsoid, and the feature value is defined as the degree of dispersion of the data in the corresponding principal axis direction; Based on the preset coverage coefficient, the semi-axis length of the hyperellipsoid in each principal axis direction is calculated using the eigenvalues. Based on the geometric center, the rotation matrix composed of eigenvectors, and the semi-axis lengths of each principal axis, a hyperellipsoid model is constructed.

8. The PCB circuit board solder joint detection method based on image recognition as described in claim 7, characterized in that, The specific process of constructing the second coverage discrimination model includes: The second coverage discrimination model is a hypersphere model constructed from normal samples; Based on the iterative self-organizing analysis algorithm, the hypersphere spatial feature vectors of normal samples are dynamically clustered, and several cluster centers are adaptively determined according to the preset splitting and merging thresholds. For each cluster center, calculate the maximum distance from all normal samples within its cluster to the cluster center, and determine the final coverage radius by combining the fine-tuning coefficient; Multiple hypersphere models are constructed with each cluster center as the center and its corresponding coverage radius as the size. These multiple hypersphere models together constitute the second coverage discrimination model. The specific process of constructing the second coverage discrimination model includes: for normal samples in the training sample set, obtaining the local geometric probability density of each point in the normal samples, calculating the global mean and global standard deviation of the local geometric probability density of all normal samples, and constructing the effective interval of the physical density of normal solder joints, wherein the effective interval of physical density is the third coverage discrimination model. Obtain the local geometric probability density of all normal samples in the training samples, calculate the mean and standard deviation of the local geometric probability density of all normal samples, and construct a third coverage discrimination model.

9. The PCB circuit board solder joint detection method based on image recognition as described in claim 8, characterized in that, Step S4, the specific process of obtaining the detection results includes: The sample to be tested is processed in steps S1 and S2 to obtain the local geometric probability density, local feature vector, and hypersphere space feature vector of a single point cloud block of the sample to be tested. Calculate the average local geometric probability density of the sample to be tested, determine whether the average local geometric probability density falls within the effective range of the third coverage discrimination model, and generate a physical discrimination result. Construct hyperellipsoidal discrimination equations for each defect type in the first coverage discrimination model, substitute the local feature vector of the sample to be tested into the hyperellipsoidal discrimination equations for each defect type in the first coverage discrimination model, determine whether it falls into the coverage area of ​​any defect type, and generate defect discrimination results; Substitute the hypersphere spatial feature vector of the sample to be tested into the second coverage discrimination model, calculate its distance to each normal cluster center, determine whether it falls into the coverage area of ​​any normal hypersphere, and generate normal discrimination results; The physical discrimination results, defect discrimination results, and normal discrimination results are converted into voting intentions on whether the weld joint is qualified or not. The majority voting strategy is used to determine the quality of the weld joint. If it is determined to be a defect, the defect type is determined based on the defect discrimination results.

10. A PCB circuit board solder joint detection system based on image recognition, characterized in that, It includes an acquisition module, a sampling module, a feature extraction module, and a detection module; The acquisition module is used to acquire 3D point cloud data of PCB solder joints and perform preprocessing. The sampling module is used to calculate the local geometric probability density of each point in the preprocessed 3D point cloud data, and to perform non-uniform downsampling on the preprocessed 3D point cloud data based on the local geometric probability density, and output a set of physical feature points. The feature extraction module is used to input the physical feature point cloud into the feature extraction network to obtain the original feature vector, and to perform local forging and global forging on the original feature vector to generate local feature vector and hypersphere space feature vector; The detection module is used to construct a multi-coverage discrimination model based on the local geometric probability density, local feature vector, and hypersphere space feature vector, obtain the coverage status of the sample to be tested in the multi-coverage discrimination model, and use an integrated voting strategy to comprehensively determine the solder joint quality and defect type, and output the detection results.

Images