Solder paste defect detection method based on machine vision

Abstract

The invention discloses a solder paste defect detection method based on machine vision, relates to the technical field of printed circuit board detection, and aims at the challenge of solder paste printing quality detection in superfine-pitch high-reliability manufacturing, high-precision three-dimensional point cloud data is collected and an attitude matrix is marked through laser triangle-grating interference dual-mode imaging and board micro-vibration sensing; constructing an adaptive reference surface based on the point cloud and real-time fitting warping curved surface parameters, executing local height remapping, and eliminating the influence of micro warping; calculating a warping composite slope index and a glare texture robust index by using the height matrix, and adjusting a confidence threshold of the boundary enhanced convolutional network to realize real contour segmentation of adjacent pads; dynamically updating imaging parameters, accumulating real scene samples, and optimizing the adaptive ability of the model; and cross-shift knowledge reuse is realized through the shared model library. According to the invention, the volume measurement consistency and the first pass yield of the ultra-fine pitch bonding pad can be improved, and the process debugging time is shortened.

Description

Technical Field

[0001] The present invention relates to the technical field of printed circuit board detection, and in particular to a solder paste defect detection method based on machine vision. Background Art

[0002] As terminal devices develop towards being thinner and lighter, the spacing between chip pins and pads continues to shrink. Solder paste printing must form a uniform and stable stack in an extremely narrow space to ensure the reliability of subsequent soldering. In order to verify the quality of each print in a timely manner on high-speed production lines, the industry generally introduces three-dimensional solder paste detection systems and machine vision algorithms to capture height, volume and shape information and complete rapid judgment in the online link. Such systems usually use grating interferometry or laser triangulation scanning to generate three-dimensional point clouds, and then combine data-driven closed-loop control to achieve early printing defect interception and parameter self-adjustment. However, when the pad spacing enters the ultra-fine range, substrate warping, strong reflective glare and adjacent contour aliasing appear simultaneously. Traditional imaging and analysis methods are difficult to maintain resolution and stability, which poses hidden dangers to subsequent mounting and reflow processes. Therefore, the production scenario of ultra-fine pitch high-reliability manufacturing has become a key scenario that urgently needs a breakthrough.

[0003] A search revealed a patent application with publication number CN116823832A, which discloses a method for detecting solder paste defects based on a 3D point cloud. The method comprises the following steps: dividing the acquired solder paste point cloud data into M blocks to obtain a block point cloud Fi, where i = 1, ..., M; extracting and merging the planar point clouds Pi from the M block point clouds Fi to obtain a substrate point cloud P; extracting and merging the non-planar point clouds Hi from the M block point clouds Fi to obtain a solder paste point cloud H; and calculating the height, volume, and offset of the solder paste based on the substrate point cloud P and the solder paste point cloud H. The method provides a method for detecting solder paste defects based on a 3D point cloud. By extracting the substrate point cloud P and the solder paste point cloud H from the solder paste point cloud data, the substrate point cloud P and the solder paste point cloud H are used to detect the 3D features of the solder paste point cloud, such as height, volume, and offset. This method more accurately reflects the actual physical state of the solder paste from the 3D features of the solder paste. Compared to traditional 2D recognition, the method offers higher detection accuracy and is effective in addressing PCB bending and warping.

[0004] Combining the above application scenarios and existing technologies: When the optical resolution of existing 3D vision systems approaches its limit, any slight vibration of the device or fluctuation of the board surface will cause the laser stripes to mismatch with the lens angle, making it impossible to correctly distinguish the true boundaries of adjacent pads. Once the outlines are mistakenly merged, the inspection software often interprets stacks that should have been judged as insufficient or continuous solder as normal signals, and defects are then transferred to downstream inspections or even directly into the reflow process. The high temperature stage will further amplify the difference in solder quantity, causing open circuits at best and bridge short circuits at worst, rapidly increasing the cost of rework and scrap. Although the industry has attempted to mitigate the above risks through methods such as multi-angle light sources, artificial intelligence edge enhancement, and digital twin simulation, there is still a lack of an integrated solution with machine vision as the core and both real-time compensation and self-learning capabilities under ultra-fine pitch conditions. This is precisely the key problem that this invention aims to solve. Summary of the Invention

[0005] (1) Technical problems solved To address the shortcomings of existing technologies, the present invention provides a machine vision-based solder paste defect detection method. This method uses dual-mode laser triangulation and grating interferometry imaging and micro-vibration sensing on the board surface to collect high-precision three-dimensional point cloud data and label the pose matrix. Based on the point cloud and real-time fitted warping surface parameters, an adaptive reference surface is constructed, and local height remapping is performed to eliminate the effects of micro-warping. The warping composite slope index and glare texture robustness index are calculated using the height matrix, and the confidence threshold of the boundary-enhanced convolutional network is adjusted to achieve true contour segmentation of adjacent pads. The method dynamically updates imaging parameters, accumulates real-world scene samples, and optimizes the model's adaptability. A shared model library enables cross-shift knowledge reuse. This method can improve the consistency of ultra-fine-pitch pad volume measurement and first-pass yield, resolving the technical issues discussed in the background technology.

[0006] (2) Technical solution To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for detecting solder paste defects based on machine vision, including using a synchronously triggered laser triangulation-grating interferometry dual-mode imaging module in conjunction with a board surface micro-vibration sensor group to collect a three-dimensional point cloud that has undergone primary distortion correction, and marking a posture matrix for each circuit board to determine the accuracy baseline for geometric compensation; Based on 3D point cloud data and posture matrix, by real-time fitting of warping surface parameters, an adaptive reference surface is constructed in each pad area and local height remapping is performed to eliminate coordinate deviations introduced by micro-warping and output a height matrix. After calculating the warping composite slope index and glare texture robustness index on the height matrix, the neighborhood separation coefficient is obtained by inputting the neighborhood separation model, and the confidence threshold of the boundary enhancement convolutional network is adjusted in real time to perform the true contour segmentation of adjacent pads.

[0007] Dynamically update the imaging exposure time and projection frequency based on the comparison of defect confidence and batch history thresholds, mark high-uncertainty primitives and store them in the incremental training cache; Verified measurement weights and parameter adjustment records are written into a shared model library and pushed to printing and quality inspection equipment to enable cross-shift model reuse.

[0008] Furthermore, the synchronously triggered laser triangulation imaging and grating interferometry imaging modules are used to collect data on the circuit board surface at the same time point to generate time-consistent 3D point cloud data. The board surface micro-vibration sensor group is used to monitor the vibration of the circuit board in real time. The time accumulation calculation is performed on the vibration acceleration data to obtain the displacement. The displacement is then applied as a correction factor to the three-dimensional point cloud data to generate vibration-corrected point cloud data.

[0009] Furthermore, based on the pre-calibrated distortion model, polynomial transformation is used to correct the distortion of the vibration-corrected point cloud data. Based on the distortion-corrected point cloud data, a rotation matrix is ​​constructed by fitting the reference plane and calculating the normal vector. At the same time, the translation vector is calculated with the geometric center of the point cloud as the reference point, and the pose matrix is ​​combined.

[0010] Furthermore, the global warping surface is fitted in real time using three-dimensional point cloud data, and the quadratic surface model parameters are solved by the least squares method to describe the overall warping deformation trend of the circuit board, and a local adaptive reference surface is constructed for each pad area.

[0011] Furthermore, reference points in the non-pad area around the pad are selected, the Euclidean distance from the reference point to the center of the pad is calculated and the weight is determined based on the distance, the local reference height is calculated using the weighted average method, and the adaptive reference plane is defined as a horizontal plane with a height equal to the local reference height; local height remapping is performed on the three-dimensional point cloud data points in the pad area, and the relative height relative to the adaptive reference plane is calculated to eliminate the deviation caused by warping.

[0012] Furthermore, the remapped relative height data is organized into a height matrix according to grid division, where the grid is based on the physical size of the pad area, the rows and columns of the height matrix correspond to the horizontal and vertical coordinate points, and the height matrix in a unified format is output.

[0013] Furthermore, the height gradient within each pad area is calculated based on the height matrix and the warpage composite slope index is derived. The complexity of the height variation within the pad area caused by substrate warpage is quantified by combining the local slope and curvature. The height matrix is ​​used to extract texture features, and the gray-level co-occurrence matrix method is used to calculate the texture contrast and obtain the glare texture robustness index, which is used to evaluate texture stability. The warping composite slope index and the glare texture robustness index are input into the pre-trained neighborhood separation model, and the neighborhood separation coefficient is output to quantify the degree of contour aliasing between adjacent pads.

[0014] Furthermore, after comparing the obtained defect confidence with the batch history threshold, the imaging exposure time and projection frequency are dynamically adjusted. The imaging exposure time and projection frequency are adjusted through defect confidence deviation calculation to optimize the imaging quality. At the same time, high-uncertainty primitives are marked and stored in the incremental training cache. The pad areas with defect confidence close to the batch history threshold are identified and the relevant data are recorded to provide real-world scene samples for subsequent model iterations.

[0015] Furthermore, the effectiveness of the measurement weight is evaluated by calculating the F1-score, and an operation is performed based on the result of comparing the F1-score with the preset threshold. If the F1-score is greater than the preset threshold, the measurement weight is written to the shared model library; if the F1-score is less than or equal to the preset threshold, the historical optimal measurement weight is retrieved from the shared model library.

[0016] Furthermore, the dynamically adjusted imaging exposure time and projection frequency are associated with the production batch number, defect type, and defect confidence and stored in a shared model library; Incremental learning is performed using high-uncertainty primitives to update the model parameters in the shared model library. The measurement weights, parameter adjustment records and model parameters in the shared model library are pushed to the printing and quality inspection equipment through the network communication protocol.

[0017] (3) Beneficial effects The present invention provides a solder paste defect detection method based on machine vision, which has the following beneficial effects: In the production scenario of ultra-fine pitch high-reliability manufacturing, the accuracy, stability, and adaptability of solder paste printing quality inspection have been significantly improved, thereby ensuring the reliability of soldering. By combining a synchronously triggered laser triangulation and grating interferometry dual-mode imaging module with a micro-vibration sensor array on the board surface, high-precision 3D point cloud data acquisition and pose matrix marking are achieved. The synchronized triggering of laser triangulation and grating interferometry ensures temporal consistency of the data, while the real-time monitoring and correction of the micro-vibration sensor array effectively eliminates interference from equipment vibration and board surface fluctuations on imaging accuracy, thereby improving the spatial resolution and stability of the point cloud data.

[0018] Based on the point cloud and real-time fitting of the warping surface parameters, an adaptive reference surface is constructed in each pad area and local height remapping is performed, eliminating the coordinate deviation introduced by micro-warping and outputting a height matrix of unified dimension. The global warping surface is combined with the local adaptive reference surface to ensure the accuracy and consistency of the height data and avoid misjudgment due to substrate deformation.

[0019] The warpage composite slope index and glare texture robustness index are calculated on the height matrix and fed into a neighborhood separation model to derive the neighborhood separation coefficient. The confidence threshold of the boundary-enhanced convolutional network is adjusted in real time to achieve true contour segmentation of adjacent pads. The synergistic mechanism of the exponentially driven separation coefficient and dynamic threshold adjustment solves the problem of contour aliasing at ultra-fine pitches and improves the accuracy of pad boundary recognition.

[0020] By dynamically updating the imaging exposure and projection frequency after comparing defect judgments with batch historical thresholds, and marking high-uncertainty primitives into the incremental training cache, real-time optimization of imaging parameters and continuous learning of the model are achieved. This can effectively suppress the impact of exposure saturation and batch drift on imaging quality, and improve the iterative capability of the model through the accumulation of real-scene samples.

[0021] By writing verified measurement weights and parameter adjustment records into a shared model library and pushing them to printing and quality inspection equipment, cross-shift model reuse and continuous self-improvement of production knowledge are achieved. The optimization results of the production process are centrally stored and shared across devices, ensuring the rapid reuse of detection parameters and models, which can significantly shorten process debugging time and improve production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 The figure is a flow chart of the solder paste defect detection method based on machine vision of the present invention. DETAILED DESCRIPTION

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0024] See also Figure 1 The present invention provides a method for detecting solder paste defects based on machine vision, comprising: Step 1: Use the synchronously triggered laser triangulation imaging and grating interferometry imaging modules to collect data on the circuit board surface at the same time point to generate time-consistent 3D point cloud data; The board surface micro-vibration sensor group is used to monitor the tiny vibrations of the circuit board in real time. The time accumulation calculation is performed on the vibration acceleration data to obtain the displacement, and the displacement is applied as a correction factor to the three-dimensional point cloud data to generate vibration-corrected point cloud data; based on the pre-calibrated distortion model, the polynomial transformation method is used to correct the distortion of the vibration-corrected point cloud data to eliminate the influence of optical system distortion on data accuracy; based on the distortion-corrected point cloud data, the rotation matrix is ​​constructed by fitting the reference plane and calculating the normal vector. At the same time, the translation vector is calculated with the geometric center of the point cloud as the reference point, and finally combined into a posture matrix to provide a unique spatial description for each circuit board.

[0025] The step 1 includes the following: Step 101: Synchronous triggering of dual-mode imaging modules Through the synchronously triggered laser triangulation imaging and grating interferometry imaging modules, data on the circuit board surface is collected at the same time point to ensure the temporal consistency of the generated three-dimensional point cloud data.

[0026] Laser triangulation works by projecting a laser beam onto the circuit board surface. Using the angular relationship between the reflected beam and the imaging sensor, triangulated geometry is used to calculate the height information of the surface profile, rapidly generating rough 3D profile data. Grating interferometry, on the other hand, projects periodic stripes onto the circuit board surface, analyzes the deformation of the stripes due to variations in surface height, and combines this with the principle of interference to calculate highly accurate depth information.

[0027] The combination of the two imaging methods utilizes the speed advantage of laser triangulation imaging in contour extraction and the precision advantage of grating interferometry imaging in depth details. Through complementary characteristics, the overall data quality is improved to form high-precision three-dimensional point cloud data. A single imaging method is difficult to meet the requirements of speed and accuracy at the same time. Synchronous triggering ensures the unity of the two data in time and space, avoids errors caused by time difference, improves the reliability of data acquisition, and the generated point cloud data can more realistically reflect the geometric characteristics of the circuit board surface.

[0028] Step 102: Application of the board surface micro-vibration sensor group During the data collection process, the board surface micro-vibration sensor group is used to monitor the circuit board's tiny vibrations in real time to reduce the impact of vibration on the three-dimensional point cloud data.

[0029] The board surface micro-vibration sensing group consists of multiple highly sensitive accelerometers located at key locations on the inspection platform. These sensors record the vibration acceleration data of the circuit board along three spatial axes and store them as a time series. The effect of vibration on imaging manifests as dynamic offset in the point cloud data. To eliminate this effect, the vibration acceleration data is accumulated over time and converted into displacement. Specifically, the acceleration values ​​are integrated twice along the time axis to obtain the displacement components corresponding to each time point. These displacement components are then applied as correction factors to the 3D point cloud data. Coordinate adjustments are performed to eliminate the offset caused by vibration, generating vibration-corrected point cloud data. Ultra-fine pitch pad inspection requires extremely stable point cloud data. Small vibrations can cause blurred boundaries or height errors. Vibration correction significantly improves the spatial consistency of the data. The corrected point cloud data more accurately reflects the pad's true position and height, avoiding misjudgments caused by vibration interference.

[0030] Step 103: Primary distortion correction Distortion correction is performed on the collected raw 3D point cloud data to eliminate the effects of optical system distortion on data accuracy. Optical system distortion primarily stems from non-ideal characteristics of the lens or projection device, which can cause deviations in the plane coordinates of the point cloud data. To perform correction, the coordinates of the point cloud data are adjusted using a polynomial transformation method based on a pre-calibrated distortion model.

[0031] Specifically, the distortion model includes a radial distortion coefficient, which is used to describe the nonlinear relationship between coordinate deviation and distance from the lens center. During the correction process, the plane coordinates of the original point cloud are input into the distortion model to calculate the corrected plane coordinates, while the height value of the point cloud remains unchanged, thereby obtaining the corrected three-dimensional point cloud data. Optical distortion is an inevitable source of error in the imaging system. If not corrected, the point cloud data will deviate from the true geometric shape of the circuit board, affecting the accuracy of subsequent analysis. The corrected point cloud data has higher geometric accuracy and can truly reflect the boundaries and positions of the pads, providing a reliable coordinate reference for subsequent warpage compensation and defect detection.

[0032] Step 104: Marking the pose matrix Based on the corrected 3D point cloud data, step one involves calculating and labeling a pose matrix for each circuit board, describing its position and orientation in 3D space. A pose matrix is ​​a homogeneous transformation matrix consisting of a rotation matrix and a translation vector. The rotation matrix is ​​constructed by fitting a reference plane to the point cloud data, calculating the plane's normal vector, and determining the circuit board's rotation angle based on the relationship between the normal vector and the reference coordinate system.

[0033] The translation vector is determined by calculating the positional offset in three-dimensional space using the geometric center of the point cloud data as a reference point. Ultimately, the rotation matrix and translation vector are combined into a complete pose matrix, providing a unique spatial description for each circuit board. During the inspection process, circuit boards may experience positional or angular variations, directly impacting the consistency of subsequent geometric analysis. The pose matrix quantifies these variations into calculable parameters. After labeling the pose matrix, the point cloud data can be adjusted based on a unified spatial reference, ensuring geometric consistency between different circuit boards and improving the stability and repeatability of the inspection process.

[0034] When used, step one generates high-precision three-dimensional point cloud data that has been vibration-corrected and distortion-corrected, and marks the posture matrix for each circuit board. This can effectively eliminate the impact of optical distortion and vibration interference on data quality, allowing the point cloud data to truly reflect the geometric features of the circuit board surface. At the same time, the standardization of spatial description is achieved through the marking of the posture matrix, which significantly improves the accuracy and stability of data acquisition in ultra-fine pitch pad detection, and provides reliable data support for subsequent warping surface fitting and defect detection.

[0035] Step 2: Use the three-dimensional point cloud data to fit the global warping surface in real time, and solve the quadratic surface model parameters by the least squares method to describe the overall warping deformation trend of the circuit board; construct a local adaptive reference surface for each pad area, wherein: select the reference point in the non-pad area around the pad, calculate the Euclidean distance from the reference point to the center of the pad and determine the weight based on the distance, use the weighted average method to calculate the local reference height, and define the adaptive reference surface as a horizontal plane with a height equal to the local reference height; perform local height remapping on the three-dimensional point cloud data points in the pad area, calculate the relative height relative to the adaptive reference surface to eliminate the deviation caused by warping; organize the remapped relative height data into a height matrix according to the grid, where the grid is based on the physical size of the pad area, the rows and columns of the height matrix correspond to the horizontal and vertical coordinate points, and output the height matrix in a unified format.

[0036] The second step includes the following: Step 201: Real-time fitting of warping surface parameters First, the three-dimensional point cloud data generated in step 1 is used to fit the global warpage surface in real time through the least squares method to describe the overall warpage deformation trend of the circuit board.

[0037] The global warping surface is represented by a quadratic surface model, which is a binary function consisting of quadratic, cross, and linear terms. The goal of the fitting process is to minimize the sum of the perpendicular distances from the 3D point cloud data points to the global warping surface. Specifically, regression analysis is used to determine the parameters of the global warping surface, minimizing the fitting error between the surface and the 3D point cloud data. This solution is based on the principle of least squares, which involves adjusting the parameters to minimize the sum of the squares of the perpendicular distances from all data points to the surface.

[0038] PCB warpage typically manifests as smooth surface changes. Quadratic surface models effectively capture this continuous deformation trend. Real-time fitting ensures the timeliness of parameter calculations, adapting to the dynamic demands of high-speed production lines. The global warpage surface provides a comprehensive reference for the subsequent construction of local adaptive reference surfaces, ensuring the consistency and accuracy of the warpage compensation process.

[0039] Step 202: Construction of adaptive reference surface Based on fitting the global warping surface, step 2 constructs a local adaptive reference surface for each pad area.

[0040] The construction process is as follows: First, in each pad area, a set of reference points located in the non-pad area around the pad is selected, and the height data of these reference points is used as the basis for the local benchmark height; then, the Euclidean distance from each reference point to the center of the pad is calculated, and the weight is determined according to the inverse of the distance. The closer the distance, the greater the weight of the reference point. To avoid division by zero, a small positive number is added to the inverse calculation; finally, the weighted average method is used to calculate the local benchmark height, that is, the height values ​​of all reference points are multiplied by the corresponding weights and then divided by the sum of all weights. The result is defined as the local benchmark height, and the adaptive benchmark plane is defined as a horizontal plane with a height equal to this local benchmark height.

[0041] Local warpage within a pad area may differ from the global warpage trend. Using a set of local reference points can more accurately reflect the actual height baseline around the pad. A weighted average method improves calculation stability, preventing fluctuations in data from a single reference point from interfering with the results. The adaptive reference surface provides a customized height reference for each pad area, improving the accuracy of subsequent height data processing.

[0042] Step 203: Local height remapping Perform local height remapping on the 3D point cloud data points within each pad area.

[0043] The specific method is: subtract the height value of the adaptive reference plane of the corresponding pad area from the height value of each three-dimensional point cloud data point to obtain the relative height relative to the adaptive reference plane. This can eliminate the influence of circuit board warping on the height data, make the height data of each pad area use the adaptive reference plane as the reference standard, and realize the standardized processing of height data.

[0044] PCB warpage can cause pad height data to deviate from its true value. Using uncorrected height data directly can reduce defect detection accuracy. Local height remapping effectively eliminates this warpage-induced deviation, ensuring consistent and comparable height data. The remapped relative height data truly reflects the actual pad stack height, providing reliable input for subsequent defect detection.

[0045] Step 203: Generation of height matrix The relative height data of each pad area after local height remapping is organized into a height matrix according to grid division.

[0046] The grid is based on the physical dimensions of the pad area. The rows and columns of the matrix correspond to discrete horizontal and vertical coordinate points, respectively. Each element in the matrix represents the relative height value of the corresponding grid position. The height matrix, as output data in a unified format, provides a standardized input format for subsequent defect detection. This structured height matrix facilitates efficient data processing and analysis and directly adapts to the data input requirements of machine vision algorithms. The generation of the height matrix ensures data organization consistency and traceability, improving the efficiency and accuracy of data processing during defect detection.

[0047] When used, step two systematically eliminates the effects of micro-warpage on the PCB, and the output height matrix has a unified dimension and standardized format. This processing method significantly improves the accuracy and reliability of height data in ultra-fine pitch pad inspection, providing solid data support for subsequent defect detection.

[0048] Step 3: Based on the height matrix, the height gradient within each pad area is calculated and the warpage composite slope index is obtained. The complexity of the height variation caused by substrate warpage within the pad area is quantified by combining the local slope and curvature. The texture features are extracted using the height matrix, and the texture contrast is calculated using the gray-level co-occurrence matrix method to obtain the glare texture robustness index, which is used to evaluate texture stability. The warpage composite slope index and the glare texture robustness index are input into the pre-trained neighborhood separation model, and the neighborhood separation coefficient is output to quantify the degree of contour aliasing between adjacent pads. The confidence threshold of the boundary-enhanced convolutional network is adjusted according to the neighborhood separation coefficient, and the confidence threshold is dynamically determined by the degree of contour aliasing to optimize the segmentation strategy; the adjusted confidence threshold is applied to the boundary-enhanced convolutional network, and pad contour segmentation is performed based on the height matrix to generate the true contour masks of adjacent pads.

[0049] The step three includes the following: Step 301: Calculation of warpage composite slope index Based on the height matrix generated in step 2, calculate the height gradient within each pad area. The height gradient represents the rate of change of the height matrix in the horizontal and vertical directions, which is used to reflect the local slope characteristics of the height data. Based on the height gradient, calculate the warpage composite slope index. The specific method is as follows: First, the height gradient amplitude is calculated at each grid point. The local slope is obtained by adding the squares of the horizontal and vertical partial derivatives and taking the square root. Simultaneously, the height Laplacian operator is calculated, and the change in local curvature is characterized by the sum of the horizontal and vertical second-order partial derivatives. The absolute value of the Laplacian operator is then normalized to the range of 0 to 1 and multiplied by the height gradient amplitude to generate a composite value that combines the slope and curvature. Finally, the composite value is averaged across all grid points within the pad area to obtain the warpage composite slope index. The warpage composite slope index, by combining slope and curvature information, quantifies the complexity of height variations within the pad area due to substrate warpage. This index effectively identifies areas with a high risk of contour aliasing and provides key input features for the neighborhood separation model, improving its ability to identify contour aliasing.

[0050] Step 302: Calculation of glare texture robustness index The texture features are extracted using the height matrix, and the texture contrast is calculated using the gray-level co-occurrence matrix method. Texture contrast is used to measure the degree of texture difference within the pad area of ​​the height matrix. Based on the texture contrast, the glare texture robustness index is calculated. The specific method is as follows: First, the mean and standard deviation of the texture contrast of all pad areas in the training set are precalculated. Then, the texture contrast of the current pad area is compared with the mean and standard deviation of the training set. Using the logistic function, the texture contrast is converted into a robustness index between 0 and 1, where values ​​closer to 1 indicate more stable texture contrast and less susceptible to interference from strong reflected glare. Using strong reflected glare can lead to instability in texture features, which in turn affects the accuracy of contour segmentation. The glare texture robustness index quantifies texture stability and provides auxiliary information for the neighborhood separation model. It enhances the system's robustness to glare interference and improves the reliability of contour segmentation.

[0051] Step 303: Application of neighborhood separation model The warping composite slope index and glare texture robustness index are fed into a pre-trained neighborhood separation model. The neighborhood separation model uses a neural network structure and outputs a neighborhood separation coefficient.

[0052] The neighborhood separation coefficient indicates the degree of contour aliasing between adjacent pads, ranging from 0 to 1, with smaller values ​​indicating greater aliasing and larger values ​​indicating greater contour differentiation. A neural network model learns the complex relationship between the warping composite slope index and the glare texture robustness index and the degree of contour aliasing, and outputs a quantized separation coefficient, providing a basis for subsequent segmentation threshold adjustment. The application of the neighborhood separation model improves the system's adaptability to varying degrees of aliasing, ensuring flexibility and accuracy in the contour segmentation process.

[0053] Step 304: Confidence threshold adjustment of boundary enhancement convolutional network The confidence threshold of the boundary-enhanced convolutional network is adjusted in real time according to the neighborhood separation coefficient.

[0054] The specific method is as follows: First, a basic confidence threshold is preset as the initial reference for segmentation; then, an adjustment factor is calculated based on the neighborhood separation coefficient, and the adjustment factor is proportional to the severity of contour aliasing; then, the basic confidence threshold is multiplied by the adjustment factor to obtain the final confidence threshold. When the neighborhood separation coefficient is small, it indicates severe aliasing, in which case the confidence threshold is increased to improve the conservatism of the segmentation; when the neighborhood separation coefficient is large, it indicates mild aliasing, in which case the confidence threshold is decreased to enhance the sensitivity of the segmentation. Different segmentation strategies are required for different degrees of aliasing, and dynamic adjustment of the confidence threshold can optimize the segmentation effect according to the actual situation. Real-time adjustment of the confidence threshold improves the adaptability and accuracy of contour segmentation, ensuring that pad boundaries can be accurately identified even under complex conditions.

[0055] Step 305: Segmentation of adjacent pad true contours The adjusted confidence threshold is applied to a boundary-enhanced convolutional network (BAN), which performs pad contour segmentation based on the height matrix and generates a true contour mask of adjacent pads. The contour mask is a binary image, where regions with a value of 1 represent pad boundaries and regions with a value of 0 represent non-boundaries. It is used to precisely mark the pad's contour. By learning the features of the height matrix, the BAN effectively extracts pad boundary information. Combined with a dynamically adjusted confidence threshold, the segmentation accuracy is further enhanced. The generated contour mask provides precise boundary information for subsequent defect detection, significantly improving the accuracy and reliability of solder paste printing quality inspection. This achieves precise segmentation of the true contours of adjacent pads, effectively resolving contour aliasing issues caused by substrate warpage and strong reflected glare in ultra-fine pitch conditions.

[0056] Step 4: After comparing the obtained defect confidence with the batch history threshold, dynamically adjust the imaging exposure time and projection frequency. Adjust the imaging exposure time and projection frequency through defect confidence deviation calculation to optimize the imaging quality. At the same time, mark high-uncertainty primitives and store them in the incremental training cache. Identify the pad areas with defect confidence close to the batch history threshold and record relevant data to provide real-world scene samples for subsequent model iterations.

[0057] The step 4 includes the following contents: Step 401: Compare defect determination with batch history thresholds First, the defects of the pad area are determined, and the determination results are compared with the batch history threshold. The specific method of defect determination is as follows: Based on the outline mask and height matrix, the defect type and defect confidence level are determined by calculating the pad's volume and shape characteristics. Defect types include insufficient solder or bridging solder. The defect confidence level is a value between 0 and 1, indicating the confidence level of the determination result. The batch history threshold is the median defect confidence level of the same defect type in historical batches, reflecting the baseline level under normal production conditions.

[0058] The difference between the current defect confidence and the batch history threshold is calculated to obtain the confidence deviation. If the confidence deviation is positive, the current defect confidence is higher than the batch history threshold, indicating that there may be imaging parameter mismatch, which is considered an abnormal situation. If the confidence deviation is negative or zero, it is considered normal.

[0059] By comparing with historical data, it is possible to promptly determine whether imaging parameters are suitable for current production conditions, ensuring the reliability of defect determination. This comparison method can effectively identify abnormal fluctuations in imaging quality, avoid misjudgments caused by imaging problems, and improve the accuracy of inspection results.

[0060] Step 402: Dynamically update imaging exposure time and projection frequency The imaging exposure time and projection frequency are adjusted according to the confidence deviation; the imaging exposure time is used to control the duration of light intensity acquisition, and the projection frequency is used to control the projection rate of the grating or laser.

[0061] The specific adjustment method is: when the confidence deviation is positive, the imaging exposure time is shortened to prevent overexposure, while the projection frequency is increased to improve detail capture. When the confidence deviation is negative or zero, the imaging exposure time and projection frequency are maintained unchanged. The adjustment range is proportional to the magnitude of the confidence deviation and is precisely controlled by a preset adjustment factor. Dynamic adjustment of imaging parameters optimizes data acquisition quality based on actual production conditions, ensuring that imaging data maintains high fidelity in different environments. This adaptive adjustment mechanism enhances reliability and reduces image quality degradation caused by environmental changes or batch differences, thereby improving the accuracy and stability of defect detection.

[0062] Step 403: Marking of high uncertainty primitives Identify and mark high-uncertainty primitives; a high-uncertainty primitive refers to a pad area where the absolute value of the difference between the defect confidence and the batch history threshold is less than a preset threshold, for example, the preset threshold can be set to 0.05.

[0063] These primitives represent areas of high uncertainty in defect determination and often contain important boundary information. Data such as the height matrix, contour mask, defect type, and defect confidence level of these highly uncertain primitives are recorded. Highly uncertain primitives reflect the complex scenarios of the production process. By labeling and recording this data, challenging real-world samples can be provided for subsequent model optimization. This labeling mechanism preserves critical information, supporting improved model performance under complex conditions, thereby enhancing the adaptability and reliability of defect detection.

[0064] Step 404: Accumulation of incremental training cache The data of highly uncertain primitives are stored in the incremental training cache, and this data is periodically used to incrementally train the neighborhood separation model and the boundary-enhanced convolutional network.

[0065] Incremental training uses real-world scene samples stored in the incremental training cache to fine-tune model parameters, improving the model's adaptability to complex ultra-fine pitch scenes. Incremental training enables the model to continuously learn new patterns and changes in the production process, maintaining its timeliness and accuracy. This continuous learning mechanism enhances the system's adaptability and ensures stable detection accuracy over long-term use.

[0066] Step 5: Evaluate the effectiveness of the measurement weights by calculating the F1-score, and perform operations based on the comparison between the F1-score and the preset threshold. If the F1-score is greater than the preset threshold, write the measurement weights to the shared model library; if the F1-score is less than or equal to the preset threshold, retrieve the historical optimal measurement weights from the shared model library. At the same time, associate the dynamically adjusted imaging exposure time and projection frequency with the production batch number, defect type, and defect confidence level and store them in the shared model library. Use high-uncertainty primitives for incremental learning to update the model parameters in the shared model library. Push the measurement weights, parameter adjustment records, and model parameters in the shared model library to the printing and quality inspection equipment via the network communication protocol. The step five includes the following: Step 501: Verification of measurement weights First, the measurement weights are validated to assess their effectiveness. The validation process is based on the precision and recall of the detection results, and the harmonic mean of the two is calculated to comprehensively evaluate the performance.

[0067] Precision is the ratio of the number of correctly detected defects to the total number of detected defects and is used to measure detection accuracy. Recall is the ratio of the number of correctly detected defects to the actual number of defects and is used to measure detection completeness. The harmonic mean is calculated by first taking the sum of the reciprocals of the precision and recall, and then taking the reciprocal of their average. The result is called the F1-score. The F1-score is compared with a preset threshold (for example, 0.9). If the F1-score reaches or exceeds the threshold, the measurement weight is considered valid; if it falls below the threshold, the measurement weight is considered invalid, and the subsequent update process is initiated. The F1-score is a widely recognized classification performance evaluation metric in the machine learning field. It objectively measures the balance between accuracy and completeness of the detection model and ensures the reliability of the measurement weight. This validation process can promptly identify inappropriate measurement weights, thereby ensuring the stability and accuracy of detection results.

[0068] Step 502: Processing of parameter adjustment records Record the dynamically adjusted imaging exposure time and projection frequency, and store these parameters in association with metadata such as production batch number, defect type, and defect confidence level; Associative storage is a process that binds each set of adjusted parameters to the corresponding production conditions and test results, forming a traceable data unit. Production environments and material properties may vary from batch to batch. By associatively storing parameter adjustment records, the correspondence between each set of parameters and specific production scenarios can be preserved, providing a basis for subsequent analysis and optimization. By accumulating this associated data, optimal parameter patterns for different production conditions can be identified, thereby improving the adaptability and overall robustness of the testing process to diverse production scenarios.

[0069] Step 503: Building a shared model library Build a centralized shared model library to store verified measurement weights, parameter adjustment records, and optimized model parameters. The shared model library categorizes and stores data according to production batch number and defect type, supporting on-demand query and reuse. When the F1-score of the measurement weight reaches the preset threshold, the current measurement weight is written to the shared model library; if the F1-score does not reach the preset threshold, the best measurement weight in history is retrieved from the shared model library for replacement. As a centralized management platform, the shared model library can systematically save parameters and models that have been verified to be effective in the production process, promoting the accumulation and reuse of knowledge. The shared model library enables the system to quickly call parameters between different shifts and equipment, reducing the time cost of repeated adjustments, thereby improving production efficiency.

[0070] Step 504: Model update and reuse The pad areas (i.e., high uncertainty primitives) with detection confidence close to the historical threshold are used as training samples, and the model parameters in the shared model library are regularly updated through an incremental learning algorithm.

[0071] The incremental learning algorithm is implemented by gradually adjusting model weights based on new samples. For example, this involves calculating the gradient of the loss function and updating the parameters using stochastic gradient descent. The update frequency is set based on production needs, such as daily or weekly, to ensure that the model parameters are aligned with the current production environment. Highly uncertain primitives often contain boundary information from the production process. Leveraging these samples for incremental learning can enhance the model's adaptability to complex scenarios. Regularly updating model parameters allows for continuous optimization of detection performance and long-term maintenance of high-precision detection capabilities.

[0072] Step 505: Push and Synchronize The latest measurement weights, parameter adjustment records, and updated model parameters in the shared model library are pushed to printing and quality inspection equipment via network communication protocols (such as MQTT or HTTP). After receiving the pushed data, the equipment automatically loads these latest parameters and models to ensure that the inspection strategy is consistent with the shared model library. The specific process of push and synchronization involves regularly checking the update status of the shared model library and transmitting the changes in real time to all relevant equipment. Parameter and model synchronization between devices ensures consistent inspection standards across the entire production line, avoiding deviations in results due to equipment differences. This real-time synchronization mechanism enables the system to quickly adapt to changes in the production environment, thereby improving the stability and reliability of the inspection process.

[0073] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0074] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0075] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is only for some logical functions. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0076] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0077] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A solder paste defect detection method based on machine vision, characterized by: include, A synchronously triggered laser triangulation-grating interferometry dual-mode imaging module is used in conjunction with a board surface micro-vibration sensor group to collect a 3D point cloud that has undergone primary distortion correction. A posture matrix is ​​then marked for each circuit board to determine the accuracy baseline for geometric compensation. Based on 3D point cloud data and posture matrix, by real-time fitting of warping surface parameters, an adaptive reference surface is constructed in each pad area and local height remapping is performed to eliminate the coordinate deviation introduced by micro-warping and output the height matrix. After calculating the warping composite slope index and glare texture robustness index on the height matrix, the neighborhood separation coefficient is obtained by inputting the neighborhood separation model. The confidence threshold of the boundary enhancement convolutional network is adjusted in real time to perform the true contour segmentation of adjacent pads. Dynamically update the imaging exposure time and projection frequency based on the comparison of defect confidence and batch history thresholds, mark high-uncertainty primitives and store them in the incremental training cache; Verified measurement weights and parameter adjustment records are written into a shared model library and pushed to printing and quality inspection equipment to enable cross-shift model reuse.

2. The method for detecting solder paste defects based on machine vision according to claim 1, wherein: The synchronously triggered laser triangulation imaging and grating interferometry imaging modules collect data on the circuit board surface at the same time point to generate time-consistent 3D point cloud data; The board surface micro-vibration sensor group is used to monitor the vibration of the circuit board in real time. The time accumulation calculation is performed on the vibration acceleration data to obtain the displacement. The displacement is then applied as a correction factor to the three-dimensional point cloud data to generate vibration-corrected point cloud data.

3. The method for detecting solder paste defects based on machine vision according to claim 2, wherein: Based on the pre-calibrated distortion model, polynomial transformation is used to correct the distortion of the point cloud data after vibration correction. Based on the distortion-corrected point cloud data, the rotation matrix is ​​constructed by fitting the reference plane and calculating the normal vector. At the same time, the translation vector is calculated using the geometric center of the point cloud as the reference point, and the pose matrix is ​​combined.

4. The method for detecting solder paste defects based on machine vision according to claim 3, wherein: The global warpage surface is fitted in real time using 3D point cloud data. The quadratic surface model parameters are solved by the least squares method to describe the overall warpage deformation trend of the circuit board, and a local adaptive reference surface is constructed for each pad area.

5. The method for detecting solder paste defects based on machine vision according to claim 4, wherein: Select reference points in the non-pad area around the pad, calculate the Euclidean distance from the reference point to the center of the pad and determine the weight based on the distance, use the weighted average method to calculate the local reference height, and define the adaptive reference plane as a horizontal plane with a height equal to the local reference height; perform local height remapping on the three-dimensional point cloud data points in the pad area, and calculate the relative height relative to the adaptive reference plane to eliminate the deviation caused by warping.

6. The method for detecting solder paste defects based on machine vision according to claim 5, wherein: The remapped relative height data is organized into a height matrix according to a grid, where the grid is based on the physical size of the pad area, the rows and columns of the height matrix correspond to the horizontal and vertical coordinate points, and the height matrix is ​​output in a unified format.

7. The method for detecting solder paste defects based on machine vision according to claim 6, wherein: Based on the height matrix, the height gradient within each pad area is calculated and the warpage composite slope index is obtained. The complexity of the height variation caused by substrate warpage within the pad area is quantified by combining the local slope and curvature. The texture features are extracted using the height matrix, and the texture contrast is calculated using the gray-level co-occurrence matrix method to obtain the glare texture robustness index, which is used to evaluate texture stability. The warpage composite slope index and glare texture robustness index are input into the pre-trained neighborhood separation model, and the neighborhood separation coefficient is output to quantify the degree of contour aliasing between adjacent pads.

8. The method for detecting solder paste defects based on machine vision according to claim 7, wherein: After comparing the obtained defect confidence with the batch history threshold, the imaging exposure time and projection frequency are dynamically adjusted. The imaging exposure time and projection frequency are adjusted by calculating the defect confidence deviation to optimize the imaging quality. High-uncertainty primitives are marked and stored in the incremental training cache. Pad areas with defect confidence close to the batch history threshold are identified and relevant data is recorded to provide real-world scenario samples for subsequent model iterations.

9. The method for detecting solder paste defects based on machine vision according to claim 8, wherein: The effectiveness of the measurement weights is evaluated by calculating the F1-score and performing an action based on the comparison between the F1-score and a preset threshold. If the F1-score is greater than the preset threshold, the measurement weights are written to the shared model library. If the F1-score is less than or equal to the preset threshold, the historical optimal measurement weight is retrieved from the shared model library.

10. The method for detecting solder paste defects based on machine vision according to claim 9, wherein: The dynamically adjusted imaging exposure time and projection frequency are associated with the production batch number, defect type and defect confidence and stored in a shared model library; Incremental learning is performed using high-uncertainty primitives to update the model parameters in the shared model library. The measurement weights, parameter adjustment records and model parameters in the shared model library are pushed to the printing and quality inspection equipment through the network communication protocol.

Images