A PCB board defect detection method and system based on visual positioning

Abstract

The application discloses a PCB board defect detection method and system based on visual positioning, and particularly relates to the technical field of machine vision detection, which realizes real-time acquisition of PCB board images in transmission, identifies the non-linear warping and non-uniform deformation characteristics of the board surface, generates deformation field estimation and initial registration residual error, calls a non-linear optical flow model, iteratively calculates the registration residual error as a feedback until it is less than a preset tolerance threshold, and outputs the residual registration error; dynamically corrects the positioning parameters and generates a compensation amount, and outputs the corrected positioning parameters and the current compensated residual error; performs inter-board transmission propagation blocking and attenuation processing on the residual error, outputs a stable positioning state, performs defect determination, and outputs a detection result; the application identifies dynamic non-linear deformation, iteratively optimizes the optical flow with the registration residual error as the feedback, improves the registration accuracy, dynamically corrects the positioning parameters to make the error closed-loop converge, simultaneously blocks the inter-board error propagation and attenuates the residual error, and ensures the stability and reliability of batch detection.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection technology, and more specifically, to a method and system for detecting defects in PCB boards based on visual positioning. Background Technology

[0002] In the electronics manufacturing industry, PCBs (Printed Circuit Boards) often exhibit warping and deformation after undergoing high-temperature processes such as reflow soldering and wave soldering. This deformation typically manifests as localized random twisting, wavy edges, and asymmetric deformation due to uneven heating. The amplitude and distribution patterns of these deformations exhibit significant non-linear characteristics, and the deformation state changes in real-time under continuous production conditions, fluctuating with production line conveyor speeds. To ensure product quality, automated optical inspection is usually performed after the PCB soldering process to identify scratches, gaps, foreign objects, or solder joint abnormalities on the board surface.

[0003] Several vision-based methods for PCB defect detection have been developed in the current technology. These methods generally first align the acquired image of the board under test with a standard template image using image registration techniques, and then determine defects through pixel-by-pixel comparison or feature comparison. Common registration strategies include using marker points on the board for overall positioning, or using feature point matching to estimate the overall rigid transformation parameters of the board surface. In practical applications, when the deformation of the PCB board is relatively gentle and manifests as overall rotation, translation, or uniform scaling, these traditional registration methods can usually achieve satisfactory positioning accuracy, thus supporting subsequent defect identification.

[0004] As electronic products become smaller and denser, the number of PCB layers increases and the line width and spacing decrease, leading to increasingly complex deformations caused by high-temperature processes. In actual production lines, the deformation of many PCBs is no longer a simple rigid body transformation or low-order polynomial deformation, but rather exhibits nonlinear characteristics such as local random twisting, edge wavy deformation, and asymmetric thermal deformation. Even more challenging is that during high-speed transport, these nonlinear deformations are not constant but dynamically change with the movement of the conveyor belt, variations in board surface stress, and fluctuations in ambient temperature. When traditional registration methods face this dynamic nonlinear deformation, they often struggle to achieve sub-pixel-level positioning accuracy in one or a few alignment operations, resulting in large residuals after registration. This can easily misjudge normal deformation differences as defects or mask genuine, minute defects within the registration error.

[0005] Furthermore, the deformation states of consecutive PCBs in a continuous production line exhibit a certain time correlation. If the registration error of the preceding board is not effectively suppressed, it can easily be transmitted to the positioning parameters of subsequent boards through the accumulation process of the control system, resulting in a collective detection deviation. Therefore, how to achieve stable positioning of PCBs with nonlinear deformation under high dynamic conditions has become a core problem that urgently needs to be solved in the field of visual positioning and inspection. Summary of the Invention

[0006] To achieve the above objectives, the present invention provides the following technical solution: A PCB board defect detection method based on vision localization includes the following steps: Real-time acquisition of PCB board images during transmission, identification of nonlinear warping and non-uniform deformation features of PCB board under dynamic conditions, including at least local random twisting of the board surface, edge wavy deformation and asymmetric thermal deformation. Based on the identified features, a deformation field estimate of the current plate is generated, and the initial registration residual of the current plate is calculated. Based on the deformation field estimation and the initial registration residual, a preset nonlinear optical flow model is called, and the registration residual is used as a feedback control quantity for iterative calculation until the registration residual is less than the preset registration error tolerance threshold, and the residual registration error is output. Based on the residual registration error, the positioning parameters are dynamically corrected and a compensation amount is generated. The corrected positioning parameters and the residual error after the current compensation are output. The residual error after current compensation is processed to block and attenuate inter-board propagation, and the stable positioning state after error suppression is output. Based on the stable positioning state, the PCB board is judged for defects, and the final detection result is output.

[0007] In a preferred embodiment, identifying the nonlinear warping and non-uniform deformation characteristics of a PCB board under dynamic conditions specifically includes the following steps: Gaussian filtering is applied to the acquired current board image to remove noise, and then histogram equalization is performed to enhance contrast, resulting in an enhanced image. Scale-invariant feature transformation points are extracted from the enhanced image and coarsely matched with corresponding feature points in the standard template image to obtain initial affine transformation parameters. Based on the initial affine transformation, the pixel offset of the local region of the image is fitted using a thin plate spline interpolation function to obtain the first offset field describing the local random distortion of the plate surface; Second derivative analysis was performed on the edge contour of the enhanced image to extract the amplitude and frequency parameters of the edge wave deformation. The following method is used to identify gray-level gradient anomaly regions caused by asymmetric thermal deformation: The enhanced image is divided into multiple non-overlapping local windows. The gray-level gradient magnitude and principal direction of the pixels in each window are calculated, and the mean and standard deviation of the gray-level gradient magnitude of the entire plate are calculated. Windows with gray-level gradient magnitudes higher than three times the mean standard deviation are marked as abnormal windows. Then, adjacent abnormal windows are merged into connected components. Connected components with an area greater than a preset area threshold after merging are determined as gray-level gradient anomaly regions. The location and contour parameters of the abnormal region are output.

[0008] In a preferred embodiment, a deformation field estimate of the current plate is generated based on the identified features, and the initial registration residual of the current plate is calculated. The implementation process is as follows: The first offset field, the amplitude and frequency parameters of edge wave deformation, and the position and contour parameters of the asymmetric thermal deformation anomaly region are fused to construct a nonlinear deformation field estimate covering the pixel coordinates of the entire board. Then, the nonlinear deformation field estimation is applied to the standard template image to generate a registered estimated image. The gray values ​​of the enhanced image and the estimated image at the same pixel position are subtracted to obtain an initial registration residual image. The grayscale differences of all pixels in the initial registration residual image are summed by squares, divided by the total number of pixels, and the square root is taken to calculate the root mean square error. Finally, the root mean square error is used as the output of the initial registration residual.

[0009] In a preferred embodiment, the preset nonlinear optical flow model is specifically as follows: The gray value of any pixel in the current enhanced image is represented as a function of the coordinates of that point. The gray value of the corresponding coordinate point in the template image after deformation compensation is represented as another function. The cube of the difference between the two is calculated and summed as the specific value of the gray nonlinear constraint term. Calculate the rate of change of the deformation field in the horizontal and vertical directions, integrate the sum of the squares of the rates of change in the two directions, and then multiply by a preset regularization coefficient to obtain the specific value of the spatial smoothing constraint term. Take the registration residual from the previous iteration and multiply it by the preset feedback gain coefficient to obtain the feedback driving force value; The gray-scale nonlinear constraint term, the spatial smoothing constraint term, and the feedback driving force value are added together according to a preset weight ratio to form a scalar energy function. Taking the variational derivative of the deformation field variable in the scalar energy function and setting the derivative to zero yields a system of linear equations about the deformation field increment. Solving this system of equations yields the optical flow field of the current iteration step.

[0010] In a preferred embodiment, iterative calculation specifically includes the following steps: The enhanced image and the initial registration residual are used as inputs, and the nonlinear deformation field estimate is used as the initial value of the optical flow field. Using the current optical flow field and the current registration residual, the linear equation system is solved to obtain the updated optical flow field. The updated optical flow field is then used to re-register the standard template image to generate a new registration estimation image. This image is then compared pixel by pixel with the enhanced image to obtain the registration residual after this iteration. The registration residual after this iteration is compared with the preset registration error tolerance threshold. If the registration residual is greater than or equal to the registration error tolerance threshold, the residual is used as a feedback control variable, and the process returns to the previous step to continue iterating. If the registration residual is less than the registration error tolerance threshold, the iteration stops and the registration residual is output as the residual registration error.

[0011] In a preferred embodiment, the positioning parameters are dynamically corrected and a compensation amount is generated based on the residual registration error, specifically including the following steps: Multiplying the residual registration error by the first correction factor yields the proportional correction component; multiplying the cumulative sum of the residual registration errors by the second correction factor yields the integral correction component; and multiplying the rate of change of the residual registration error by the third correction factor yields the differential correction component. The proportional correction component, integral correction component, and differential correction component are added together to obtain the correction amount of the positioning parameters. The correction amount is converted into the motion step size or angle increment of the actuator as the compensation amount.

[0012] The output includes the corrected positioning parameters and the residual error after the current compensation. The specific implementation process is as follows: Add the current positioning parameters to the compensation amount to obtain the corrected positioning parameters; The corrected positioning parameters are sent to the production line control system to adjust the image acquisition timing of subsequent PCB boards; Subtract the compensation amount from the residual registration error and multiply it by the preset correction efficiency factor to obtain the residual error after the current compensation. The absolute value of the residual difference after the current compensation is compared with the preset allowable deviation range: if it exceeds the allowable range, the process returns to the dynamic correction step to continue correction; if it is within the allowable range, the current compensation process is terminated. Finally, the corrected positioning parameters and the residual error after the current compensation are output.

[0013] In a preferred embodiment, the stable positioning state after error suppression is output specifically includes the following steps: Obtain the residual error after the current compensation, and use this residual error as the historical error record of the current board; Set a propagation blocking switch so that when the compensation process of the current board is completed, the historical error record is immediately cleared to zero, preventing it from being transmitted to the next board. For the residual error sequence generated during the multi-level compensation iteration process within the current board, the latest residual error value is multiplied by a preset attenuation factor to obtain the residual error after attenuation processing. The attenuation factor is a positive number that is greater than zero and less than one. The residual error after attenuation is regarded as a stable residual error and is superimposed and compensated with the current deformation field estimate of the plate to obtain the corrected deformation field. The corrected deformation field is output as the stable positioning state of the current plate.

[0014] In a preferred embodiment, the process of performing defect determination on the PCB board based on the stable positioning state is as follows: Using the corrected deformation field in the stable positioning state, the deformation field is applied to the standard template image to generate the final registration estimation image; The final registered and estimated image is compared pixel by pixel with the real-time acquired and preprocessed enhanced image, and the absolute value of the gray level difference of each pixel is calculated. The absolute value of the grayscale difference of each pixel is compared with a preset defect discrimination threshold. The defect discrimination threshold is used to distinguish between normal grayscale fluctuations and true defects. If the absolute value of the grayscale difference of a pixel is greater than the defect discrimination threshold, the pixel is marked as a candidate defect point. Connected components of adjacent candidate defect points are merged. Connected components with an area smaller than a preset area threshold are filtered out as noise. The remaining connected components are determined as real defect regions, and the position and size of each real defect region are recorded. Output the defect determination result, including whether there is no defect and an alarm is triggered, along with the location information of all actual defective areas.

[0015] In a preferred embodiment, a PCB board defect detection system based on vision positioning includes: The feature recognition module is used to acquire PCB board images in real time and identify nonlinear warping and non-uniform deformation features of the PCB board under dynamic conditions. The features include at least local random twisting of the board surface, edge wave deformation and asymmetric thermal deformation. The deformation estimation module is used to generate a deformation field estimate of the current plate based on the identified features and to calculate the initial registration residual of the current plate. The optical flow iteration module is used to call a preset nonlinear optical flow model based on the deformation field estimation and the initial registration residual, and use the registration residual as a feedback control quantity for iterative calculation until the registration residual is less than the preset registration error tolerance threshold, and output the residual registration error. The positioning compensation module is used to dynamically correct the positioning parameters based on the residual registration error and generate the compensation amount, and output the corrected positioning parameters and the residual error after the current compensation. The detection output module is used to perform inter-board propagation blocking and attenuation processing on the residual error after the current compensation, output the stable positioning state after error suppression, perform defect judgment on the PCB board based on the stable positioning state, and finally output the detection result.

[0016] The technical effects and advantages of this invention are as follows: This invention acquires real-time images of a PCB board during transmission, identifies nonlinear warping and non-uniform deformation features, including local random distortion of the board surface, edge wavy deformation, and asymmetric thermal deformation, and generates a deformation field estimate and initial registration residual based on the identified features. Then, a preset nonlinear optical flow model is invoked, and iterative calculations are performed using the registration residual as feedback control until the residual is less than the registration error tolerance threshold. This mechanism differs from the single approximation of global deformation in existing technologies, accurately capturing local nonlinear deformation under high dynamic conditions, significantly improving registration accuracy, and providing a reliable positioning basis for subsequent defect determination.

[0017] This invention dynamically corrects positioning parameters and generates compensation amounts based on residual registration errors, outputting the corrected positioning parameters and the residual error after the current compensation. This closed-loop feedback structure enables the system to proactively adjust the positioning reference based on the registration residuals of the previous board or previous iteration, rather than passively accepting fixed parameters. By feeding back the compensated residual error to the positioning stage in real time, the positioning error is converged step by step, effectively solving the problem in traditional methods where fixed parameters lead to an increase in residual error after multiple iterations. This improves the adaptability and robustness of the detection system in dynamic production lines.

[0018] This invention performs inter-board propagation blocking and attenuation processing on the residual error after current compensation, outputting a stable positioning state after error suppression, and finally completing the defect determination. Through the inter-board propagation blocking mechanism, the residual error of the current board is immediately cleared to zero after the compensation process ends, avoiding the direct transmission of errors from the previous board to the subsequent board; simultaneously, attenuation processing further suppresses the residual error sequence generated during multi-level compensation iterations within the board. This design cuts off the chain propagation path of collective deviations, making the detection of each PCB board completely independent and stable, thereby maintaining consistent detection accuracy in large-scale continuous production and reducing the risk of batch false alarms or missed alarms. Attached Figure Description

[0019] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 This is a schematic diagram of a PCB board defect detection method based on vision positioning in this invention.

[0020] Figure 2 This is a schematic diagram of a PCB board defect detection system based on vision positioning according to the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Reference Figure 1 - Figure 2 The following examples were obtained: Example 1: A PCB board defect detection method based on vision positioning, comprising the following steps: Real-time acquisition of PCB board images during transmission, identification of nonlinear warping and non-uniform deformation features of PCB board under dynamic conditions, including at least local random twisting of the board surface, edge wavy deformation and asymmetric thermal deformation. Based on the identified features, a deformation field estimate of the current plate is generated, and the initial registration residual of the current plate is calculated. Based on the deformation field estimation and the initial registration residual, a preset nonlinear optical flow model is called, and the registration residual is used as a feedback control quantity for iterative calculation until the registration residual is less than the preset registration error tolerance threshold, and the residual registration error is output. Based on the residual registration error, the positioning parameters are dynamically corrected and a compensation amount is generated. The corrected positioning parameters and the residual error after the current compensation are output. The residual error after current compensation is processed to block and attenuate inter-board propagation, and the stable positioning state after error suppression is output. Based on the stable positioning state, the PCB board is judged for defects, and the final detection result is output.

[0023] Identifying the nonlinear warping and non-uniform deformation characteristics of PCB boards under dynamic conditions includes the following steps: The acquired current board image is subjected to Gaussian filtering to remove noise, followed by histogram equalization to enhance contrast, resulting in an enhanced image. Gaussian filtering uses a 5x5 convolution kernel with a standard deviation of 1.5. This parameter effectively suppresses sensor noise while preserving edge details. For example, for a 100mm x 80mm PCB board with an image resolution of 20 million pixels, this filtering parameter can improve the signal-to-noise ratio by approximately 12 dB. Histogram equalization stretches the grayscale distribution across the entire dynamic range, preventing subsequent feature extraction from failing due to uneven lighting. In practice, the reflectivity difference between the green solder mask and copper foil on the board surface is significant; after equalization, the contrast can be improved by more than 30%.

[0024] Scale-invariant feature transformation points are extracted from the enhanced image and coarsely matched with corresponding feature points in the standard template image to obtain initial affine transformation parameters. The extraction threshold for scale-invariant feature transformation points is set to 0.04, meaning only candidate points with contrast higher than this value are retained, thus obtaining approximately 500 to 2000 stable feature points per image. Coarse matching can be performed using the Euclidean distance nearest neighbor to second nearest neighbor ratio method, with a ratio threshold set to 0.7, which can eliminate more than 80% of mismatched points. Based on the matched point pairs, the initial affine transformation matrix is ​​iteratively calculated using a random sampling consensus algorithm, with an upper limit of 2000 iterations and an error tolerance of three pixels, thereby obtaining the initial values ​​for rotation, translation, and scaling parameters.

[0025] Based on the initial affine transformation, a thin-plate spline interpolation function is used to fit the pixel offsets of local image regions, obtaining a first offset field describing the local random distortion of the board surface. The thin-plate spline interpolation function uses an embedded radial basis function as a framework, and the residual offset after coarse matching is used as a control point displacement constraint. The bending energy coefficient is set to 1.0, thus fitting local distortions while avoiding overfitting. For example, if a batch of PCBs experiences a 0.3 mm corner warpage due to reflow soldering temperature deviation, this fitting can generate a continuous and smooth offset field, accurately describing the gradient change from the board edge to the center.

[0026] Second-order derivative analysis is performed on the edge contours of the enhanced image to extract the amplitude and frequency parameters of the edge wavy deformation. The first-order gradient can be calculated using the Sobel operator, and then the second derivative is obtained by differentiating the gradient direction. The locations where the second derivative is zero at the edge points are marked as quasi-edges. The amplitude of the wavy deformation is taken as the maximum offset between adjacent zero-crossing points, and the frequency parameter is taken as the reciprocal of the number of wave crests per unit length. Typical amplitude ranges are 20 to 100 micrometers, and frequency ranges are 0.5 to 2 Hz. For example, for flexible PCB boards, an edge wavy amplitude exceeding 80 micrometers can cause subsequent component positioning failures; this method can accurately extract this value.

[0027] The following method is used to identify grayscale gradient anomalies caused by asymmetric thermal deformation: The enhanced image is divided into multiple non-overlapping local windows. The grayscale gradient magnitude and principal direction of pixels within each window are calculated, and the mean and standard deviation of the grayscale gradient magnitude across the entire board are statistically analyzed. Windows with grayscale gradient magnitudes exceeding three times the mean standard deviation are marked as anomalous windows. Adjacent anomalous windows are then merged into connected components. Connected components with an area greater than a preset area threshold after merging are identified as grayscale gradient anomaly regions. The location and contour parameters of these anomalous regions are output. The local window size is set based on the minimum feature size of the PCB board, typically 32 pixels wide by 32 pixels high, ensuring each window contains approximately 1,000 pixels sufficient to characterize the local texture. The three-standard-deviation threshold is derived from the Laida criterion in statistics; under a normal distribution, only 0.3% of windows are misclassified due to random noise.

[0028] The preset area threshold needs to be combined with the specific defect tolerance. For example, for consumer electronics PCBs, this threshold is set to 50 square millimeters. This means that only continuous anomalies with an area exceeding 50 square millimeters are considered thermal deformation areas, while isolated windows smaller than this value are discarded as noise. Connected region merging uses the eight-neighborhood connection rule. When two anomaly windows have adjacent corners, they are considered to be the same region. For example, if a PCB has a large area of ​​grayscale gradient anomaly near the power module, the merged connected region area is 120 square millimeters, exceeding the preset threshold of 50 square millimeters. Therefore, it is judged as a grayscale gradient anomaly region, and the output position is the coordinates of the boundary rectangle of this region, with the contour parameters being the sequence of boundary points.

[0029] Thin-plate spline interpolation is a mature existing technique in image registration. Its theoretical basis comes from the physical model of a thin, elastic plate producing minimum bending deformation under given control point constraints. This function has direct applications in medical image registration, remote sensing image geometric correction, and industrial visual inspection, and is a well-known mathematical tool in the field. This invention directly uses this existing technique to fit local pixel offsets. The standard form of the thin-plate spline interpolation function can be described as follows: Given a set of control points, each control point contains its coordinates in the original image and its corresponding coordinates in the target image. The thin-plate spline interpolation function calculates a smooth mapping from the original coordinate space to the target coordinate space. This mapping can accurately pass through all control points while minimizing the bending energy of the entire mapped surface. The mathematical expression for bending energy is the double integral of the sum of the squares of the second-order partial derivatives in two directions at all points on the surface over the entire image domain. From an algorithmic implementation perspective, the solution process of the thin-plate spline interpolation function consists of three steps.

[0030] The first step is to construct the radial basis function matrix between the control points. Let the total number of control points be denoted as the total number of control points, which is determined based on the PCB board size and the number of feature points, typically between one hundred and five hundred. For any two control points, calculate their Euclidean distance in the original image plane. Then, multiply the square of the distance by its natural logarithm as the value of the radial basis function. Mathematically, this radial basis function is written as the square of the distance multiplied by its natural logarithm. Arrange the radial basis function values ​​between all pairwise control points into a square matrix, where the number of rows and columns of the matrix are equal to the total number of control points.

[0031] The second step involves constructing the coordinate matrix and offset vector of the control points. The x-coordinate and y-coordinate of each control point in the original image are collected and combined with a full row and column matrix to form a coordinate expansion matrix with a dimension equal to three times the total number of control points. Simultaneously, the pixel offset at each control point is collected. This offset is divided into two components: horizontal offset and vertical offset, forming two independent vectors. The length of each vector is equal to the total number of control points.

[0032] The third step involves solving for the thin-plate spline interpolation coefficients. This is done by combining the radial basis function matrix with the coordinate extension matrix and its transpose to form a block-wise linear equation system. The unknowns in this system include the radial basis function weighting coefficients for each control point and the coefficients of the affine transformation. The affine transformation includes a constant term, an x-coordinate coefficient, and a y-coordinate coefficient. Solving this linear equation system yields the radial basis function weighting coefficients and the affine transformation coefficients. The linear equation system can be solved using Gaussian elimination or LU decomposition, both standard algorithms in numerical computation. On conventional computers, these methods can be completed within tens of milliseconds for systems with up to 500 control points.

[0033] After obtaining the above coefficients, for any pixel in the image whose offset needs to be calculated, the pixel offset is calculated as follows: First, calculate the Euclidean distance between the pixel and each control point. Then, multiply the square of this distance by the natural logarithm of the distance as the value of the radial basis function. Next, multiply the radial basis function weight coefficient corresponding to each control point by the above radial basis function value and sum them to obtain an intermediate value. Finally, add the constant term in the affine transformation part, the x-coordinate multiplied by the x-coordinate coefficient, and the y-coordinate multiplied by the y-coordinate coefficient, and sum them with the aforementioned intermediate value to obtain the offset value at that pixel. Perform the above calculation sequentially on all pixels in the entire image to obtain the horizontal and vertical offsets at each pixel. Organizing these offsets according to pixel positions forms a continuous and smooth two-dimensional offset field, which is the first offset field describing the local random distortion of the board surface.

[0034] In a specific implementation of this invention, the input data for the thin-plate spline interpolation function originates from the residual offset after the initial affine transformation. Specifically, after obtaining the initial affine transformation parameters, the standard template image is transformed according to these parameters to obtain an affine-registered image. Then, matching control point pairs are extracted between the affine-registered image and the enhanced image, and the coordinate difference between each control point pair is the residual offset. These residual offsets are used as constraints for the thin-plate spline interpolation, requiring the thin-plate spline surface to pass precisely through the residual offset at each control point. The first offset field obtained by this fitting only describes the local nonlinear deformation components that the affine transformation cannot correct, while global linear deformations such as rotation, translation, and scaling have already been processed by the initial affine transformation. This layered processing method is entirely based on the existing use of the thin-plate spline interpolation function in the prior art, namely, using it to perform local nonlinear fitting on the residual after global transformation, which is a conventional application scenario of this function.

[0035] From a numerical stability perspective, the solution process of the thin-plate spline interpolation function is numerically stable when the number of control points is large and their distribution is relatively uniform. However, if the number of control points exceeds one thousand, the time required to solve the linear equation system increases significantly. Therefore, in this invention, the number of control points is limited to less than five hundred by controlling the response threshold during feature point extraction to ensure real-time computation. The offset field generated by the thin-plate spline interpolation function has the characteristic of being second-order continuously differentiable. This means that the offset will not exhibit abrupt changes or discontinuities in space, which is appropriate for describing the continuous and smooth torsional deformation of a PCB board surface caused by thermal stress or internal material stress. For example, a PCB board may develop a bulge near the power supply area due to localized heating. The offset of its edge pixels gradually increases from zero to 0.5 mm and then gradually decreases back to zero. The thin-plate spline interpolation function can accurately fit this trend of offset change measured from the control points with a smooth surface, and in areas without control points, it provides a natural transition offset estimate based on the principle of minimizing bending energy, thereby generating a continuous deformation description covering the entire board. All calculation processes of the above-mentioned thin plate spline interpolation function belong to the mathematical methods disclosed in the prior art.

[0036] Based on the identified features, a deformation field estimate of the current plate is generated, and the initial registration residual of the current plate is calculated. The implementation process is as follows: The first offset field, the amplitude and frequency parameters of the edge wave deformation, and the position and contour parameters of the asymmetric thermal deformation anomaly region are fused to construct a nonlinear deformation field estimate covering the pixel coordinates of the entire board. The first offset field is obtained by fitting a thin plate spline interpolation function, which stores the horizontal and vertical offsets caused by local random distortion at each pixel coordinate position on the board surface. Taking an actual PCB board as an example, the value range of this offset is usually between -0.3 mm and +0.3 mm, depending on the severity of the distortion on the board surface. The amplitude and frequency parameters of the edge wave deformation are used to correct the deformation field in the board edge region. For example, when the amplitude parameter is 50 micrometers and the frequency parameter is 1.0 Hz, it indicates that there is a wave-like deformation with a period of about 1.0 mm and an amplitude of 50 micrometers at the board edge. At this time, a sinusoidal offset correction is superimposed on the corresponding pixel coordinates of the board edge, with a maximum offset of 50 micrometers at the peak and a negative 50 micrometers at the trough.

[0037] The location and contour parameters of the asymmetric thermal deformation anomaly region are used to apply an overall additional offset gradient within the region. For example, if the location of a region is an octagonal region in the upper left corner of the board, its contour parameters describe the boundary of the region. Thermal deformation causes the region to shift towards the center of the board, with the magnitude gradually increasing from zero millimeters at the edge of the region to 0.2 millimeters at the center of the region. During fusion, this additional offset is superimposed on every pixel point within the region.

[0038] The specific operation of fusion is as follows: First, a two-dimensional vector grid with the same size as the image resolution is generated. Each cell in the grid stores an offset vector with an initial value of zero millimeters horizontally and zero millimeters vertically. Then, the offset in the first offset field is directly filled into the cell with the corresponding coordinate. Next, according to the amplitude and frequency parameters of the edge wave deformation, the sinusoidal correction amount is superimposed on the original offset amount in the plate edge region. Finally, according to the position and contour parameters of the asymmetric thermal deformation anomaly region, the additional offset amount is superimposed on the original offset amount in the region according to the Gaussian distribution function.

[0039] The three offsets are superimposed using a simple linear addition method. That is, the final total offset at each pixel equals the offset of the first offset field plus the edge wave correction offset plus the thermal deformation additional offset. This yields the total offset at every pixel coordinate across the entire board. The total offsets of all pixels together constitute a two-dimensional vector field, which is called the nonlinear deformation field estimation. Taking a PCB board with a resolution of 50 million pixels as an example, this estimation requires storing 50 million horizontal offset values ​​and 50 million vertical offset values, occupying approximately 800 megabytes of memory, which can be processed in real time on current industrial control computers.

[0040] The nonlinear deformation field estimation is applied to the standard template image to generate a registered estimated image. The grayscale values ​​at the same pixel location are subtracted between the enhanced image and the estimated image to obtain an initial registration residual image. Specifically, each pixel in the standard template image is traversed, its original pixel coordinates are read, the total offset at that coordinate is extracted from the nonlinear deformation field estimation, the original coordinates are added to the total offset to obtain a new floating-point coordinate, the grayscale value at that floating-point coordinate is extracted from the standard template image using bilinear interpolation, and finally, this grayscale value is filled into the estimated image at the same position as the original coordinate.

[0041] For example, in the standard template image, the point located at pixel column coordinate 520 and row coordinate 380 has a total offset of 0.08 mm horizontally and -0.05 mm vertically. This translates to a pixel offset of approximately 10 pixels horizontally and -6 pixels vertically. Therefore, the actual sampling point's coordinates are column coordinate 530 and row coordinate 374. The grayscale value of this point can be extracted from the standard template image using bilinear interpolation, and then this grayscale value can be filled into the estimated image at column 520 and row 380.

[0042] After performing the above processing on all pixels, an estimated image after nonlinear deformation registration is obtained. Then, the enhanced image and the estimated image are subtracted pixel-by-pixel for grayscale values. That is, at each pixel coordinate, the grayscale value of the enhanced image is subtracted from the grayscale value of the estimated image, and the absolute value of the difference is taken to obtain the grayscale difference value for that pixel. The grayscale difference values ​​of all pixels are collected together to form the initial registration residual map. For example, if a PCB board has a scratch defect with a length of 0.15 mm, the grayscale value corresponding to this scratch in the enhanced image is 40, while the grayscale value corresponding to the absence of a scratch in the estimated image is 120. The absolute difference between the two is 80 grayscale levels, and this value will be clearly recorded at the corresponding position in the initial registration residual map.

[0043] The root mean square error (RMSE) is calculated by summing the squared grayscale differences of all pixels in the initial registration residual image, dividing by the total number of pixels, and then taking the square root. This RMS error is then output as the initial registration residual. The calculation process is as follows: First, iterate through each pixel in the initial registration residual image, read its grayscale difference, multiply this difference by itself to get the square value, then sum the squared values ​​of all pixels in the image to get a total value. Next, divide this total value by the total number of pixels in the image to get the mean square error (MSE), and finally take the square root of the MSE to obtain the RMS error. The total number of pixels equals the number of pixels in the width direction multiplied by the number of pixels in the height direction. For example, for a 20-megapixel camera, the total number of pixels is 20 million.

[0044] The root mean square error (RMSE) value has the same unit as the gray level, and its magnitude directly reflects the accuracy of the nonlinear deformation field estimation. For example, for a qualified PCB board without defects and with its deformation fully fitted, the difference between the enhanced image and the estimated image mainly comes from sensor noise and sub-pixel residual deformation; in this case, the RMSE value is typically between three and six. For a defective board with multiple defects, the large gray-level difference contributed by the defective areas can increase the RMSE value to between fifteen and twenty-five. For a board with severe nonlinear warping that has not yet been fully fitted, the initial registration residual may reach twenty to forty. In actual production lines, based on statistical experience, the initial registration residual of rigid FR4 PCB boards generally does not exceed five, while the initial registration residual of flexible PCB boards after reflow soldering may reach eight to fifteen.

[0045] This invention does not absolutely limit the above numerical ranges, as these values ​​will vary depending on factors such as camera resolution, light intensity, board material color, and filtering parameters. However, the root mean square error (RMSE), as a dimensionless standardized statistical indicator, can provide a consistent evaluation of registration accuracy across different scenarios. The calculated RMSE value is used as the initial registration residual output. This value will be fed into subsequent optical flow iteration steps to determine whether further deformation field correction is needed and as a reference for feedback control during the iteration process.

[0046] The pre-defined nonlinear optical flow model is as follows: The grayscale value of any pixel in the current enhanced image is represented as a function of that point's coordinates. The grayscale value of the corresponding coordinate point in the deformation-compensated template image is represented as another function. The cube of the difference between the two is calculated and summed, serving as the specific value for the grayscale nonlinear constraint term. The current enhanced image refers to the image obtained after Gaussian filtering and histogram equalization, where each pixel has unique row and column coordinates, and the grayscale value is the brightness value of that point. The deformation-compensated template image refers to the image obtained by resampling pixels from the standard template image according to the currently estimated deformation field.

[0047] For each pixel coordinate, the grayscale value of that point is read from the current enhanced image, and simultaneously from the grayscale value of the same point in the deformation-compensated template image. The difference between these two grayscale values ​​is then calculated, and the cube of this difference is taken as the contribution of that point. The sum of the cubes of the differences across all pixels is the specific value of the grayscale nonlinear constraint term. For example, in a local region of 100 by 100 pixels, if the grayscale difference of the vast majority of pixels is around 2, and the grayscale difference of a few points is 10, the cube operation will amplify the difference of 10 to 1000, while the difference of 2 is only 8. Therefore, this constraint term significantly increases the weight of large error regions.

[0048] The rate of change of the deformation field in the horizontal and vertical directions is calculated. The sum of the squares of the rates of change in the two directions is integrated and then multiplied by a preset regularization coefficient to obtain the specific value of the spatial smoothing constraint term. The deformation field is a two-dimensional vector field, where each pixel stores the horizontal and vertical offsets. For the horizontal offset, the rate of change along the row and column directions is calculated separately. Similarly, for the vertical offset, the rate of change along the row and column directions is also calculated separately.

[0049] Squaring the four rates of change and summing them yields the local roughness measure at that pixel. This measure is then summed over all pixels in the entire image domain (integration). The integral reflects the overall roughness of the deformation field. A larger value indicates a more drastic change in the deformation field, while a smaller value indicates a flatter field. Multiplying this integral by a preset regularization coefficient yields the specific value of the spatial smoothness constraint term. The preset regularization coefficient is a fixed value ranging from 0.001 to 0.1. For common rigid FR4 PCB boards, this coefficient is set to 0.005; for flexible PCB boards, it is set to 0.02. These values ​​are fixed parameters determined through extensive experiments and do not change during the iteration process.

[0050] The registration residual from the previous iteration is multiplied by a preset feedback gain coefficient to obtain the feedback driving force value. The registration residual from the previous iteration refers to the residual registration error value obtained after the previous optical flow calculation, and is a scalar. The preset feedback gain coefficient is a fixed positive value, ranging from 0.5 to 1.5; in this invention, it is specifically set to 1.0. Multiplying the registration residual by the feedback gain coefficient yields the feedback driving force value. This value is used as an additional term in the subsequent energy function. For example, if the registration residual from the previous iteration is 8.3 and the feedback gain coefficient is 1.0, the feedback driving force value is 8.3; if the registration residual is 20.5, the feedback driving force value is 20.5.

[0051] The gray-scale nonlinear constraint term, the spatial smoothing constraint term, and the feedback driving force value are summed according to a preset weight ratio to form a scalar energy function. The preset weight ratio refers to assigning fixed weighting coefficients to the three terms. The weight coefficient of the gray-scale nonlinear constraint term is set to 0.9, the weight coefficient of the spatial smoothing constraint term is the regularization coefficient mentioned earlier, and the weight coefficient of the feedback driving force value is set to 0.1. Multiplying the gray-scale nonlinear constraint term by 0.9, adding the spatial smoothing constraint term multiplied by the regularization coefficient, and adding the feedback driving force value multiplied by 0.1, the sum is the scalar energy function. The smaller the value of this energy function, the more the current deformation field simultaneously satisfies the three objectives: small gray-scale difference, smooth deformation, and consistent direction with the previous residual. The weight ratio is set based on the principle of magnitude balance, so that the three terms have similar numerical contribution ranges.

[0052] Taking the variational derivative of the deformation field variable in the scalar energy function and setting it to zero yields a system of linear equations concerning the deformation field increment. Solving this system of equations yields the optical flow field for the current iteration step. The variational derivative is a standard mathematical operation for finding the extrema of a functional, aiming to identify the deformation field that minimizes the energy function. Taking the variational derivative of the unknown deformation field function in the energy function and setting the result to zero transforms the optimization problem into a partial differential equation. In practical numerical solutions, the finite difference method is used to discretize this partial differential equation, resulting in a large sparse linear system of equations. The unknowns of this system are the deformation field increments at each pixel. The successive over-relaxation iteration method is used to solve this linear system of equations, with a convergence precision of 0.001 pixels. The increment obtained at each pixel is added to the current deformation field to obtain the updated optical flow field, which is the output of the current iteration step.

[0053] The iterative calculation specifically includes the following steps: The enhanced image and initial registration residual are used as inputs, and the nonlinear deformation field estimate is used as the initial value of the optical flow field. The enhanced image refers to the image after Gaussian filtering and histogram equalization, which has removed noise and improved contrast. The initial registration residual is the root mean square error value calculated above, which reflects the accuracy of the initial deformation field estimate. The nonlinear deformation field estimate is the deformation field covering the pixel coordinates of the entire board, which is formed by fusing the first offset field, the amplitude and frequency parameters of edge wave deformation, and the position and contour parameters of asymmetric thermal deformation anomaly regions. These three data are used as the starting point for iterative calculations, with the nonlinear deformation field estimate directly set as the initial value of the optical flow field, i.e., the deformation field used in the first iteration. For example, for a 300 mm by 250 mm PCB board, the nonlinear deformation field estimate stores the horizontal and vertical offsets of each pixel, the initial registration residual is 5.3, and the enhanced image has a total of 20 million pixels. These data are fed into the iteration loop together.

[0054] Using the current optical flow field and the current registration residual, a system of linear equations is solved to obtain the updated optical flow field. This updated optical flow field is then used to re-register the standard template image, generating a new registration estimate image. This new estimate image is then compared pixel-by-pixel with the enhanced image to obtain the registration residual after this iteration. In the current iteration step, the optical flow field currently in use is first retrieved. This optical flow field was the aforementioned nonlinear deformation field estimate in the first iteration, and in subsequent iterations, it is the optical flow field updated in the previous round. Simultaneously, the current registration residual is retrieved. This residual was the initial registration residual in the first iteration, and in subsequent iterations, it is the registration residual calculated in the previous round.

[0055] Following the energy function construction method in the pre-defined nonlinear optical flow model—namely, calculating the variational derivative after a weighted sum of the gray-level nonlinear constraint term, the spatial smoothing constraint term, and the feedback driving force numerical value—a system of linear equations concerning the deformation field increment is established. The resulting increment is added to the current optical flow field to obtain the updated optical flow field. This updated optical flow field is then used to resample the standard template image. Specifically, each pixel of the standard template image is traversed, and the new sampling coordinates are found based on the offset of that pixel in the optical flow field. Finally, bilinear interpolation is used to extract the gray-level value and fill it into the new image. This generated new image is called the new registration estimation image.

[0056] The enhanced image and the new registration estimation image are subtracted at the same pixel position by their gray values, and the absolute value is taken to obtain a difference image. Then, the sum of the squares of the differences of all pixels in the difference image is divided by the total number of pixels, and the square root is taken to calculate a root mean square error value. This value is the registration residual after this iteration. For example, in the first iteration, the initial registration residual is 5.3, and after the above calculation, the registration residual after this iteration is 3.8.

[0057] The registration residual after this iteration is compared with a preset registration error tolerance threshold, which is a fixed value set in advance to determine whether the current registration accuracy meets the requirements. This threshold is set based on the registration accuracy requirements of subsequent defect detection steps. Specifically, defect detection needs to be able to distinguish defects larger than 20 micrometers, corresponding to approximately eight grayscale levels of difference. Therefore, the registration residual tolerance threshold is set to 3.0. If the registration residual is less than 3.0, it indicates that the registration error is less than the level that can interfere with defect detection, and the iteration can be stopped. If the registration residual is still greater than or equal to 3.0, it indicates that the deformation field needs further correction. This value is determined based on extensive experimental statistics. For ordinary rigid PCB boards, two to three iterations are generally sufficient to reduce the registration residual from 5.0 to below 3.0. For flexible boards with large deformation, this threshold can be relaxed to 4.0. In this invention, for general application scenarios, the preset registration error tolerance threshold can be fixed at 3.0.

[0058] If the registration residual after this iteration is greater than or equal to the preset registration error tolerance threshold, then this residual is used as a feedback control variable, and the iteration returns to the previous step. Specifically, when the calculated registration residual after this iteration is 3.8, and the preset registration error tolerance threshold is 3.0, 3.8 is greater than 3.0, therefore the stopping condition is not met. In this case, 3.8 is used as a feedback control variable, that is, it is sent to the feedback driving force calculation stage in the next iteration, multiplied by the preset feedback gain coefficient, and used to construct the energy function. Then, the operation of solving the linear equation system using the current optical flow field and the current registration residual is re-executed, that is, the next iteration begins. For example, at the beginning of the second iteration, the optical flow field is already the optical flow field after the first update, and the current registration residual is 3.8. After solving the linear equation system again, the optical flow field after the second update is obtained. After re-registration, the registration residual after the second iteration is calculated. This process is repeated until the stopping condition is met.

[0059] If the registration residual after this iteration is less than the preset registration error tolerance threshold, the iteration stops, and the registration residual after this iteration is output as the residual registration error. For example, after two iterations, the calculated registration residual value after the second iteration is 2.7, which is less than the preset registration error tolerance threshold of 3.0. At this point, the iteration process terminates immediately. A third iteration is not performed, and the value of 2.7 is output as the residual registration error. This residual registration error will be used in subsequent dynamic correction steps for positioning parameters. In actual production, for most qualified PCB boards, the registration residual usually decreases below the threshold after two to three iterations. For a very small number of boards with particularly severe deformation, the number of iterations may reach five. However, to avoid infinite loops, a maximum number of iterations, such as ten, can be set in the program. If convergence is not achieved after reaching the maximum number of iterations, the current residual is directly output, and a registration failure alarm is given.

[0060] Based on the residual registration error, the positioning parameters are dynamically corrected and compensation is generated. This process includes the following steps: The residual registration error is multiplied by a first correction coefficient to obtain the proportional correction component. The cumulative sum of the residual registration errors is multiplied by a second correction coefficient to obtain the integral correction component. The rate of change of the residual registration error is multiplied by a third correction coefficient to obtain the differential correction component. The residual registration error refers to the value output after optical flow iteration calculation that is less than the preset registration error tolerance threshold. For example, in a certain detection, the residual registration error is 2.7. The first correction coefficient is a fixed value, ranging from 0.5 to 2.0. In this invention, it is set to 1.0. Its function is to directly map the residual registration error to the correction component proportionally. For example, 2.7 multiplied by 1.0 yields a proportional correction component of 2.7. The cumulative sum of the residual registration errors refers to the algebraic sum of all residual registration errors from the start of this correction to the current time. Each time a correction is performed, the current residual registration error is accumulated once. At the first correction, the cumulative sum is equal to the current residual registration error itself.

[0061] The second correction coefficient ranges from 0.01 to 0.1, and is set to 0.05 in this invention. Its function is to control the correction strength of the integral term for long-term cumulative errors. For example, if the cumulative sum of residual registration errors is 8.1, multiplying it by 0.05 yields an integral correction component of 0.405. The rate of change of residual registration error refers to the difference between the current residual registration error and the previous residual registration error, divided by the time interval between the previous correction and the current correction. The rate of change is set to zero for the first correction. The third correction coefficient ranges from 0.1 to 1.0, and is set to 0.5 in this invention. It is used to adjust the response strength to the trend of error change. For example, if the rate of change is 0.6, multiplying it by 0.5 yields a differential correction component of 0.3. The specific values ​​of the above three correction coefficients are fixed parameters determined through tuning experiments and do not change with the detection process.

[0062] The proportional, integral, and differential correction components are added together to obtain the correction amount for the positioning parameters. Continuing the example above, the proportional correction component is 2.7, the integral correction component is 0.405, and the differential correction component is 0.3. Adding these together gives a correction amount of 3.405 for the positioning parameters. This correction amount is in the same unit as the residual registration error, i.e., the dimensionless root mean square error unit, and needs to be converted to the actual movement step size or angle increment of the actuator. For example, if a residual registration error of 2.7 corresponds to an actual position deviation of 36 micrometers, then a correction amount of 3.405 corresponds to a position deviation of approximately 45 micrometers.

[0063] The correction amount is converted into the motion step size or angle increment of the actuator, which serves as the compensation amount. The actuator can be a conveyor belt servo motor used to adjust the position of the plate, or a camera trigger module used to adjust the image acquisition timing. The conversion relationship is achieved through a fixed conversion coefficient, which is the motion step size or angle increment corresponding to each unit of correction amount of the actuator. For example, for a conveyor belt servo motor, if the conversion coefficient is set to a movement distance of 10 micrometers per unit of correction amount, then the motion step size after converting the correction amount of 3.405 is 34.05 micrometers, which is rounded up to 34 micrometers as the compensation amount. For a camera trigger module, if the conversion coefficient is set to a phase offset of 0.5 degrees per unit of correction amount, then the compensation amount is 1.7 degrees. This conversion coefficient is pre-calibrated based on the physical parameters of the actuator and is a fixed value.

[0064] The output includes the corrected positioning parameters and the residual error after compensation. The specific implementation process is as follows: The current positioning parameters are added to the compensation amount to obtain the corrected positioning parameters. The current positioning parameters refer to the positioning reference values ​​before this correction. For example, if the current conveyor belt reference position is 0 millimeters and the camera's reference trigger delay time is 0 milliseconds, the compensation amount is a step size of 34 micrometers. After addition, the corrected positioning parameters are 34 micrometers. For instance, if the current camera trigger delay is 10 milliseconds, and the compensation amount is a 1.7-degree phase shift converted to a 0.09-millisecond delay increment, then the corrected positioning parameters are 10.09 milliseconds.

[0065] The corrected positioning parameters are sent to the production line control system to adjust the image acquisition timing of subsequent PCB boards. After receiving the corrected positioning parameters, the production line control system writes them into the corresponding register of the motion controller. Starting from the next PCB board, the timing of triggering the camera when the conveyor belt reaches its position will be executed according to the new positioning parameters. For example, if the corrected positioning parameters are 34 micrometers, it means that the stopping point of the next board will trigger the image capture 34 micrometers earlier than before.

[0066] The residual registration error is obtained by subtracting the compensation amount from the residual registration error and multiplying it by a preset correction efficiency factor. The preset correction efficiency factor is a coefficient with a fixed value range of 0.5 to 0.9. In this invention, it is set to 0.8. It is used to simulate the proportion of error actually eliminated by the compensation action, because the actuator has mechanical backlash and response delay, and cannot eliminate all errors 100%. For example, if the residual registration error is 2.7 and the compensation amount is 3.405, multiplying them together gives 2.724. Multiplying this by the correction efficiency factor of 0.8 gives 2.179. Subtracting 2.179 from the residual registration error of 2.7 gives 0.521. This value is the residual error after the current compensation, which reflects the amount of remaining error that was not completely eliminated by this compensation.

[0067] The absolute value of the residual error after the current compensation is compared with a preset tolerance range. The preset tolerance range is a fixed value, such as 0.5, indicating that when the absolute value of the residual error is less than or equal to 0.5, the compensation is considered complete and no further correction is needed. This tolerance range is set based on the positioning accuracy requirements of subsequent defect detection, and is generally one-third to one-half of the registration error tolerance threshold. For example, if the absolute value of the residual error after the current compensation is 0.521, which is greater than the tolerance range of 0.5, the termination condition is not met.

[0068] If the error exceeds the allowable range, the system returns to the dynamic correction step for further correction. Specifically, when 0.521 is greater than 0.5, the system will not output the final result. Instead, it will use the current compensated residual error of 0.521 as the new residual registration error and re-execute the previous proportional, integral, and differential calculations—the second correction. In the second correction, the cumulative sum of the residual registration error will be added to 0.521, and the rate of change will be recalculated, resulting in a new compensation amount. This process will be repeated until the absolute value of the residual error falls within the allowable deviation range.

[0069] If the error is within the acceptable range, the current compensation process terminates, and the corrected positioning parameters and the residual error after the current compensation are output. For example, after the second correction, the calculated absolute value of the residual error after the current compensation is 0.4, which is less than 0.5, at which point the iteration stops. The system outputs the corrected positioning parameters obtained after the second correction along with the residual error value of 0.4. The output corrected positioning parameters are used to continuously adjust the acquisition timing of subsequent boards, and the output residual error is used for the next step of inter-board propagation blocking and step-by-step attenuation processing. In the above process, all coefficients, thresholds, and conversion factors are fixed preset values, determined through offline calibration, and do not change dynamically with online detection, ensuring the determinism and reproducibility of the control behavior. For example, in actual production, continuous testing of one thousand PCB boards can bring the residual error of most boards into the acceptable range after one correction. A few boards with large deformations require two to three corrections. If more than three corrections are needed, the board is marked as having a registration anomaly and an alarm is triggered.

[0070] The stable positioning state after output error suppression includes the following steps: The residual error after the current compensation is obtained and recorded as the historical error record for the current board. The residual error after the current compensation refers to the value output in the previous step. For example, the residual error output for a PCB board after positioning parameter correction is 3.2. This error value reflects the registration deviation that still remains on the current board after optical flow iteration and positioning compensation. This value is recorded as the historical error record for this board and used for subsequent propagation blocking and attenuation processing. For example, if a PCB board numbered 123 has a residual error of 3.2 after the current compensation, then the historical error record will store the value 3.2, along with the board number and timestamp.

[0071] A propagation blocking switch is set up so that the historical error record is immediately cleared to zero after the compensation process of the current board is completed, preventing its propagation to subsequent boards. The propagation blocking switch is a logic control flag, which can be represented by a Boolean variable in the software implementation. After the entire compensation process of a PCB board is completed, that is, after the board has gone through the entire process of optical flow iteration, positioning correction, residual error calculation and attenuation processing, the system will immediately clear the value stored in the historical error record to zero, that is, change 3.2 to zero. The purpose of this is to prevent the residual error of the current board from being mistakenly used as the initial historical error of the next board. For example, if it is not cleared, when the 124th board starts processing, it may mistakenly take the 3.2 of the 123rd board as its own historical error starting point, resulting in a cumulative effect of errors. By forcibly clearing the record after each board is processed by the propagation blocking switch, it is ensured that the boards are completely independent and there is no error propagation.

[0072] For the residual error sequence generated during the multi-level compensation iteration process within the current board, the latest residual error value is multiplied by a preset attenuation factor to obtain the attenuated residual error. The attenuation factor is a positive number greater than zero and less than one. During the processing of the current board, positioning compensation may require multiple iterations. For example, after the first compensation, the residual error is 3.2; after the second compensation, it is 1.5; and after the third compensation, it is 0.8. These values, arranged in chronological order, form a residual error sequence. The latest value in the sequence, i.e., the residual error output after the last compensation, is taken, assumed to be 0.8.

[0073] The preset attenuation factor is a positive number with a fixed value between 0.2 and 0.8; in this invention, it is set to 0.5. Multiplying the latest residual error of 0.8 by the attenuation factor of 0.5 yields 0.4, which is the residual error after attenuation processing. The purpose of the attenuation factor is to further compress the residual error after the last compensation. In practical engineering, the residual error is already very small after multiple compensations; multiplying it by an attenuation factor less than one can further suppress potential measurement noise, making subsequent deformation field corrections more stable. The specific value of the attenuation factor is determined experimentally; 0.5 is an empirical value that avoids both excessive attenuation leading to information loss and insufficient attenuation rendering it ineffective.

[0074] The residual error after attenuation is considered as a stable residual error and is superimposed on the current board's deformation field estimate for compensation, resulting in the corrected deformation field. For example, a residual error of 0.4 after attenuation is considered a stable residual deviation. The current board's deformation field estimate refers to the initial deformation field covering all board pixel coordinates, generated by fusing the first offset field, edge wave deformation parameters, and thermal deformation anomaly region parameters. This deformation field stores, for example, the horizontal and vertical offsets of each pixel.

[0075] The stabilized residual error is considered as an additional offset evenly distributed across every pixel on the entire board. This means that a 0.4 pixel offset is added to each pixel horizontally and vertically. Physically, this 0.4 pixel residual error represents a tiny overall offset of the entire board surface relative to its ideal position. Evenly adding this offset to each pixel is equivalent to performing a final global correction to the deformation field. The result of this superposition is the corrected deformation field.

[0076] For example, a point in the deformation field that was originally located at column coordinate 100 and row coordinate 200 has an offset of 1.2 pixels in the horizontal direction and -0.5 pixels in the vertical direction. After superimposing 0.4 pixels, it becomes 1.6 pixels in the horizontal direction and -0.1 pixels in the vertical direction. The same superposition operation is performed on all pixels to obtain a new deformation field.

[0077] The corrected deformation field is output as the stable positioning state of the current board. The corrected deformation field is a two-dimensional vector field that records the final offset of each pixel. This deformation field has integrated all the information from initial feature recognition, optical flow iteration, positioning compensation to attenuation processing, and represents the system's optimal estimate of the current PCB board deformation. The deformation field is output as the stable positioning state of the current board for use in subsequent defect determination steps.

[0078] For example, for a PCB board measuring 150 mm by 100 mm with a resolution of 12 million pixels, the output stable positioning state is a data structure containing 12 million horizontal offset values ​​and 12 million vertical offset values, occupying approximately 192 megabytes of memory. This data is directly sent to the defect detection module to register the standard template image onto the actual deformation state of the current board, thereby accurately detecting defects such as scratches, notches, and foreign objects. All parameters in the above process, such as the attenuation factor of 0.5 and the logic of the propagation blocking switch, are fixed presets and are not dynamically adjusted during the detection process, ensuring that the processing of each board is independent and the results are repeatable.

[0079] The process of performing defect determination on the PCB board based on the stable positioning state is as follows: Using the corrected deformation field in a stable positioning state, this deformation field is applied to the standard template image to generate the final registration estimation image. The corrected deformation field refers to the two-dimensional offset field obtained after attenuation processing and superposition compensation with the original deformation field, which stores the horizontal and vertical offsets to be applied at each pixel coordinate. The standard template image is resampled according to this deformation field, that is, each pixel in the standard template image is traversed, the corresponding offset is retrieved from the deformation field based on the pixel coordinate, the new sampling coordinates are calculated, and then the gray value at that coordinate is extracted from the standard template image using bilinear interpolation and filled into the same original coordinate position in the final registration estimation image. The final registration estimation image generated in this way has fully compensated for the nonlinear warping, edge wavy deformation, and asymmetric thermal deformation of the PCB board. For example, on a PCB board with local concavity, the pads that were originally arranged in a straight line in the standard template image will be offset to coordinates that match the curved position in the actual image after the deformation field is applied, so that the final registration estimation image is perfectly aligned with the actual captured enhanced image in spatial geometry.

[0080] The final registration estimation image is compared pixel-by-pixel with the real-time acquired and preprocessed enhanced image, and the absolute value of the grayscale difference for each pixel is calculated. The real-time acquired and preprocessed enhanced image refers to the image obtained after Gaussian filtering and histogram equalization, which contains the true grayscale information of the PCB board. The final registration estimation image is the ideal grayscale image generated after deformation registration of the standard template. For each pixel coordinate, the grayscale value of the enhanced image is subtracted from the grayscale value of the estimated image, and the absolute value is taken to obtain the absolute value of the grayscale difference at that point. For example, at a certain pixel, the grayscale value of the enhanced image is 120, and the grayscale value of the estimated image is 100, then the absolute value of the grayscale difference is 20. The same operation is performed on all pixels to obtain a grayscale difference absolute value image with the same resolution as the original image. In this image, the locations with larger absolute values ​​of grayscale difference usually correspond to actual defects or residual registration errors, while the absolute values ​​of grayscale difference in normal areas are close to zero because the registration is accurate and defect-free.

[0081] The absolute value of the grayscale difference of each pixel is compared with a preset defect discrimination threshold, which is used to distinguish between normal grayscale fluctuations and true defects. The preset defect discrimination threshold is a fixed value, and its unit is grayscale level. This threshold is set based on the maximum grayscale fluctuation range caused by normal texture differences on the PCB board surface, uneven lighting, and sensor noise. Through experimental statistics, for an inspection system using standard lighting and a 20-megapixel industrial camera, the absolute value of the grayscale difference in a normal defect-free area is generally between five and twelve. Therefore, this invention sets the preset defect discrimination threshold to fifteen. This value ensures that normal fluctuations are not misjudged as defects, while effectively capturing scratches or gaps with a minimum size of 0.05 mm, because such defects will cause at least twenty grayscale level changes. If the absolute value of the grayscale difference of a pixel is greater than fifteen, the pixel is marked as a candidate defect point; if it is less than or equal to fifteen, the point is considered normal. For example, if there is a gap of 0.1 mm in length on a pad, the grayscale value of the enhanced image at the gap is 40, the estimated grayscale value is 90, and the absolute value of the grayscale difference is 50, which is greater than 15, then this pixel is marked as a candidate defect point.

[0082] Adjacent candidate defect points are merged into connected components. Connected components with an area smaller than a preset area threshold are filtered out as noise. The remaining connected components are identified as real defect regions, and the position and size of each real defect region are recorded. The eight-neighbor rule is used for merging connected components: if another candidate defect point exists in any of the eight directions (up, down, left, right, upper left, upper right, lower left, lower right) of a candidate defect point, then the two points are considered connected and belong to the same connected component. All interconnected candidate defect points are merged into a single connected component, and the total number of pixels contained in this component is calculated. This total number is multiplied by the actual physical area represented by each pixel (e.g., if each pixel has a side length of 0.05 mm, then the area of ​​a single pixel is 0.0025 square millimeters) to obtain the physical area of ​​the connected component.

[0083] The preset area threshold is a fixed value. Its purpose is to eliminate isolated noise points with excessively small areas to avoid false alarms. This threshold is set based on the minimum acceptable defect size in actual production. For example, for consumer-grade PCBs, only defects with a length greater than 0.1 mm or an area greater than 0.01 square millimeters require an alarm. Considering that a single pixel may be misjudged due to noise, the area threshold is set to three pixels, meaning only connected regions with an area greater than 0.0075 square millimeters are retained. If the merged area is less than three pixels, it is considered noise and directly filtered out without reporting. If the area is greater than or equal to three pixels, the connected region is determined to be a real defect area.

[0084] For each real defective region, record its location information, such as the coordinates of the top-left and bottom-right corners of the region's smallest bounding rectangle, or the coordinate sequence of all points on the region's boundary. Simultaneously, record the region's dimensions, such as width, height, and area. For example, on a PCB board, three candidate regions are obtained through connected component merging, with areas of two pixels, fifteen pixels, and eight pixels respectively. The two-pixel region is filtered out because it is smaller than the area threshold of three pixels. The fifteen-pixel and eight-pixel regions are retained as real defective regions, recorded as defect A and defect B respectively. Defect A is located within a rectangular region at column 32, row 60, with dimensions of 0.75 mm wide and 0.5 mm high. Defect B is located at column 120, row 240, with dimensions of 0.4 mm wide and 0.3 mm high.

[0085] The system outputs the defect determination result, including whether the board passes without defects or an alarm indicates a defect, along with the location information of all actual defective areas. If, after the above connected component merging and threshold filtering, the area of ​​any connected component reaches the preset area threshold, the determination result is "pass without defects," and the system outputs a pass signal, allowing the PCB board to proceed to the next process. If one or more actual defective areas exist, the determination result is "an alarm indicates a defect," and the system generates an alarm signal. Simultaneously, it packages and outputs the location and size information of all recorded actual defective areas. This information can be used for subsequent manual re-inspection or automatic sorting devices to remove defective PCB boards from the production line.

[0086] For example, the output is a data structure containing a flag indicating a defect (true) and detailed data for two defect areas: Defect A might be a scratch, located at column 32-60 (top left corner) and column 47-70 (bottom right corner) of the rectangle; Defect B might be a notch, located at column 120-240 (top left corner) and column 128-246 (bottom right corner) of the rectangle. All thresholds (the defect detection threshold is set to 15, and the area threshold is set to 3 pixels) are preset fixed values ​​determined through offline calibration experiments and do not change dynamically with online detection, ensuring the stability and consistency of the detection standard.

[0087] Example 2: A PCB board defect detection system based on vision positioning, comprising: The feature recognition module is used to acquire PCB board images in real time and identify nonlinear warping and non-uniform deformation features of the PCB board under dynamic conditions. The features include at least local random twisting of the board surface, edge wave deformation and asymmetric thermal deformation. The deformation estimation module is used to generate a deformation field estimate of the current plate based on the identified features and to calculate the initial registration residual of the current plate. The optical flow iteration module is used to call a preset nonlinear optical flow model based on the deformation field estimation and the initial registration residual, and use the registration residual as a feedback control quantity for iterative calculation until the registration residual is less than the preset registration error tolerance threshold, and output the residual registration error. The positioning compensation module is used to dynamically correct the positioning parameters based on the residual registration error and generate the compensation amount, and output the corrected positioning parameters and the residual error after the current compensation. The detection output module is used to perform inter-board propagation blocking and attenuation processing on the residual error after the current compensation, output the stable positioning state after error suppression, perform defect judgment on the PCB board based on the stable positioning state, and finally output the detection result.

[0088] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0089] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0090] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0091] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0092] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A PCB board defect detection method based on vision positioning, characterized in that, Includes the following steps: Real-time acquisition of PCB board images during transmission, identification of nonlinear warping and non-uniform deformation features of PCB board under dynamic conditions, features including at least local random twisting of board surface, edge wavy deformation and asymmetric thermal deformation. Based on the identified features, a deformation field estimate of the current plate is generated, and the initial registration residual of the current plate is calculated. Based on the deformation field estimation and the initial registration residual, a preset nonlinear optical flow model is invoked, and the registration residual is used as a feedback control quantity for iterative calculation until the registration residual is less than the preset registration error tolerance threshold, and the residual registration error is output. Based on the residual registration error, the positioning parameters are dynamically corrected and a compensation amount is generated. The corrected positioning parameters and the residual error after the current compensation are output. The residual error after current compensation is processed to block and attenuate inter-board propagation, and the stable positioning state after error suppression is output. Based on the stable positioning state, the PCB board is judged for defects, and the final detection result is output.

2. The PCB board defect detection method based on vision positioning according to claim 1, characterized in that, Identifying the nonlinear warping and non-uniform deformation characteristics of PCB boards under dynamic conditions includes the following steps: Gaussian filtering is applied to the acquired current board image to remove noise, and then histogram equalization is performed to enhance contrast, resulting in an enhanced image. Scale-invariant feature transformation points are extracted from the enhanced image and coarsely matched with corresponding feature points in the standard template image to obtain initial affine transformation parameters. Based on the initial affine transformation, the pixel offset of the local region of the image is fitted using a thin plate spline interpolation function to obtain the first offset field describing the local random distortion of the plate surface; Second derivative analysis was performed on the edge contour of the enhanced image to extract the amplitude and frequency parameters of the edge wave deformation. The following method is used to identify gray-level gradient anomaly regions caused by asymmetric thermal deformation: the enhanced image is divided into multiple non-overlapping local windows, the gray-level gradient magnitude and principal direction of pixels in each window are calculated, and the mean and standard deviation of the gray-level gradient magnitude of the entire plate are calculated. Windows with gray-level gradient magnitudes higher than three times the mean standard deviation are marked as abnormal windows. Then, adjacent abnormal windows are merged into connected components. Connected components with an area greater than a preset area threshold after merging are determined as gray-level gradient anomaly regions. The location and contour parameters of the abnormal region are output.

3. The PCB board defect detection method based on vision positioning according to claim 2, characterized in that, Based on the identified features, a deformation field estimate of the current plate is generated, and the initial registration residual of the current plate is calculated. The implementation process is as follows: The first offset field, the amplitude and frequency parameters of edge wave deformation, and the position and contour parameters of the asymmetric thermal deformation anomaly region are fused to construct a nonlinear deformation field estimate covering the pixel coordinates of the entire board. Then, the nonlinear deformation field estimation is applied to the standard template image to generate a registered estimated image. The gray values ​​of the enhanced image and the estimated image at the same pixel position are subtracted to obtain an initial registration residual image. The grayscale differences of all pixels in the initial registration residual image are summed by squares, divided by the total number of pixels, and the square root is taken to calculate the root mean square error. Finally, the root mean square error is used as the output of the initial registration residual.

4. The PCB board defect detection method based on vision positioning according to claim 3, characterized in that, The pre-defined nonlinear optical flow model is as follows: The gray value of any pixel in the current enhanced image is represented as a function of the coordinates of that point. The gray value of the corresponding coordinate point in the template image after deformation compensation is represented as another function. The cube of the difference between the two is calculated and summed as the specific value of the gray nonlinear constraint term. Calculate the rate of change of the deformation field in the horizontal and vertical directions, integrate the sum of the squares of the rates of change in the two directions, and then multiply by a preset regularization coefficient to obtain the specific value of the spatial smoothing constraint term. Take the registration residual from the previous iteration and multiply it by the preset feedback gain coefficient to obtain the feedback driving force value; The gray-scale nonlinear constraint term, the spatial smoothing constraint term, and the feedback driving force value are added together according to a preset weight ratio to form a scalar energy function. Taking the variational derivative of the deformation field variable in the scalar energy function and setting the derivative to zero yields a system of linear equations about the deformation field increment. Solving this system of equations yields the optical flow field of the current iteration step.

5. The PCB board defect detection method based on vision positioning according to claim 4, characterized in that, The iterative calculation specifically includes the following steps: The enhanced image and the initial registration residual are used as inputs, and the nonlinear deformation field estimate is used as the initial value of the optical flow field. Using the current optical flow field and the current registration residual, the linear equation system is solved to obtain the updated optical flow field. The updated optical flow field is then used to re-register the standard template image to generate a new registration estimation image. This image is then compared pixel by pixel with the enhanced image to obtain the registration residual after this iteration. The registration residual after this iteration is compared with the preset registration error tolerance threshold. If the registration residual is greater than or equal to the registration error tolerance threshold, the residual is used as a feedback control variable, and the process returns to the previous step to continue iterating. If the registration residual is less than the registration error tolerance threshold, the iteration stops and the registration residual is output as the residual registration error.

6. The PCB board defect detection method based on vision positioning according to claim 5, characterized in that, Based on the residual registration error, the positioning parameters are dynamically corrected and compensation is generated. This process includes the following steps: Multiplying the residual registration error by the first correction factor yields the proportional correction component; multiplying the cumulative sum of the residual registration errors by the second correction factor yields the integral correction component; and multiplying the rate of change of the residual registration error by the third correction factor yields the differential correction component. The proportional correction component, integral correction component, and differential correction component are added together to obtain the correction amount of the positioning parameters. The correction amount is converted into the motion step size or angle increment of the actuator as the compensation amount.

7. The PCB board defect detection method based on vision positioning according to claim 6, characterized in that, The output includes the corrected positioning parameters and the residual error after the current compensation. The specific implementation process is as follows: Add the current positioning parameters to the compensation amount to obtain the corrected positioning parameters; The corrected positioning parameters are sent to the production line control system to adjust the image acquisition timing of subsequent PCB boards; Subtract the compensation amount from the residual registration error and multiply it by the preset correction efficiency factor to obtain the residual error after the current compensation. The absolute value of the residual difference after the current compensation is compared with the preset allowable deviation range: if it exceeds the allowable range, the process returns to the dynamic correction step to continue correction; if it is within the allowable range, the current compensation process is terminated. Finally, the corrected positioning parameters and the residual error after the current compensation are output.

8. The PCB board defect detection method based on vision positioning according to claim 7, characterized in that, The stable positioning state after output error suppression includes the following steps: Obtain the residual error after the current compensation, and use this residual error as the historical error record of the current board; Set a propagation blocking switch so that when the compensation process of the current board is completed, the historical error record is immediately cleared to zero, preventing it from being transmitted to the next board. For the residual error sequence generated during the multi-level compensation iteration process within the current board, the latest residual error value is multiplied by a preset attenuation factor to obtain the residual error after attenuation processing. The attenuation factor is a positive number that is greater than zero and less than one. The residual error after attenuation is regarded as a stable residual error and is superimposed and compensated with the current deformation field estimate of the plate to obtain the corrected deformation field. The corrected deformation field is output as the stable positioning state of the current plate.

9. The PCB board defect detection method based on vision positioning according to claim 8, characterized in that, The process of performing defect determination on the PCB board based on the stable positioning state is as follows: Using the corrected deformation field in the stable positioning state, the deformation field is applied to the standard template image to generate the final registration estimation image; The final registered and estimated image is compared pixel by pixel with the real-time acquired and preprocessed enhanced image, and the absolute value of the gray level difference of each pixel is calculated. The absolute value of the grayscale difference of each pixel is compared with a preset defect discrimination threshold, which is used to distinguish between normal grayscale fluctuations and true defects. If the absolute value of the grayscale difference of a pixel is greater than the defect discrimination threshold, then the pixel is marked as a candidate defect point; Connected components of adjacent candidate defect points are merged. Connected components with an area smaller than a preset area threshold are filtered out as noise. The remaining connected components are determined as real defect regions, and the position and size of each real defect region are recorded. Output the defect determination result, including whether there is no defect and an alarm is triggered, along with the location information of all actual defective areas.

10. A PCB board defect detection system based on vision positioning, used to implement the PCB board defect detection method based on vision positioning as described in any one of claims 1-9, characterized in that, include: The feature recognition module is used to acquire PCB board images in real time and identify nonlinear warping and non-uniform deformation features of the PCB board under dynamic conditions. The features include at least local random twisting of the board surface, edge wave deformation and asymmetric thermal deformation. The deformation estimation module is used to generate a deformation field estimate of the current plate based on the identified features and to calculate the initial registration residual of the current plate. The optical flow iteration module is used to call a preset nonlinear optical flow model based on the deformation field estimation and the initial registration residual, and use the registration residual as a feedback control quantity for iterative calculation until the registration residual is less than the preset registration error tolerance threshold, and output the residual registration error. The positioning compensation module is used to dynamically correct the positioning parameters based on the residual registration error and generate the compensation amount, and output the corrected positioning parameters and the residual error after the current compensation. The detection output module is used to perform inter-board propagation blocking and attenuation processing on the residual error after the current compensation, output the stable positioning state after error suppression, perform defect judgment on the PCB board based on the stable positioning state, and finally output the detection result.

Images