Tab false welding detection method and system based on gray difference

Abstract

The present application relates to the technical field of visual detection, in particular to a tab virtual welding detection method and system based on gray level difference, comprising the following steps: collecting tab non-welding template image and tab to-be-tested image, performing gray level difference comparison operation on the tab non-welding template image and the tab to-be-tested image, and generating local mutual information distribution matrix; constructing an image region grid control point coordinate set, and calculating the pixel displacement vector corresponding to the grid control point. In the present application, the image pixel neighborhood deformation relationship is described by the partial derivative matrix set and the Jacobian determinant numerical set, and finally the virtual welding position boundary coordinate set is obtained according to the unit constant reference, the preset abnormal limit parameter, the continuous adjacent pixel position connected region coordinate and the area boundary extraction relationship, so as to improve the virtual welding area positioning accuracy, enhance the defect recognition stability in the weak gray level difference scene, and reduce the false detection and missed detection caused by simply relying on the edge or brightness threshold.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, and in particular to a method and system for detecting poor solder joints on electrodes based on grayscale differences. Background Technology

[0002] Visual inspection technology mainly involves using image acquisition, grayscale analysis, texture recognition, edge extraction, region segmentation, feature comparison, and defect determination to automatically inspect the surface condition, structural morphology, assembly position, and processing quality of industrial products.

[0003] Current technologies primarily rely on conventional visual processing procedures such as image acquisition, grayscale analysis, texture recognition, edge extraction, region segmentation, feature comparison, and defect determination. In actual operation, the surface condition of a product is typically transformed into two-dimensional appearance features such as brightness, texture, and edge or region contours. Detection and judgment depend more on direct visual results such as local grayscale differences, boundary clarity, or defect area area. When there is reflection, indentation, material texture differences, slight cold solder joints, or indistinct transitions at the welding edge in the tab welding area, grayscale changes may present a localized, continuous, gradual change, and the edge contour may not form a clear break. Region segmentation results are easily affected by background brightness fluctuations, leading to the cold solder joint location being misjudged as normal texture or ordinary shadow. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and to propose a method and system for detecting poor solder joints on electrode tabs based on grayscale differences.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a method for detecting poor solder joints on electrode tabs based on grayscale differences, comprising the following steps:

[0006] Collect the unsoldered template image and the image to be tested of the electrode, perform grayscale level difference comparison calculation on the unsoldered template image and the image to be tested of the electrode, and generate a local mutual information distribution matrix;

[0007] Construct a set of grid control point coordinates covering the image region, calculate the pixel displacement vectors corresponding to the grid control points, construct a similarity gradient by combining the values ​​in the local mutual information distribution matrix, and iteratively generate the displacement control point grid matrix.

[0008] Calculate the principal elongation values ​​of adjacent control points within the displacement control point grid matrix, and generate a regularized penalty stress tensor.

[0009] The regularized penalty stress tensor is converted into a regularized constraint force vector by performing divergence operation. The displacement vectors of the control points in the displacement control point grid matrix are then superimposed with the regularized constraint force vector to generate a two-dimensional displacement vector field.

[0010] Partial derivatives are calculated along the horizontal and vertical coordinate axes for the displacement components of each pixel position in the two-dimensional displacement vector field to generate a set of partial derivative matrices.

[0011] Based on the set of partial derivative matrices, calculate and generate the set of Jacobian determinants.

[0012] The values ​​in the Jacobian determinant set are subtracted from the unit constant benchmark. The coordinates of the connected regions of consecutive adjacent pixels whose absolute difference is less than a preset abnormal boundary parameter are extracted to generate a suspected connected region of a poor solder joint. Based on the suspected connected region of a poor solder joint, the boundary coordinate set of the poor solder joint position is obtained.

[0013] Preferably, the step of obtaining the displacement control point grid matrix is ​​as follows:

[0014] Acquire an unsoldered template image of the electrode and an image of the electrode to be tested. Read the gray level value of each pixel position in the unsoldered template image of the electrode and read the gray level value of the same pixel position in the image of the electrode to be tested. Pair the gray level values ​​of the same pixel position point by point. According to the correspondence between the horizontal coordinate and the vertical coordinate of the pixel, perform gray level difference comparison calculation item by item and extract the gray level mutual information feature value at the corresponding position.

[0015] According to the pixel horizontal coordinate order and pixel vertical coordinate order of the detection area on the electrode surface, the gray-level mutual information feature value of each corresponding position is written into the matrix unit under the same coordinate index. The row and column correspondence between the matrix unit and the pixel position is verified. The gray-level mutual information feature value of adjacent pixels is used to fill in the missing coordinate index position to generate a local mutual information distribution matrix.

[0016] A set of grid control point coordinates covering the image region is constructed. The pixel coordinates of each grid control point in the unwelded template image of the tab are read one by one. The pixel coordinates of each grid control point in the image to be tested of the tab are read one by one. The horizontal and vertical pixel displacement components corresponding to the same grid control point are calculated. The values ​​of the neighborhood of the same coordinates in the local mutual information distribution matrix are converted into similarity gradients. The horizontal and vertical pixel displacement components are iteratively updated according to the similarity gradients to generate the displacement control point grid matrix.

[0017] Preferably, the step of obtaining the regularization penalty stress tensor is as follows:

[0018] Read the horizontal coordinates, vertical coordinates, horizontal pixel displacement components, and vertical pixel displacement components of each control point in the displacement control point grid matrix. Establish pairing relationships between adjacent control points according to the horizontal and vertical adjacent order. Perform difference calculation on the coordinate interval of each group of adjacent control points and on the displacement component change of each group of adjacent control points. Map the displacement component change to the coordinate interval direction and calculate the main elongation value of adjacent control points.

[0019] The control point position index corresponding to the principal elongation value of the adjacent control points is called item by item, the elastic constraint parameter under the corresponding control point position index is read, the principal elongation value of the adjacent control points and the elastic constraint parameter are multiplied item by item, and the product operation result is written into the tensor row and column positions according to the horizontal arrangement order and the vertical arrangement order of the control points to generate a regularized penalty stress tensor.

[0020] Preferably, the steps for obtaining the two-dimensional displacement vector field are as follows:

[0021] Read the displacement vectors of the control points within the displacement control point grid matrix, match each control point displacement vector with the regularized penalty stress tensor according to the same control point position index, perform divergence operation on the regularized penalty stress tensor to extract the regularized constraint force vector, perform vector superposition and summation operation on the matched control point displacement vectors and regularized constraint force vectors, distribute the superposition and summation operation values ​​according to the coordinate distance from all pixel positions in the detection area of ​​the tab surface to adjacent control points, and analyze the lateral displacement component and longitudinal displacement component of each pixel position to generate a two-dimensional displacement vector field.

[0022] Preferably, the steps for obtaining the set of partial derivative matrices are as follows:

[0023] Based on the two-dimensional displacement vector field, the lateral displacement component and the longitudinal displacement component of each pixel position in the detection area of ​​the tab surface are read one by one. According to the increasing order of the pixel lateral coordinate, the changes in the lateral displacement component and the longitudinal displacement component at adjacent lateral coordinates of the same pixel position are extracted. According to the increasing order of the pixel longitudinal coordinate, the changes in the lateral displacement component and the longitudinal displacement component at adjacent longitudinal coordinates of the same pixel position are extracted. The four types of displacement component changes are mapped to the pixel lateral coordinate interval and the pixel longitudinal coordinate interval, respectively. Partial derivative calculations are performed, and the four partial derivative component terms are arranged to generate a set of partial derivative matrices.

[0024] Preferably, the step of obtaining the set of Jacobian determinants is as follows:

[0025] Based on the set of partial derivative matrices, according to the matrix row and column indices of each pixel position, read the two partial derivative component terms located at the main diagonal position. Add the unit constant reference to the two partial derivative component terms to generate the deformation gradient main diagonal component. Perform pixel-by-pixel multiplication on the deformation gradient main diagonal component to obtain the deformation gradient main diagonal component product. Read the two partial derivative component terms located at the secondary diagonal position and perform pixel-by-pixel multiplication to obtain the secondary diagonal partial derivative component product. Bind the deformation gradient main diagonal component product and the secondary diagonal partial derivative component product according to the pixel horizontal coordinate order and pixel vertical coordinate order to form the diagonal partial derivative component product.

[0026] Based on the product of the diagonal partial derivative components, the product of the main diagonal components of the deformed gradient corresponding to each pixel position is called one by one, and the product of the secondary diagonal partial derivative components corresponding to the same pixel position is called one by one. According to the pixel horizontal coordinate index and pixel vertical coordinate index, the product of the main diagonal components of the deformed gradient is used as the subtrahend, and the product of the secondary diagonal partial derivative components is used as the subtraction term. The difference is calculated, and the result of the difference operation at each pixel position is written into the numerical position under the same coordinate index to generate the Jacobian determinant numerical set.

[0027] Preferably, the step of obtaining the suspected connected components of the poor solder joint is as follows:

[0028] Read the Jacobian determinant value of each pixel position in the set of Jacobian determinant values, call the unit constant reference item by item according to the horizontal coordinate order and the vertical coordinate order of the pixels, subtract the unit constant reference from each Jacobian determinant value, take the absolute value of each difference result, write the absolute value of the difference to the value position under the same pixel coordinate index, compare the absolute value of the difference item by item with the preset abnormality limit parameter, mark the pixel position where the absolute value of the difference is less than the preset abnormality limit parameter, and obtain the pixel position mark within the abnormality limit;

[0029] Based on the pixel position markings within the abnormal boundary, the marked pixel positions are scanned row by row according to the horizontal and vertical adjacency relationships of the pixels, and the marked pixel positions are verified column by column. The continuous connection relationships in the adjacent directions of each marked pixel position are recorded. The pixel position coordinates with continuous connection relationships are grouped into the same connected region coordinates, and the individual pixel position coordinates that do not form continuous adjacency relationships are removed to generate suspected connected regions with poor soldering.

[0030] Preferably, the step of obtaining the boundary coordinate set of the cold solder joint location is as follows:

[0031] The number of pixels in each suspected poor solder joint connected region is counted one by one. The total number of pixels in the detection area of ​​the tab surface is read. The number of pixels in each suspected poor solder joint connected region is divided by the total number of pixels in the detection area of ​​the tab surface to obtain the area ratio of the corresponding connected region. The area ratio of the corresponding connected region is compared with the area tolerance threshold one by one. The suspected poor solder joint connected regions that exceed the area tolerance threshold are selected. The outermost adjacent pixel position of the selected suspected poor solder joint connected region is used to extract the outer coordinate set of the connected region edge to generate the boundary coordinate set of the poor solder joint position.

[0032] The present invention also provides a system comprising:

[0033] Image acquisition and registration module: used to acquire the unsoldered template image of the electrode and the image to be tested of the electrode, perform grayscale level difference comparison calculation on the unsoldered template image of the electrode and the image to be tested of the electrode, generate a local mutual information distribution matrix, construct a set of grid control point coordinates covering the image area, calculate the pixel displacement vector corresponding to the grid control point, construct a similarity gradient by combining the values ​​in the local mutual information distribution matrix, and iteratively generate the displacement control point grid matrix.

[0034] Displacement vector field generation module: used to calculate the principal elongation values ​​of adjacent control points in the displacement control point grid matrix, generate a regularized penalty stress tensor, perform divergence operation on the regularized penalty stress tensor to convert it into a regularized constraint force vector, and perform vector superposition operation on the displacement vector of the control point in the displacement control point grid matrix and the regularized constraint force vector to generate a two-dimensional displacement vector field;

[0035] Jacobian determinant calculation module: used to perform partial derivative calculations on the displacement components of each pixel position in the two-dimensional displacement vector field along the horizontal and vertical coordinate axes, generate a set of partial derivative matrices, and calculate and generate a set of Jacobian determinant values ​​based on the set of partial derivative matrices;

[0036] The cold solder joint area identification module is used to calculate the difference between each value in the Jacobian determinant set and the unit constant benchmark, extract the coordinates of the connected regions of consecutive adjacent pixels whose absolute difference is less than a preset abnormal boundary parameter, generate a suspected cold solder joint connected region, and obtain the cold solder joint location boundary coordinate set based on the suspected cold solder joint connected region.

[0037] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0038] This invention acquires an unwelded template image and a test image of the electrode tab, and generates a local mutual information distribution matrix based on grayscale level differences. This transforms grayscale changes from simple single-point brightness comparisons into spatially correlated local features. Furthermore, by combining the coordinate set of grid control points in the covered image region, the pixel displacement vectors corresponding to the grid control points, and similarity gradient iterations, a displacement control point grid matrix is ​​formed. This allows for the continuous representation of minute grayscale shifts, local morphological misalignments, and surface continuity changes caused by incomplete welding in the electrode tab welding area. Further, it utilizes the principal elongation values ​​of adjacent control points, the regularized penalty stress tensor, and... The regularized constraint force vector and the control point displacement vector are superimposed to generate a two-dimensional displacement vector field, which enables local gray-level anomalies to be expanded into the lateral and longitudinal displacement responses of the pixel positions in the entire detection area. Then, the deformation relationship of the pixel neighborhood is characterized by the set of partial derivative matrices and the set of Jacobian determinants. Finally, the boundary coordinate set of the weld failure position is obtained based on the unit constant benchmark, the preset anomaly limit parameters, the coordinates of the connected regions of continuous adjacent pixel positions and the area boundary extraction relationship. This improves the positioning accuracy of the weld failure area, enhances the defect recognition stability in weak gray-level difference scenarios, and reduces false detections and missed detections caused by simply relying on the edge or brightness threshold. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the local mutual information distribution matrix;

[0040] Figure 2 This is a schematic diagram of a two-dimensional displacement vector field. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0042] Please see Figure 1-2 This invention provides a technical solution for detecting poor solder joints on electrode tabs based on grayscale differences, comprising the following steps:

[0043] Collect the unsoldered template image and the image to be tested of the electrode, perform grayscale level difference comparison calculation on the unsoldered template image and the image to be tested of the electrode, generate a local mutual information distribution matrix, construct a set of grid control point coordinates covering the image area, calculate the pixel displacement vector corresponding to the grid control point, construct a similarity gradient by combining the values ​​in the local mutual information distribution matrix, and iteratively generate the displacement control point grid matrix.

[0044] Calculate the principal elongation values ​​of adjacent control points within the displacement control point grid matrix, generate a regularized penalty stress tensor, perform divergence calculation on the regularized penalty stress tensor to convert it into a regularized constraint force vector, and perform vector superposition operation on the displacement vector of the control point within the displacement control point grid matrix and the regularized constraint force vector to generate a two-dimensional displacement vector field.

[0045] The partial derivatives of the displacement components of each pixel position in the two-dimensional displacement vector field are calculated along the horizontal and vertical coordinate axes to generate a set of partial derivative matrices. Based on the set of partial derivative matrices, the set of Jacobian determinants is calculated.

[0046] The values ​​in the Jacobian determinant set are subtracted from the unit constant benchmark. The coordinates of the connected regions of consecutive adjacent pixels whose absolute difference is less than the preset abnormal boundary parameter are extracted to generate the suspected connected region of the false solder joint. Based on the suspected connected region of the false solder joint, the boundary coordinate set of the false solder joint position is obtained.

[0047] The steps for obtaining the displacement control point grid matrix are as follows:

[0048] Acquire an unsoldered template image of the electrode and an image of the electrode to be tested. Read the gray level value of each pixel position in the unsoldered template image of the electrode and read the gray level value of the same pixel position in the image of the electrode to be tested. Pair the gray level values ​​of the same pixel position point by point. According to the correspondence between the horizontal coordinate and the vertical coordinate of the pixel, perform gray level difference comparison calculation item by item and extract the gray level mutual information feature value at the corresponding position.

[0049] According to the order of the horizontal and vertical coordinates of the pixels in the detection area of ​​the electrode surface, the gray-level mutual information feature value of each corresponding position is written into the matrix cell under the same coordinate index. The row and column correspondence between the matrix cell and the pixel position is verified. The gray-level mutual information feature value of adjacent pixels is used to fill in the missing coordinate index position to generate a local mutual information distribution matrix.

[0050] A set of grid control point coordinates covering the image region is constructed. The pixel coordinates of each grid control point in the unwelded template image of the tab are read one by one. The pixel coordinates of each grid control point in the image to be tested of the tab are read one by one. The horizontal and vertical pixel displacement components corresponding to the same grid control point are calculated. The values ​​of the neighborhood of the same coordinates in the local mutual information distribution matrix are transformed into similarity gradients. The horizontal and vertical pixel displacement components are iteratively updated based on the similarity gradients to generate the displacement control point grid matrix.

[0051] Specifically, the collected images of the unsoldered tab template and the tab under test are first converted to 8-bit grayscale images to ensure that the grayscale level of each pixel is between 0 and 255. Then, the coordinates of each pixel position within the detection area on the tab surface are traversed. ,in The horizontal axis is... Using the vertical axis, read the grayscale value at that location from the unsoldered electrode template image. Simultaneously, grayscale values ​​at the exact same coordinate positions are read from the electrode image under test. Next, for each pixel Define a space of size centered at the center. The neighboring window, for example, setting That is, a 5×5 window, within which the grayscale values ​​of all pixels in the template image and the image to be tested are statistically compared. ,in It is the grayscale value within the template image window. It is the corresponding grayscale value within the window of the image to be tested, based on the data collected within the window. Estimate the local joint probability distribution of each gray-level pair. With marginal probability distribution and Subsequently, the mutual information model from information theory was applied to calculate this. Gray-level mutual information feature value of location Its calculation method follows the mutual information entropy formula:

[0052] ;

[0053] in, Represents the pixel position The mutual information value calculated at that location. and These represent all possible grayscale values ​​within the neighborhood windows of the template image and the image to be tested, respectively. The joint probability is calculated by pairing the grayscale values ​​within a statistical window with their frequencies of occurrence. and The edge probability is calculated by statistically analyzing the frequency of occurrence of individual grayscale values ​​within each window. The base of the logarithmic operation is chosen to be 2, so that the mutual information unit is bits. This calculation process is repeated for all pixel positions within the detection area to extract the grayscale mutual information feature value at the corresponding position.

[0054] Based on the pixel horizontal and vertical coordinate order of the detection area on the electrode surface, a two-dimensional matrix with the exact same size as the detection area image is created. This matrix is ​​named the Local Mutual Information Distribution Matrix. Its row index corresponds to the vertical coordinate of the pixel, and its column index corresponds to the horizontal coordinate of the pixel. Then, the values ​​for each pixel from the previous step are... Calculated gray-scale mutual information feature values Enter the corresponding cells of the matrix. After the write operation is complete, the matrix undergoes an integrity check to examine for missing cells or invalid values ​​(such as NaN or infinity) caused by computational anomalies (e.g., a zero probability in mutual information calculation leading to an undefined logarithm). The check process involves traversing all cells of the matrix, identifying the indices of these invalid values, and then determining the location of each identified missing coordinate index. The neighborhood average interpolation method is used for padding. Specifically, it involves identifying the positions of the pixels connected in the 8-neighborhood. , , , , , , , It then reads the grayscale mutual information feature values ​​from the matrix cells of these eight adjacent positions, removes the neighbors that are still invalid, calculates the average of all remaining valid neighbor values, and assigns this average to the missing position. If all eight neighbors of a missing position are invalid, the search range is expanded to 24 neighborhoods until a valid value is found or the preset maximum search radius is reached, such as 5 pixels. After completing the filling operation for all missing positions, a local mutual information distribution matrix is ​​finally generated.

[0055] The constructed set of control point coordinates for the overlay image region is first created by defining a uniform rectangular grid on the detection area of ​​the unsoldered electrode template image. For example, if the detection area is 400×300 pixels, the grid spacing can be set to 20 pixels, thus generating a 20×15 control point grid. The initial coordinates of each control point in the template image are then recorded. ,in For the grid index, then, for the map under test, for each control point, a block matching method such as Normalized Cross-Correlation (NCC) is used. The best matching block is found within its neighboring search area (e.g., a 40×40 pixel range) to obtain its initial corresponding coordinates in the image to be tested. Initial horizontal pixel displacement components and vertical pixel displacement components That is and The coordinate difference is then calculated. Next, the values ​​of the same coordinate neighborhoods within the local mutual information distribution matrix are transformed into similarity gradients, i.e., the local mutual information distribution matrix is ​​numerically differentiated to obtain the gradient field. This gradient vector indicates the direction of the fastest similarity growth. Based on this similarity gradient, the displacement components of each control point are iteratively updated. The update rule is expressed as follows: at the first... In the next iteration, the new displacement components From the previous displacement component Adding an increment related to the gradient, we get, i.e. ,in It represents the current position of the control point on the map to be measured. It is the iteration step size, a fixed positive number, for example, set to 0.5, used to control the magnitude of each update. This iteration process continues until the sum of the update amounts of all control point displacement components is less than a preset convergence threshold (e.g., 0.01 pixels) or the maximum number of iterations (e.g., 200 times) is reached. The final set of displacement components constitutes the displacement control point grid matrix.

[0056] The steps to obtain the regularization penalty stress tensor are as follows:

[0057] Read the horizontal coordinates, vertical coordinates, horizontal pixel displacement components, and vertical pixel displacement components of each control point in the displacement control point grid matrix. Establish the pairing relationship between adjacent control points according to the horizontal and vertical adjacent order respectively. Perform difference calculation on the coordinate interval of each group of adjacent control points and the difference calculation on the displacement component change of each group of adjacent control points. Map the displacement component change to the coordinate interval direction and calculate the main elongation value of adjacent control points.

[0058] The system sequentially calls the control point position index corresponding to the principal elongation value of adjacent control points, reads the elastic constraint parameter under the corresponding control point position index, performs a product operation on the principal elongation values ​​and elastic constraint parameters of adjacent control points, and writes the product operation result into the tensor row and column positions according to the horizontal and vertical arrangement order of control points to generate a regularized penalty stress tensor.

[0059] Specifically, the horizontal and vertical coordinates, horizontal pixel displacement components, and vertical pixel displacement components of each control point in the displacement control point grid matrix are read. Adjacent control points are paired according to the grid's topology. Specifically, for any non-boundary control point... It will be adjacent to the control points on its right. and the control point adjacent below Establish pairing relationships for each group of horizontally adjacent control point pairs ( and The coordinate interval is the initial horizontal spacing of the grid. Similarly, the coordinate interval between vertically adjacent control point pairs is the initial vertical spacing. Next, the difference in displacement component changes for each pair of adjacent control points is calculated. For example, for a pair of horizontally adjacent control points, the lateral difference in displacement component changes is... The longitudinal difference is Then, the displacement component changes are mapped to the coordinate interval direction to calculate the elongation. Here, the idea of ​​the Green-Lagrange strain tensor is used to calculate the elongation. For the horizontal direction, the principal elongation is... The calculation takes into account the gradient of displacement in this direction, and its approximate calculation is as follows: Similarly, for the vertical direction, the principal elongation... The approximate calculation is as follows: By traversing all adjacent control point pairs, a principal elongation value is calculated for each pair, forming a set containing principal elongation values ​​in all horizontal and vertical directions.

[0060] The system sequentially calls the control point position indices corresponding to the principal elongation values ​​of adjacent control points and reads the hyperelastic regularization penalty parameters. These hyperelastic regularization penalty parameters specifically refer to the purely mathematical penalty parameters defined when applying the hyperelastic regularization model for non-rigid image registration, i.e., the deformation penalty weights. and volume penalty weight These parameters are preset as dimensionless algorithm hyperparameters. For example, to ensure the smoothness of mesh deformation, shear penalty weights can be set. The first penalty weight is 1.0. The value is 5.0. These unitless pure values ​​are applied as global mathematical constraint parameters to all control point locations. Subsequently, the principal elongation values ​​of adjacent control points and the elastic constraint parameters are multiplied term by term to penalize the tensor. The components are related to local pixel strain. Here, a simplified regularization penalty tensor is calculated to constrain anomalous abrupt changes in the displacement field, for the principal elongation calculated along the transverse direction. Its corresponding lateral penalty component can be approximated as ,in It is a modified Jacobian determinant. In this step, to simplify the calculation, the volume change term can be temporarily ignored (i.e., (Item), then the penalty component is approximately: Similarly, the penalty component along the longitudinal direction is Shearing penalty component This is related to the shear deformation of the mesh, and all calculated penalty components (such as...) Write the row and column positions of a tensor of the same size as the control point grid, based on the corresponding control point grid positions. For example, based on the control point pairs... and Calculated Values ​​associated with grid edges The regularization penalty tensor is generated by integrating the calculation results of all adjacent control point pairs.

[0061] The steps for obtaining a two-dimensional displacement vector field are as follows:

[0062] Read the displacement vectors of control points within the displacement control point grid matrix. Match each control point displacement vector with a regularized penalty stress tensor according to the index of the same control point position. Perform divergence operation on the regularized penalty stress tensor to extract the regularized constraint force vector. Perform vector superposition and summation operation on the matched control point displacement vectors and regularized constraint force vectors. Distribute the superposition and summation operation values ​​according to the coordinate distance from all pixel positions in the detection area of ​​the tab surface to adjacent control points. Analyze the lateral displacement component and longitudinal displacement component of each pixel position to generate a two-dimensional displacement vector field.

[0063] Specifically, read the displacement vector of each control point within the displacement control point grid matrix. These displacement vectors represent the offsets of control points from their positions in the template image to their positions in the image to be tested. Simultaneously, each control point displacement vector is matched to the regularization penalty tensor generated in the previous step, based on its position index in the grid. Next, at the corresponding position, the divergence operation is performed on the regularization penalty tensor to extract the regularization constraint force vector. On a discrete control point grid, divergence calculation ( This can be achieved through the finite difference method, for example, at control points. The lateral constraint force component at a given point is calculated by combining the difference between the penalty components on its left and right sides with the difference between the shear penalty components on its upper and lower sides. This virtual constraint force vector is... This represents the mathematical smoothing penalty vector generated during the image registration iteration process to maintain the continuity of the grid topology. Subsequently, the displacement vector of each control point is... The regularized constraint force vector calculated at the same control point location By performing vector superposition and summation, a comprehensive displacement update vector is obtained. ,in It is a regularization weighting coefficient used to balance the effects of displacement driven by mutual information features and topological smoothness constraints. Based on empirical image noise levels, for example, a value of 0.1 is set to prevent excessive dominance of the smoothing constraint in the displacement field, which could mask the true deformation features of the electrode. Finally, to interpolate the displacement information at discrete control points onto the entire pixel plane, B-spline interpolation or thin-plate spline interpolation methods are used. The values ​​obtained from the above superposition and summation operation are distributed according to the coordinate distances from the pixel positions on the electrode surface detection area to their four adjacent control points. Specifically, for each pixel... final displacement It is the composite displacement update vector of its surrounding control points. The weighted average is calculated based on the distance between the pixel and each control point. Through this interpolation process, the horizontal and vertical displacement components of each pixel position are analyzed to generate a smooth, continuous, and topology-preserving two-dimensional displacement vector field covering the entire detection area.

[0064] The steps to obtain the set of partial derivative matrices are as follows:

[0065] Based on the two-dimensional displacement vector field, the lateral and longitudinal displacement components of each pixel position in the detection area of ​​the electrode surface are read one by one. According to the increasing order of the pixel lateral coordinates, the changes in the lateral and longitudinal displacement components at adjacent lateral coordinates of the same pixel position are extracted. According to the increasing order of the pixel longitudinal coordinates, the changes in the lateral and longitudinal displacement components at adjacent longitudinal coordinates of the same pixel position are extracted. The four types of displacement component changes are mapped to the pixel lateral coordinate interval and the pixel longitudinal coordinate interval, respectively. Partial derivative operations are performed, and the four partial derivative component terms are arranged to generate a set of partial derivative matrices.

[0066] Specifically, based on the two-dimensional displacement vector field, which represents each pixel within the detection region... Provide a displacement vector ,in It is the lateral displacement component. This involves the longitudinal displacement component. Then, the lateral and longitudinal displacement components of each pixel in the detection area on the electrode surface are read sequentially. To calculate the displacement gradient, the finite difference method is used to approximate the partial derivatives. Specifically, following the increasing order of the pixel's lateral coordinates, for each pixel... Extract its right-side adjacent pixels displacement components Calculate the change of the lateral displacement component along the lateral direction. and the change of the longitudinal displacement component along the transverse direction. Similarly, extract the pixels below the pixel in ascending order of their vertical coordinates. displacement components Calculate the change of the lateral displacement component along the longitudinal direction. and the change of the longitudinal displacement component along the longitudinal direction. The four types of displacement component changes are mapped to pixel coordinate intervals (in this scenario, the pixel coordinate interval is 1 unit), and partial derivatives are calculated, i.e. , , , The four calculated partial derivative components are arranged in the form of a standard displacement gradient tensor for each pixel. Construct a 2×2 partial derivative matrix Its form is This process is repeated for all pixels within the detection area to generate a set of partial derivative matrices of the same size as the image.

[0067] The steps to obtain the set of Jacobian determinants are as follows:

[0068] Based on the set of partial derivative matrices, according to the matrix row and column indices of each pixel position, read the two partial derivative components located on the main diagonal. Add the unit constant reference to each of the two partial derivative components to generate the main diagonal component of the deformed gradient. Perform pixel-by-pixel multiplication on the main diagonal component of the deformed gradient to obtain the product of the main diagonal components of the deformed gradient. Read the two partial derivative components located on the secondary diagonal and perform pixel-by-pixel multiplication to obtain the product of the secondary diagonal components. Bind the product of the main diagonal component of the deformed gradient and the product of the secondary diagonal component of the deformed gradient according to the horizontal coordinate order and the vertical coordinate order of the pixels to form the product of the diagonal partial derivative components.

[0069] Based on the product of the diagonal partial derivative components, the product of the main diagonal components of the deformed gradient corresponding to each pixel position is called one by one, and the product of the secondary diagonal partial derivative components corresponding to the same pixel position is called one by one. According to the pixel horizontal coordinate index and pixel vertical coordinate index, the product of the main diagonal components of the deformed gradient is used as the subtrahend, and the product of the secondary diagonal partial derivative components is used as the subtraction term. The difference is calculated, and the result of the difference operation at each pixel position is written into the numerical position under the same coordinate index to generate the set of Jacobian determinants.

[0070] Specifically, based on the set of partial derivative matrices, for each pixel position in the detection region... Call its corresponding 2×2 partial derivative matrix The goal of this step is to calculate the deformation gradient tensor. The components of the deformation gradient tensor are defined as follows: ,in It is a 2×2 identity matrix, with 1s on the main diagonal and 0s on the secondary diagonal. Therefore, the deformable gradient tensor... The specific form is First, read the two component terms located on the main diagonal of the partial derivative matrix. and Then, add the unit constant reference 1 to each of them to generate the main diagonal components of the deformation gradient. and The two main diagonal components are multiplied pixel by pixel to obtain the product of the main diagonal components. Next, read the two component terms at the secondary diagonal position of the partial derivative matrix. and These two components are also deformation gradient tensors. subdiagonal component and Perform pixel-by-pixel multiplication on them to obtain the product of the partial derivative components on the subdiagonal. Multiply the main diagonal components calculated for each pixel position. Multiplying the partial derivative components of the diagonal Position binding is performed according to the horizontal and vertical coordinate order of pixels, forming a product of diagonal partial derivative components.

[0071] Based on the product of the diagonal partial derivative components, this step essentially calculates the deformation gradient tensor at each pixel location. The determinant of Jacobi, i.e. This value reflects the rate of change of the local area, and is calculated by multiplying the main diagonal components of the deformation gradient corresponding to each pixel location. This value has already been calculated in the previous step. Simultaneously, the same pixel positions are called one by one. Product of corresponding diagonal partial derivative components And ensure that the pixel coordinate indices of the two maintain a one-to-one correspondence. For each pixel position, multiply the main diagonal components of the deformation gradient. As the subtracted term, the product of the partial derivative components on the secondary diagonal. As a subtraction term, the operation of subtraction and finding the difference is performed. This calculation process is the standard method for calculating a second-order determinant: , position of each pixel The result of the subtraction and difference operation completed above As a separate numerical value, it is written into a new two-dimensional array of the same size as the detection region image, and the coordinate index of its storage location is ensured. The calculation is performed by traversing all pixels within the detection area and repeating the calculation, with the coordinate index of the pixel used in the calculation being exactly the same. Finally, a complete set of values ​​containing the Jacobian determinant of each pixel is generated.

[0072] The steps for obtaining suspected connected components with poor solder joints are as follows:

[0073] Read the Jacobian determinant value of each pixel position in the Jacobian determinant value set, call the unit constant reference item by item according to the horizontal coordinate order and the vertical coordinate order of the pixels, subtract the unit constant reference from each Jacobian determinant value, take the absolute value of each difference result, write the absolute value of the difference to the value position under the same pixel coordinate index, compare the absolute value of the difference item by item with the preset anomaly limit parameter, mark the pixel position where the absolute value of the difference is less than the preset anomaly limit parameter, and obtain the pixel position mark within the anomaly limit;

[0074] Based on the pixel position markings within the abnormal boundary, the marked pixel positions are scanned row by row according to the horizontal and vertical adjacency relationships of the pixels, and the marked pixel positions are verified column by column. The continuous connection relationships in the adjacent directions of each marked pixel position are recorded. The pixel position coordinates with continuous connection relationships are grouped into the same connected region coordinates, and the individual pixel position coordinates that do not form continuous adjacency relationships are removed to generate suspected connected regions with poor soldering.

[0075] Specifically, read the Jacobian determinant value at each pixel position within the set of Jacobian determinant values. This value represents the proportion of change in local area relative to the original area after image deformation. This applies to ideal conditions without deformation. It should be equal to 1. The unit constant reference, i.e., the value 1, is called item by item according to the pixel horizontal coordinate order and pixel vertical coordinate order. The Jacobian determinant value for each pixel position is then calculated. Subtracting the unit constant reference 1 yields the difference. This difference represents the net change in area. Then, the absolute value of each difference is taken, i.e. The absolute value of this difference is then written to the numerical position under the same pixel coordinate index to form a deformation degree map. Next, each absolute value of the difference in this map is compared with a preset anomaly limit parameter. The anomaly limit parameter is set based on the statistical analysis of the small deformation of the electrode surface under normal welding process. For example, 100 normal welding samples are collected, and the deformation of their entire surface is calculated. average value and standard deviation Abnormal boundary parameters This can be set to a value slightly higher than the normal fluctuation, for example... For example, by calculation , The threshold is then set to For each pixel, if the absolute value of its difference Less than this preset abnormal threshold parameter (Right now If the deformation of the pixel position is within the normal range, it is determined and marked, thus obtaining the pixel position mark within the abnormal boundary.

[0076] Based on the pixel position markings within the anomaly boundaries, these markings form a binary image. Pixels with a value of 1 represent regions with minimal deformation and suspected ineffective fusion, while pixels with a value of 0 represent regions with normal deformation. Then, a connected component analysis algorithm is applied to identify the clustered regions of these marked pixels. First, an empty queue and an access state matrix of the same size as the marked image are initialized. Then, starting from the top left corner of the image, each pixel is scanned row by row. When a marked (value 1) pixel that has not yet been accessed is encountered, it is used as the seed point of a new connected region, its coordinates are pushed into the queue, and it is added to the current set of coordinates of the new connected region. Then, a loop is started. As long as the queue is not empty, a pixel coordinate is popped from the queue, and its 8 neighboring pixels (top, bottom, top, bottom) are checked. (Left, Right, Top Left, Top Right, Bottom Left, Bottom Right) If a neighboring pixel is also marked as 1 and has not been visited, it is marked as visited, its coordinates are pushed into the queue, and added to the current connected region coordinate set. This loop continues until the queue is empty, indicating that all marked pixels connected to the seed point have been found, forming a complete connected region. The scanning process continues until all pixels have been visited. After processing, all generated connected regions are post-processed to remove individual pixel position coordinates that do not form a valid continuous adjacent relationship, that is, connected regions with a number of pixels less than a minimum size threshold. For example, if the threshold is set to 3 pixels, any connected region with a total number of pixels less than 3 is considered noise and is removed. What is finally retained are suspected connected regions with poor solder joints.

[0077] The steps to obtain the boundary coordinate set of the cold solder joint location are as follows:

[0078] The number of pixels in each suspected poor solder joint connected region is counted one by one. The total number of pixels in the detection area of ​​the tab surface is read. The number of pixels in each suspected poor solder joint connected region is divided by the total number of pixels in the detection area of ​​the tab surface to obtain the area ratio of the corresponding connected region. The area ratio of the corresponding connected region is compared with the area tolerance threshold one by one. Possible poor solder joint connected regions that exceed the area tolerance threshold are selected. The outermost adjacent pixel position of the selected suspected poor solder joint connected region is used to extract the outer coordinate set of the connected region edge to generate the boundary coordinate set of the poor solder joint position.

[0079] Specifically, the number of pixels contained in each suspected virtual solder joint is counted one by one. This number is obtained by calculating the total number of coordinate points in the coordinate set of each connected region, denoted as . Simultaneously, the total number of pixels in the detection area on the electrode surface is read. This number is equal to the product of the width and height of the detection area image, denoted as . For each suspected connected component with a false solder joint, count the number of pixels within it. Divide by the total number of pixels in the detection area The area ratio of the connected region is calculated. Next, we will determine the area percentage of each connected region. With a preset area tolerance threshold In comparison, this threshold is set to filter out suspected areas that are too small to constitute a substantial defect. It is based on industry standards or extensive experimental data. For example, analysis of samples known to have solder joint defects reveals that the area of ​​significant solder joint defects typically accounts for no less than 0.1% of the total area. Therefore, the area tolerance threshold can be set accordingly. Set to 0.001, if the area percentage of a suspected connected component with a poor solder joint is... Greater than or equal to If the region is identified as a significant defect requiring attention, all suspected connected regions with poor solder joints exceeding the area tolerance threshold are selected. For each selected suspected connected region with poor solder joints, an edge tracking algorithm is used to traverse along the adjacent positions of the outermost pixel of the connected region. Specifically, starting from a boundary point of the region, the next pixel that constitutes the boundary is found in a clockwise or counterclockwise direction, and its coordinates are recorded until the starting point is returned. All recorded coordinate points are collected to generate the boundary coordinate set of the poor solder joint position.

[0080] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for detecting poor solder joints on electrode tabs based on grayscale differences, characterized in that, Includes the following steps: Collect the unsoldered template image and the image to be tested of the electrode, perform grayscale level difference comparison calculation on the unsoldered template image and the image to be tested of the electrode, and generate a local mutual information distribution matrix; Construct a set of grid control point coordinates covering the image region, calculate the pixel displacement vectors corresponding to the grid control points, construct a similarity gradient by combining the values ​​in the local mutual information distribution matrix, and iteratively generate the displacement control point grid matrix. Calculate the principal elongation values ​​of adjacent control points within the displacement control point grid matrix, and generate a regularized penalty stress tensor. The regularized penalty stress tensor is converted into a regularized constraint force vector by performing divergence operation. The displacement vectors of the control points in the displacement control point grid matrix are then superimposed with the regularized constraint force vector to generate a two-dimensional displacement vector field. Partial derivatives are calculated along the horizontal and vertical coordinate axes for the displacement components of each pixel position in the two-dimensional displacement vector field to generate a set of partial derivative matrices. Based on the set of partial derivative matrices, calculate and generate the set of Jacobian determinants. The values ​​in the Jacobian determinant set are subtracted from the unit constant benchmark. The coordinates of the connected regions of consecutive adjacent pixels whose absolute difference is less than a preset abnormal boundary parameter are extracted to generate a suspected connected region of a poor solder joint. Based on the suspected connected region of a poor solder joint, the boundary coordinate set of the poor solder joint position is obtained.

2. The method for detecting poor solder joints on electrode tabs based on grayscale differences according to claim 1, characterized in that, The steps for obtaining the displacement control point grid matrix are as follows: Acquire an unsoldered template image of the electrode and an image of the electrode to be tested. Read the gray level value of each pixel position in the unsoldered template image of the electrode and read the gray level value of the same pixel position in the image of the electrode to be tested. Pair the gray level values ​​of the same pixel position point by point. According to the correspondence between the horizontal coordinate and the vertical coordinate of the pixel, perform gray level difference comparison calculation item by item and extract the gray level mutual information feature value at the corresponding position. According to the pixel horizontal coordinate order and pixel vertical coordinate order of the detection area on the electrode surface, the gray-level mutual information feature value of each corresponding position is written into the matrix unit under the same coordinate index. The row and column correspondence between the matrix unit and the pixel position is verified. The gray-level mutual information feature value of adjacent pixels is used to fill in the missing coordinate index position to generate a local mutual information distribution matrix. A set of grid control point coordinates covering the image region is constructed. The pixel coordinates of each grid control point in the unwelded template image of the tab are read one by one. The pixel coordinates of each grid control point in the image to be tested of the tab are read one by one. The horizontal and vertical pixel displacement components corresponding to the same grid control point are calculated. The values ​​of the neighborhood of the same coordinates in the local mutual information distribution matrix are converted into similarity gradients. The horizontal and vertical pixel displacement components are iteratively updated according to the similarity gradients to generate the displacement control point grid matrix.

3. The method for detecting poor solder joints on electrode tabs based on grayscale differences according to claim 1, characterized in that, The steps for obtaining the regularization penalty stress tensor are as follows: Read the horizontal coordinates, vertical coordinates, horizontal pixel displacement components, and vertical pixel displacement components of each control point in the displacement control point grid matrix. Establish pairing relationships between adjacent control points according to the horizontal and vertical adjacent order. Perform difference calculation on the coordinate interval of each group of adjacent control points and on the displacement component change of each group of adjacent control points. Map the displacement component change to the coordinate interval direction and calculate the main elongation value of adjacent control points. The control point position index corresponding to the principal elongation value of the adjacent control points is called item by item, the elastic constraint parameter under the corresponding control point position index is read, the principal elongation value of the adjacent control points and the elastic constraint parameter are multiplied item by item, and the product operation result is written into the tensor row and column positions according to the horizontal arrangement order and the vertical arrangement order of the control points to generate a regularized penalty stress tensor.

4. The method for detecting poor solder joints on electrode tabs based on grayscale differences according to claim 1, characterized in that, The steps for obtaining the two-dimensional displacement vector field are as follows: Read the displacement vectors of the control points within the displacement control point grid matrix, match each control point displacement vector with the regularized penalty stress tensor according to the same control point position index, perform divergence operation on the regularized penalty stress tensor to extract the regularized constraint force vector, perform vector superposition and summation operation on the matched control point displacement vectors and regularized constraint force vectors, distribute the superposition and summation operation values ​​according to the coordinate distance from all pixel positions in the detection area of ​​the tab surface to adjacent control points, and analyze the lateral displacement component and longitudinal displacement component of each pixel position to generate a two-dimensional displacement vector field.

5. The method for detecting poor solder joints on electrode tabs based on grayscale differences according to claim 1, characterized in that, The steps for obtaining the set of partial derivative matrices are as follows: Based on the two-dimensional displacement vector field, the lateral displacement component and the longitudinal displacement component of each pixel position in the detection area of ​​the tab surface are read one by one. According to the increasing order of the pixel lateral coordinate, the changes in the lateral displacement component and the longitudinal displacement component at adjacent lateral coordinates of the same pixel position are extracted. According to the increasing order of the pixel longitudinal coordinate, the changes in the lateral displacement component and the longitudinal displacement component at adjacent longitudinal coordinates of the same pixel position are extracted. The four types of displacement component changes are mapped to the pixel lateral coordinate interval and the pixel longitudinal coordinate interval, respectively. Partial derivative calculations are performed, and the four partial derivative component terms are arranged to generate a set of partial derivative matrices.

6. The method for detecting poor solder joints on electrode tabs based on grayscale differences according to claim 1, characterized in that, The steps for obtaining the set of Jacobian determinants are as follows: Based on the set of partial derivative matrices, according to the matrix row and column indices of each pixel position, read the two partial derivative component terms located at the main diagonal position. Add the unit constant reference to the two partial derivative component terms to generate the deformation gradient main diagonal component. Perform pixel-by-pixel multiplication on the deformation gradient main diagonal component to obtain the deformation gradient main diagonal component product. Read the two partial derivative component terms located at the secondary diagonal position and perform pixel-by-pixel multiplication to obtain the secondary diagonal partial derivative component product. Bind the deformation gradient main diagonal component product and the secondary diagonal partial derivative component product according to the pixel horizontal coordinate order and pixel vertical coordinate order to form the diagonal partial derivative component product. Based on the product of the diagonal partial derivative components, the product of the main diagonal components of the deformed gradient corresponding to each pixel position is called one by one, and the product of the secondary diagonal partial derivative components corresponding to the same pixel position is called one by one. According to the pixel horizontal coordinate index and pixel vertical coordinate index, the product of the main diagonal components of the deformed gradient is used as the subtrahend, and the product of the secondary diagonal partial derivative components is used as the subtraction term. The difference is calculated, and the result of the difference operation at each pixel position is written into the numerical position under the same coordinate index to generate the Jacobian determinant numerical set.

7. The method for detecting poor solder joints of electrode tabs based on grayscale differences according to claim 1, characterized in that, The steps for obtaining the suspected connected components with poor solder joints are as follows: Read the Jacobian determinant value of each pixel position in the set of Jacobian determinant values, call the unit constant reference item by item according to the horizontal coordinate order and the vertical coordinate order of the pixels, subtract the unit constant reference from each Jacobian determinant value, take the absolute value of each difference result, write the absolute value of the difference to the value position under the same pixel coordinate index, compare the absolute value of the difference item by item with the preset abnormality limit parameter, mark the pixel position where the absolute value of the difference is less than the preset abnormality limit parameter, and obtain the pixel position mark within the abnormality limit; Based on the pixel position markings within the abnormal boundary, the marked pixel positions are scanned row by row according to the horizontal and vertical adjacency relationships of the pixels, and the marked pixel positions are verified column by column. The continuous connection relationships in the adjacent directions of each marked pixel position are recorded. The pixel position coordinates with continuous connection relationships are grouped into the same connected region coordinates, and the individual pixel position coordinates that do not form continuous adjacency relationships are removed to generate suspected connected regions with poor soldering.

8. The method for detecting poor solder joints of electrode tabs based on grayscale differences according to claim 1, characterized in that, The steps for obtaining the boundary coordinate set of the cold solder joint location are as follows: The number of pixels in each suspected poor solder joint connected region is counted one by one. The total number of pixels in the detection area of ​​the tab surface is read. The number of pixels in each suspected poor solder joint connected region is divided by the total number of pixels in the detection area of ​​the tab surface to obtain the area ratio of the corresponding connected region. The area ratio of the corresponding connected region is compared with the area tolerance threshold one by one. The suspected poor solder joint connected regions that exceed the area tolerance threshold are selected. The outermost adjacent pixel position of the selected suspected poor solder joint connected region is used to extract the outer coordinate set of the connected region edge to generate the boundary coordinate set of the poor solder joint position.

9. The system for detecting electrode solder joint defects based on grayscale differences according to any one of claims 1-8, characterized in that, include: Image acquisition and registration module: used to acquire the unsoldered template image of the electrode and the image to be tested of the electrode, perform grayscale level difference comparison calculation on the unsoldered template image of the electrode and the image to be tested of the electrode, generate a local mutual information distribution matrix, construct a set of grid control point coordinates covering the image area, calculate the pixel displacement vector corresponding to the grid control point, construct a similarity gradient by combining the values ​​in the local mutual information distribution matrix, and iteratively generate the displacement control point grid matrix. Displacement vector field generation module: used to calculate the principal elongation values ​​of adjacent control points in the displacement control point grid matrix, generate a regularized penalty stress tensor, perform divergence operation on the regularized penalty stress tensor to convert it into a regularized constraint force vector, and perform vector superposition operation on the displacement vector of the control point in the displacement control point grid matrix and the regularized constraint force vector to generate a two-dimensional displacement vector field; Jacobian determinant calculation module: used to perform partial derivative calculations on the displacement components of each pixel position in the two-dimensional displacement vector field along the horizontal and vertical coordinate axes, generate a set of partial derivative matrices, and calculate and generate a set of Jacobian determinant values ​​based on the set of partial derivative matrices; The cold solder joint area identification module is used to calculate the difference between each value in the Jacobian determinant set and the unit constant benchmark, extract the coordinates of the connected regions of consecutive adjacent pixels whose absolute difference is less than a preset abnormal boundary parameter, generate a suspected cold solder joint connected region, and obtain the cold solder joint location boundary coordinate set based on the suspected cold solder joint connected region.

Images