High-precision production line sensor fast positioning method based on an industrial camera

Abstract

This invention discloses a high-precision production line sensor rapid positioning method based on an industrial camera, relating to the technical field of industrial cameras. The method includes: using an industrial camera to capture an image of a test object from any angle under visible light; filtering all pixels in the image and calculating the gradient magnitude of each pixel, designating pixels with gradient magnitudes greater than or equal to a preset weak edge gradient magnitude threshold as foreground points; rapidly fitting the filtering result using a facet model to obtain sub-pixel points that meet the conditions, designating them as edge points; traversing all edge points to find and connect ridges, recording the coordinates of all edge points on the ridges; transferring all obtained ridge information and the coordinates of all used edge points to a robotic arm on the production line; and the robotic arm operating the industrial camera sensor based on the received information to complete sensor positioning.

Description

Technical Field

[0001] This invention relates to the technical field of industrial cameras, and more specifically, to a method for rapid positioning of high-precision production line sensors based on industrial cameras. Background Technology

[0002] Inspection technology is one of the fundamental technologies of modern industry. The effectiveness of locating, measuring, or fitting geometric features of targets directly affects the quality control of workpieces or assemblies. The higher the accuracy of edge feature detection, the higher the manufacturing precision and the higher the level of defect control. Therefore, edge technology plays a very important role in industrial inspection processes, especially in high-precision workpiece inspection such as sensors. The higher the sensor contour positioning accuracy, the higher the yield rate.

[0003] On a sensor manufacturing production line, sensor positioning is achieved by acquiring images through industrial cameras. A host computer extracts edge contours from these images, using the edge information to locate the sensor. A robotic arm then processes the sensor based on its position coordinates. Throughout the entire process, the speed and accuracy of the sensor edge information directly impact production line efficiency. In traditional machine vision measurement, acquiring higher-resolution images and then performing sub-pixel edge extraction improves the accuracy of fitting, registration, and defect control.

[0004] The conventional approach to subpixel edge extraction involves first extracting pixel edges, then extracting subpixel edges. This method separates the extraction of subpixel information into two parts: pixel edge extraction and subpixel edge extraction. This process introduces two types of problems: 1. Pixel edges may have multiple pixels with the same gradient value in their approximate eight-neighbor gradient direction. However, subpixel edges only appear on edge gradient ridges, and there are no cases where multiple gradient points exist in the normal direction of the ridge. Therefore, there are accuracy issues with wide edge points with multiple pixels. 2. The two extraction steps undoubtedly increase the computational load of the algorithm. For industrial production lines requiring high-resolution defect detection, the above method directly impacts the operating efficiency of the production line. Summary of the Invention

[0005] The purpose of this invention is to provide a production line sensor localization method that can solve the problem of coarse edges in the sensor contours extracted from the production line and has a fast sub-pixel edge detection speed.

[0006] The technical solution of this invention is: to provide a high-precision production line sensor rapid positioning method based on an industrial camera, the method comprising:

[0007] S1. Use an industrial camera to capture an image of the test object from any angle under visible light using a sensor;

[0008] S2. Filter all pixels in the image and calculate the gradient magnitude Amp of each pixel. Set the pixels with gradient magnitude Amp greater than or equal to the preset weak edge gradient magnitude threshold as the foreground pixels.

[0009] Performing a second-order Taylor expansion of the two-dimensional gradient image f(x, y) at (x0, y0) yields the first and second partial derivatives g at the foreground point. x g y g xx g yy and g xy ;

[0010] S3. The partial derivatives above are calculated using the facet model:

[0011]

[0012] The normal vector n of the foreground point is represented as (n x n y The normal vector n has a magnitude of 1. The subpixel coordinates of the foreground point are calculated by Taylor expansion of the gradient of the foreground point in the normal direction.

[0013] The maxima and minima of the second derivative of the image correspond to the two eigenvalues ​​of the Hessian matrix, respectively. The normal vector n of the foreground point can be obtained by finding the eigenvector corresponding to the largest eigenvalue of the Hessian matrix. The Hessian matrix is ​​represented as:

[0014]

[0015] S4. Connect the sub-pixel edge points. First, construct two auxiliary tables: the selection range table and the selection record table. The selection range table includes the coordinates of all sub-pixel edge points within 1 / 2 pixel range of the foreground point, and the selection record table records the coordinates of all extracted sub-pixel edge points.

[0016] The traversal starts from the first edge point in the selection range table that meets the traversal conditions, and searches for the ridge line. The traversal rules include: the selected current point must be included in the selection range table and not appear in the selection record table.

[0017] After selecting a new current point, calculate the loss function Δloss for the pixels surrounding the current point as a candidate point, and select the point with the smallest loss function Δloss as the second point of the ridge line; the formula for calculating the loss function Δloss is as follows:

[0018]

[0019] Where ||p1,p2||2 is the L2 distance between the current point and the candidate point, and ||p1,p2||2 < 0.5, ||p1,p2||2 = 100; The pixel gradient weights between the current point and the candidate points are... is the weight of the largest eigenvalue of the Hessian matrix, where thresh = 0.5;

[0020] Once a ridge line is connected, all edge points of that ridge line are written into the point selection record table.

[0021] S5. Transfer all the obtained ridge lines and the coordinate information of all the edge points used to the robotic arm on the production line;

[0022] S6. The robotic arm operates on the industrial camera's sensor based on the received information to complete sensor positioning.

[0023] In any of the above technical solutions, further, the method for calculating the sub-pixel coordinates of the foreground point in step S3 includes: performing a Taylor expansion of the gradient of the foreground point in the normal direction:

[0024]

[0025] For edge points, let We can obtain:

[0026]

[0027] The sub-pixel coordinates of the foreground point are:

[0028] (p x p y )=(tn x +x0, tn y +y0).

[0029] In any of the above technical solutions, the method for determining candidate points in step S4 further includes:

[0030] When searching for the second point of the current ridge, the loss function Δloss is calculated for the eight adjacent pixels around the starting point in eight directions, and the point corresponding to the minimum value of the eight loss functions Δloss is taken as the second point.

[0031] Starting from the search for the third point, only calculate the loss function Δloss of the forward direction of the previous point and the two adjacent pixels of the two adjacent directions in the eight directions. The point corresponding to the minimum value of the three loss functions Δloss is taken as the next point.

[0032] In any of the above technical solutions, further, in step S4, the stopping conditions for connecting edge points include: the candidate point with the smallest loss function Δloss appears in the point selection record table, at which point the last found candidate point is used as the termination point and connected to the previous point, so that the two ridge lines are connected to each other; the candidate point with the smallest loss function Δloss is located outside the image boundary.

[0033] In any of the above technical solutions, further, after the edge point connection stops in step S4, if it is determined that the ridge termination point and the starting point are not the same point, a reverse connection is made from the termination point to the starting point.

[0034] In any of the above technical solutions, further, in step S2, Sobel filtering is used for filtering all pixels of the image. The gradient magnitude Amp and angle Dir of the Sobel filter are calculated as follows:

[0035] Amp = (|dx| + |dy|) / 4

[0036] Dir = atan2(dy, dx)

[0037] If the gradient magnitude Amp of a pixel is less than the preset weak edge gradient magnitude threshold, then the angle Dir is not calculated for that pixel.

[0038] In any of the above technical solutions, further, the formula for performing a second-order Taylor expansion of the two-dimensional gradient image f(x, y) at (x0, y0) in step S2 is as follows:

[0039]

[0040] Where g x and g y Let g be the first partial derivative on the foreground point. xx g yy and g xy Let be the second partial derivative of the foreground point.

[0041] In any of the above technical solutions, further, in step S3, during the facet model calculation of the partial derivatives, g x g y g xx and g yy The convolution template coefficients are all 1 / 6, g xy The convolution template coefficient is 1 / 4.

[0042] In any of the above technical solutions, further, the sub-pixel precision input value in step S3 is 1 / 2, that is:

[0043] |tn x |≤0.5,|tn y|≤0.5.

[0044] In any of the above technical solutions, the point range table in step S4 is further constructed using run-length encoding.

[0045] The beneficial effects of this invention are:

[0046] The technical solution of this invention calculates the image gradient using common simple edge detection operators, performs threshold segmentation on the gradient map using a preset threshold to obtain the foreground region, calculates the sub-pixel information of the foreground points, and dynamically extracts edge ridges using a joint loss function. This solution integrates pixel-level and sub-pixel-level edge extraction into a single operation, and the use of a joint loss function will prevent the appearance of "thick" ridges. Attached Figure Description

[0047] The advantages of the above and additional aspects of the present invention will become apparent and readily understood in the description of the embodiments in conjunction with the following drawings, wherein:

[0048] Figure 1 This is a flowchart of a high-precision production line sensor rapid positioning method based on an industrial camera according to an embodiment of the present invention;

[0049] Figure 2 This is a Sobel filter template for a high-precision production line sensor rapid positioning method based on an industrial camera, according to an embodiment of the present invention.

[0050] Figure 3 This is a facet convolution template for a high-precision production line sensor rapid positioning method based on an industrial camera, according to an embodiment of the present invention.

[0051] Figure 4 This is a pixel numbering diagram of a high-precision production line sensor rapid positioning method based on an industrial camera according to an embodiment of the present invention.

[0052] Figure 5 This is a schematic diagram illustrating the numbering of subsequent pixels in a high-precision production line sensor rapid positioning method based on an industrial camera according to an embodiment of the present invention.

[0053] Figure 6 This is a flowchart of an edge point connection method for a high-precision production line sensor rapid positioning method based on an industrial camera, according to an embodiment of the present invention. Detailed Implementation

[0054] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0055] In the following description, many specific details are set forth in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0056] like Figure 1 As shown, this embodiment provides a high-precision production line sensor rapid positioning method based on an industrial camera. The method includes:

[0057] S1. Use an industrial camera to take an image of the test object from any angle under visible light using a sensor.

[0058] In this embodiment, the industrial camera on the production line is a visible light band area array industrial camera that can communicate with a host computer. The method provided in this embodiment is applicable to the vast majority of industrial cameras.

[0059] S2. Filter the image captured in step S1. In this embodiment, fast Sobel calculation is used to filter all pixels of the image. The Sobel filtering template is as follows: Figure 2 As shown, the gradient magnitude Amp and angle Dir of the Sobel filter are calculated as follows:

[0060] Amp = (|dx| + |dy|) / 4

[0061] Dir = atan2(dy, dx)

[0062] First, calculate the gradient magnitude Amp of the pixel. Compare the calculation result with the preset weak edge gradient magnitude threshold. If the gradient magnitude Amp of the pixel is less than the preset threshold, do not calculate the angle Dir for that point. Pixels with gradient magnitude Amp greater than or equal to the preset threshold are set as foreground points.

[0063] Subpixel information is calculated for the foreground point, and a second-order Taylor expansion is performed on the two-dimensional gradient image f(x, y) at (x0, y0):

[0064]

[0065] Where g x g y g xx g yy and g xy These are the first and second partial derivatives at the foreground point, respectively.

[0066] S3. The first and second partial derivatives g on the foreground point obtained in step S2. x g y g xxg yy and g xy The computation is performed using a facet model, and the convolution template of the facet model is as follows: Figure 3 As shown, where g x g y g xx and g yy The coefficients of the template are all 1 / 6, g xy The template coefficient is 1 / 4, and the calculation result of the facet model is:

[0067]

[0068] The normal vector n of the foreground point is represented as (n x n y The normal vector n has a magnitude of 1. The Taylor expansion of the gradient of the foreground point along the normal direction is:

[0069]

[0070] For edge points, let We can obtain:

[0071]

[0072] The sub-pixel coordinates of the foreground point are:

[0073] (p x p y )=(tn x +x0, tn y +y0)

[0074] In this embodiment, the sub-pixel precision input value is 1 / 2, that is:

[0075] |tn x |≤0.5,|tn y |≤0.5.

[0076] The maxima and minima of the second derivative of the image correspond to the two eigenvalues ​​of the Hessian matrix, respectively. Therefore, the normal vector n of the foreground point can be obtained by finding the eigenvector corresponding to the largest eigenvalue of the Hessian matrix. The Hessian matrix is ​​represented as:

[0077]

[0078] S4. Connect the sub-pixel edge points. First, construct two auxiliary tables: the selection range table and the selection record table. The selection range table includes the coordinates of all sub-pixel edge points within 1 / 2 pixel range of the foreground point, and the selection record table records the coordinates of all extracted sub-pixel edge points. The selection range table is constructed using run-length encoding.

[0079] The traversal begins from the first edge point in the selection range table that meets the traversal conditions, searching for ridge lines. The traversal rules include: the selected point must be included in the selection range table and not appear in the selection record table.

[0080] Using edge points that meet the traversal conditions as starting points, calculate the loss function Δloss for the eight pixels surrounding the starting point, and select the point with the smallest loss function Δloss as the second point of the ridge line and connect them. The formula for calculating the loss function Δloss is as follows:

[0081]

[0082] Where ||p1,p2||2 is the L2 distance between the current point and the candidate point, and ||p1,p2||2 < 0.5, ||p1,p2||2 = 100; The pixel gradient weights between the current point and the candidate points are... is the weight of the largest eigenvalue of the Hessian matrix, where thresh = 0.5.

[0083] Starting with the search for the third point, since the edge contours of the image do not undergo excessive curvature at the pixel level, such as... Figure 4 As shown, the eight neighboring pixels around a pixel are numbered. The loss function Δloss is calculated only for the pixels corresponding to the previous pixel and its two adjacent pixels (a total of three numbers). For example, if the second point is pixel number 'a' of the first point, finding the third point only requires calculating the loss function Δloss for the pixels in the b, a, and h directions of the second point. Figure 5 As shown, if the first point is taken as a reference, the selection range of the third point in the above example is three pixels: b1, a1, and h1.

[0084] The stopping conditions for connecting edge points include: the candidate point with the minimum loss function Δloss appears in the point selection record table, and the last candidate point found is used as the termination point and connected to the previous point to connect the two ridge lines; the candidate point with the minimum loss function Δloss is located outside the image boundary.

[0085] like Figure 5 As shown, the overall process of connecting edge points also includes reverse connection: after finding the end point of a ridge, if the end point is not the same as the starting point of the ridge, then start from the end point and connect in reverse once. The purpose of this process is to connect the starting point to other ridges as well. After a ridge is connected, all edge points of the ridge are written into the point selection record table.

[0086] S5. Transfer all the ridge lines and edge point coordinates obtained in step S4 to the robotic arm on the production line.

[0087] S6. The robotic arm operates on the industrial camera's sensor based on the received information to complete sensor positioning.

[0088] In summary, this invention proposes a high-precision production line sensor rapid positioning method based on industrial cameras, comprising:

[0089] S1. Use an industrial camera to take an image of the test object from any angle under visible light using a sensor.

[0090] S2. Filter all pixels in the image and calculate the gradient magnitude Amp of each pixel. Set the pixels with gradient magnitude Amp greater than or equal to the preset weak edge gradient magnitude threshold as the foreground pixels.

[0091] Performing a second-order Taylor expansion of the two-dimensional gradient image f(x, y) at (x0, y0) yields the first and second partial derivatives g at the foreground point. x g y g xx g yy and g xy .

[0092] S3. The partial derivatives above are calculated using the facet model:

[0093]

[0094] The normal vector n of the foreground point is represented as (n x n y The normal vector n has a magnitude of 1. The Taylor expansion of the gradient of the foreground point along the normal direction is:

[0095]

[0096] For edge points, let We can obtain:

[0097]

[0098] The sub-pixel coordinates of the foreground point are:

[0099] (p x p y )=(tn x +x0,th y +y0)

[0100] The maxima and minima of the second derivative of the image correspond to the two eigenvalues ​​of the Hessian matrix, respectively. The normal vector n of the foreground point can be obtained by finding the eigenvector corresponding to the largest eigenvalue of the Hessian matrix. The Hessian matrix is ​​represented as:

[0101]

[0102] S4. Connect the sub-pixel edge points. First, construct two auxiliary tables: the selection range table and the selection record table. The selection range table includes the coordinates of all sub-pixel edge points within 1 / 2 pixel range of the foreground point, and the selection record table records the coordinates of all extracted sub-pixel edge points.

[0103] The traversal begins from the first edge point in the selection range table that meets the traversal conditions, searching for the ridge line. The traversal rules include: the selected current point must be included in the selection range table and not appear in the selection record table.

[0104] After selecting a new current point, calculate the loss function Δloss for the pixels surrounding the current point as a candidate point, and select the point with the smallest loss function Δloss as the second point of the ridge line; the formula for calculating the loss function Δloss is as follows:

[0105]

[0106] Where ||p1,p2||2 is the L2 distance between the current point and the candidate point, and ||p1,p2||2 < 0.5, ||p1,p2||2 = 100; The pixel gradient weights between the current point and the candidate points are... is the weight of the largest eigenvalue of the Hessian matrix, where thresh = 0.5.

[0107] Once a ridge line is connected, all edge points of that ridge line are written into the point selection record table.

[0108] S5. Transfer all obtained ridge lines and the coordinate information of all used edge points to the robotic arm on the production line.

[0109] S6. The robotic arm operates on the industrial camera's sensor based on the received information to complete sensor positioning.

[0110] The steps in this invention can be adjusted, combined, or deleted according to actual needs.

[0111] Although the invention has been disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and not intended to limit the application of the invention. The scope of protection of the invention is defined by the appended claims and may include various variations, modifications, and equivalents made to the invention without departing from the scope and spirit of the invention.

Claims

1. A high-precision production line sensor rapid positioning method based on industrial cameras, characterized in that, The method includes: S1. Use an industrial camera to take an image of the test object from any angle under visible light using a sensor; S2. Filter all pixels of the image and calculate the gradient magnitude of each pixel. , gradient magnitude Pixels with a weak edge gradient magnitude greater than or equal to a preset threshold are set as foreground pixels; Two-dimensional gradient image exist A second-order Taylor expansion at the point yields the first and second partial derivatives at the point. , , , and ; S3. The first and second partial derivatives of the above foreground points are calculated using the facet model: ； The normal vector n of the foreground point is denoted as The normal vector n has a magnitude of 1. The sub-pixel coordinates of the foreground point are calculated by performing a Taylor expansion of the gradient of the foreground point along the normal direction. The maxima and minima of the second derivative of the image correspond to the two eigenvalues ​​of the Hessian matrix. The normal vector n of the foreground point is obtained by finding the eigenvector corresponding to the largest eigenvalue of the Hessian matrix. The Hessian matrix is ​​represented as: ； S4. Connect the sub-pixel edge points. First, construct two auxiliary tables: a selection range table and a selection record table. The selection range table includes the coordinates of all sub-pixel edge points within 1 / 2 pixel range of the foreground point, and the selection record table records the coordinates of all extracted sub-pixel edge points. The traversal starts from the first edge point in the selected point range table that meets the traversal conditions, and searches for the ridge line. The traversal conditions include: the selected current point must be included in the selected point range table and not appear in the selected point record table. After selecting a new current point, calculate the loss function of the pixels surrounding the current point as a candidate point. , loss function The point with the smallest value is taken as the second point of the ridge; loss function The calculation formula is as follows: ； in Let L be the norm distance between the current point and the candidate point, and hour, ; The pixel gradient weights between the current point and the candidate points. The weights of the largest eigenvalues ​​of the Hessian matrix are denoted as , where . ; Once a ridge line is connected, all edge points of that ridge line are written into the point selection record table. S5. Transfer all the obtained ridge lines and the coordinate information of all the edge points used to the robotic arm on the production line; S6. The robotic arm operates on the sensor of the industrial camera based on the received information to complete the sensor positioning.

2. The high-precision production line sensor rapid positioning method based on industrial cameras as described in claim 1, characterized in that, The method for calculating the sub-pixel coordinates of the foreground point in step S3 includes: performing a Taylor expansion of the gradient of the foreground point along the normal direction. ； For edge points, let We can obtain: ； The sub-pixel coordinates of the foreground point are: 。 3. The high-precision production line sensor rapid positioning method based on industrial cameras as described in claim 1, characterized in that, The method for determining candidate points in step S4 includes: When searching for the second point of the current ridge, the loss function is calculated for all neighboring pixels in eight directions around the starting point. Eight loss functions The point corresponding to the minimum value is taken as the second point; Starting from the search for the third point, only the loss function of the forward direction from the previous point and the two adjacent pixels in the two adjacent directions is calculated among the eight directions. The three loss functions The point corresponding to the minimum value is taken as the next point.

4. The high-precision production line sensor rapid positioning method based on industrial cameras as described in claim 1, characterized in that, In step S4, the stopping condition for connecting edge points includes: loss function. The smallest candidate point appears in the selected point record table. The last candidate point found is then used as the termination point, and a line is drawn from the previous point to connect the two ridge lines. Loss function. The smallest candidate point is located outside the image boundary.

5. The high-precision production line sensor rapid positioning method based on industrial cameras as described in claim 4, characterized in that, After the edge point connection stops in step S4, if it is determined that the ridge line termination point and the starting point are not the same point, a reverse connection is made from the termination point to the starting point.

6. The high-precision production line sensor rapid positioning method based on industrial cameras as described in claim 1, characterized in that, In step S2, Sobel filtering is applied to all pixels of the image, and the gradient magnitude of the Sobel filter is... and angle The calculation method is as follows: ； ； If the gradient magnitude of this pixel If the gradient magnitude is less than the preset threshold for weak edge gradients, the angle will not be calculated for that point. .

7. The high-precision production line sensor rapid positioning method based on industrial cameras as described in claim 1, characterized in that, In step S2, the two-dimensional gradient image is... exist The expression for the second-order Taylor expansion at point is: ； in and Let be the first partial derivative on the foreground point. , and Let be the second partial derivative of the foreground point.

8. The high-precision production line sensor rapid positioning method based on industrial cameras as described in claim 1, characterized in that, In step S3, during the facet model calculation of the partial derivatives... , , and The convolution template coefficients are all 1 / 6. The convolution template coefficient is 1 / 4.

9. The high-precision production line sensor rapid positioning method based on industrial cameras as described in claim 2, characterized in that, The constraints on the sub-pixel coordinates of the foreground point in step S3 include: 。 10. The high-precision production line sensor rapid positioning method based on industrial cameras as described in claim 1, characterized in that, The point selection range table in step S4 is constructed using run-length encoding.

Images