A edge extraction method based on theoretical information

Abstract

The present invention discloses an edge extraction method based on theoretical information, comprising: converting the theoretical information of feature A in a digital model to the camera coordinate system according to a pre-calibrated conversion relationship between the coordinate system of the object to be measured and the camera coordinate system; obtaining a plurality of sampling points in the camera coordinate system using the converted theoretical information and a preset sampling step, wherein the sampling points are all located on feature A, and calculating the gradient direction of each sampling point; searching for edge point coordinates along the gradient direction of a single sampling point coordinate as the center, and traversing all sampling points; storing the coordinates of all searched edge points as the edge information of feature A; the method uses the theoretical information in the digital model to assist in locating the position of the edge point, and can obtain the position of the real edge point to be measured more accurately and quickly, meeting the detection accuracy requirements of the high-precision processing and manufacturing industry; and is suitable for edge extraction of circular holes, slot holes, polygonal holes, and edges.

Description

Technical Field

[0001] The present invention relates to the field of edge detection, and in particular to an edge extraction method based on theoretical information. Background Art

[0002] With the continuous development of intelligent manufacturing, the requirements for the measurement and positioning of geometric features (holes, edges and lines) on workpieces are constantly increasing. Traditional methods for measuring geometric features include three-coordinate measurement, trackers, and lidar. As visual measurement products continue to mature, measurement methods based on visual detection are gradually being widely used. The measurement process includes: acquiring images, extracting edge point information of features in the image, fitting features using edge point information, and obtaining key point information of features (such as hole center coordinates, radius, starting position of edges, etc.). Among them, the accuracy of edge point information extraction directly affects the accuracy of feature fitting. Traditional edge extraction methods rely solely on the acquired image information and obtain edges based on the texture and grayscale information of the image. They are easily interfered by background or noise. When the image contrast is poor and the texture is not rich, the accuracy of the edge detection results is poor. Summary of the Invention

[0003] To address the above technical issues, the present invention provides an edge extraction method based on theoretical information. Compared with traditional edge extraction algorithms, this method uses theoretical information from digital models to assist in locating the location of edge points. This method can more accurately and quickly obtain the location of the actual edge points to be measured. For images with low contrast and little texture detail, the edge detection results can still remain effective, meeting the detection accuracy requirements of high-precision manufacturing. This method is suitable for edge extraction of circular holes, slots, polygonal holes (rectangular holes, triangles, parallelograms, pentagons, etc.), and edge features.

[0004] To this end, the technical solutions of the present invention are as follows:

[0005] A method for edge extraction based on theoretical information is applicable to the acquisition of surface data of an object to be measured as a three-dimensional point cloud. The following steps are used to extract edge information of a surface feature A of the object to be measured, where the feature A is composed of one or more of a circle, an arc, and a straight line:

[0006] S1. According to the pre-calibrated conversion relationship between the coordinate system of the object to be measured and the camera coordinate system, the theoretical information of feature A in the digital model is converted to the camera coordinate system;

[0007] The theoretical information of the circle or arc segment of feature A in the mathematical model includes the coordinates of the center, the normal vector and the radius;

[0008] The theoretical information of the straight line segment of feature A in the mathematical model includes the starting point, length, direction vector and normal vector of the straight line;

[0009] S2. In the camera coordinate system, using the converted theoretical information and a preset sampling step size, obtain multiple sampling points, where all of the sampling points are located on feature A;

[0010] Calculate the first-order derivative of each sampling point to obtain the tangential direction of each sampling point, and then obtain the gradient direction of each point by cross-multiplying the normal vector and each tangential direction;

[0011] S3. Search for points in the 3D point cloud within a preset search range, with the coordinates of a single sampling point as the center and the positive and negative directions of the corresponding gradient directions as the search directions. If a point is found, the first point found is recorded as the edge point coordinate; otherwise, it is considered that there is no edge point here, and the search is continued with the next sampling point as the center until all sampling points are traversed;

[0012] The coordinates of all searched edge points are stored as the edge information of feature A.

[0013] When acquiring a two-dimensional image of the surface of the object to be measured, the present invention further discloses another edge extraction method based on theoretical information, which is applicable to the case where the acquired surface data of the object to be measured is a two-dimensional image. The edge information of a feature A on the surface of the object to be measured is extracted using the following steps, where the feature A is composed of one or more of a circle, an arc, and a straight line:

[0014] Step 1: Based on the pre-calibrated conversion relationship between the coordinate system of the object to be measured and the camera coordinate system, the theoretical information of feature A in the digital model is converted to the camera coordinate system;

[0015] The theoretical information of the circle or arc segment of feature A in the mathematical model includes the coordinates of the center, the normal vector and the radius;

[0016] The theoretical information of the straight line segment of feature A in the mathematical model includes the starting point, length, direction vector and normal vector of the straight line;

[0017] Step 2: In the camera coordinate system, using the converted theoretical information and the preset sampling step size, obtain multiple sampling points, where all of the sampling points are located on feature A;

[0018] Calculate the first-order derivative of each sampling point to obtain the tangential direction of each point, and then obtain the gradient direction of each point by cross-multiplying the normal vector and each tangential direction;

[0019] Step 3: Convert the sampling points and tangential directions to the image coordinate system. Then, according to the camera distortion parameters, add radial distortion and tangential distortion to each converted sampling point coordinate and tangential direction to obtain the distorted sampling point coordinate and tangential direction.

[0020] Using the camera intrinsic parameter matrix, the coordinates of each distorted sampling point and the tangential direction are converted to the pixel coordinate system, and the gradient direction is calculated by perpendicularly comparing the gradient direction to the tangential direction.

[0021] Step 4: Search for points in the two-dimensional image within a preset search range, using the coordinates of a single converted sampling point as the center and the positive and negative directions of the corresponding gradient directions as the search directions. The point with the largest gradient change is recorded as the edge point coordinate; otherwise, it is considered that there is no edge point, and the search is continued with the next sampling point as the center until all sampling points are traversed;

[0022] The coordinates of all searched edge points are stored as the edge information of feature A.

[0023] Furthermore, for the circle or arc segment of feature A, the method for obtaining sampling points and calculating the tangential direction of each sampling point is as follows:

[0024] The converted theoretical information includes the center coordinates C(x0,y0,z0), the normal vector and radius R;

[0025] With the center coordinates C(x0,y0,z0) as the origin, the normal vector is the direction of one of the coordinate axes, and the other coordinate axis is The directions of the remaining coordinate axes are in, is a vector passing through the origin and not parallel to the normal;

[0026] Will Normalize to get the normalized vector

[0027] Coordinate axis The intersection point with the arc / circle is the first sampling point. Multiple sampling points are obtained by sampling the arc / circle multiple times in a counterclockwise direction with the central angle θ as the preset sampling step.

[0028] Calculate the first-order derivative of each sampling point to get the tangential direction of each point, and then cross-multiply the normal vector with each tangential direction to get the gradient direction of each point. k ,y k ,z k ), tangential direction (dx k ,dy k ,dz k ), gradient direction (gx k ,gy k ,gz k ) is calculated as follows:

[0029] x k =cos((k-1)×θ)×R×i1+sin((k-1)×θ)×R×i2+x0

[0030] y k=cos((k-1)×θ)×R×j1+sin((k-1)×θ)×R×j2+y0

[0031] z k =cos((k-1)×θ)×R×k1+sin((k-1)×θ)×R×k2+z0

[0032] dx k =cos((k-1)×θ)×R×i2-sink(k-1)×θ)×R×i1

[0033] dy k =cos((k-1)×θ)×R×j2-sin((k-1)×θ)×R×j1

[0034] dz k =cos((k-1)×θ)×R×k2-sin((k-1)×θ)×R×k1

[0035] gx k =j*dz k -k*dy k

[0036] gy k =k*dx k -i*dz k

[0037] gz k =i*dy k -j*dx k

[0038] Where θ is the central angle between adjacent sampling points; k = 1, 2, 3, ..., M, represents the number of the sampling point. When feature A is a circle, M = 360° / θ; when feature A is an arc, M = the central angle of the arc / θ.

[0039] Preferably, θ is set to a value of 0.5 to 5°.

[0040] Furthermore, for the straight line segment of feature A, the method of obtaining sampling points and calculating the gradient direction of each sampling point is as follows:

[0041] The converted theoretical information includes the starting point P(x 0p ,y 0p ,z 0p ), length L, direction vector and normal vector

[0042] Taking the starting point P as the first adopted point, along the direction vector Multiple sampling points are obtained on the straight line with the preset sampling step size of the point spacing dL. The first-order derivative of each sampling point is calculated to obtain the tangential direction of each point. The gradient direction is then obtained by cross-producting the normal vector with each tangential direction.

[0043] Furthermore, the coordinates of the kth sampling point (x k ,y k ,z k ), tangential direction (dx k ,dy k ,dz k ), gradient direction (gx k ,dy k ,gz k ) is calculated as follows:

[0044] x k =dL×(k-1)×i3+x 0p

[0045] y k =dL×(k-1)×j3+y 0p

[0046] z k =dL×(k-1)×k3+z 0p

[0047] dx k =i3

[0048] dy k =j3

[0049] dz k =k3

[0050] gx k =j*dz k -k*dy k

[0051] gy k =k*dx k -i*dz k

[0052] gz k =i*dy k -i*dx k

[0053] Wherein, dL represents the distance between adjacent sampling points; k=1, 2, 3...M1 represents the number of the sampling point, and M1=L / dL.

[0054] Preferably, the value of dL is 1 to 5 mm.

[0055] Furthermore, in step 3, the sampling points and the tangential direction are converted to the image coordinate system, and then the coordinates of each converted sampling point (m k , n k ) and the gradient direction (dm k ,dn k ) Add radial distortion and tangential distortion to obtain the coordinates of the distorted sampling points (p k ,q k ) and the gradient direction (dq k , -dp k )The formula is as follows:

[0056] p k =m k ×[1+k1×r+k2×r 2 +k3×r 3 ]+2×p1×m k ×n k +p2×[r+2×m k ×m k ]

[0057] q k =n k ×[1+k1×r+k2×r 2 +k3×r 3 ]+p1×[r+2×n k ×n k ]+2×p2×m k ×n k

[0058] dq k =n k ×[k1×d r +2×k2×r×d r +3×k3×r 2 ×d r ]+dn k ×[1+k1×r+k2×r 2 +k3×r 3 ]+2×p1×(m k ×dm k +3×n k ×dn k )+2×p2×(m k ×dn k +n k ×dm k )

[0059] dp k =m k ×[k1×d r +2×k2×r×d r+3×k3×r 2 ×d r ]+dm k ×[1+k1×r+k2×r 2 +k3×r 3 ]+2×p1×(m k ×dn k +n k ×dm k )+2×p2×(3×m k ×dm k +n k ×dn k )

[0060] in, d r =2×(m k ×dm k +n k ×dn k ), k1, k2, k3 represent radial distortion parameters, and p1, p2 represent tangential distortion parameters.

[0061] The present invention has the following beneficial effects:

[0062] ① This method implements point-to-point projection of theoretical information (converted to the camera coordinate system), which is more in line with the principle of the pinhole imaging model. The projected theoretical information is sampled on the features, and the gradient information is obtained while obtaining the coordinates of the sampling points. The gradient information constrains the direction information of the edge, reduces noise interference, narrows the edge search range, and improves the accuracy of edge point search and detection efficiency.

[0063] ② When the sensor only collects two-dimensional images, this method fully considers the distortion of the camera lens, introduces radial distortion and tangential distortion into the coordinates and gradient directions of the sampling points, so as to better restore the original appearance of the features, improve the robustness of the algorithm, and solve the problem that the existing projection model lacks distortion information and leads to inaccurate detection results. The detection results are more reliable. This method is suitable for edge extraction of circular holes, slotted holes, polygonal holes (rectangular holes, triangular holes, parallelogram holes, pentagonal holes, etc.), and edge features. For images with low contrast and few texture details, the edge detection results can still remain effective, meeting the detection accuracy requirements of high-precision processing and manufacturing industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 Schematic diagram of the three-dimensional point cloud of the circular hole feature collected in Example 1;

[0065] Figure 2 This is a schematic diagram of the cube search range in Example 1;

[0066] Figure 3Schematic diagram of a two-dimensional image of a circular hole feature collected in Example 2;

[0067] Figure 4 Schematic diagram of the sampling point positions after distortion in Example 2;

[0068] Figure 5 Schematic diagram of the gradient direction after distortion in Example 2;

[0069] Figure 6 This is a schematic diagram of the rectangular search range in Example 2. DETAILED DESCRIPTION

[0070] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0071] In specific implementation, there are three types of features A:

[0072] 1. Consisting only of circles / arcs, such as circular holes and semicircles;

[0073] 2. Composed only of straight lines, such as polygonal holes (rectangular holes, triangular holes, parallelogram holes, pentagonal holes, etc.) and edge lines;

[0074] 3. Formed by a combination of arcs and straight lines, such as slots.

[0075] Example 1

[0076] An edge extraction method based on theoretical information, this embodiment is suitable for obtaining the surface data of the object to be measured as a three-dimensional point cloud (such as Figure 1 As shown), the following steps are used to extract edge information of a surface feature A of the object to be measured, where the feature A is composed of one or more of a circle, an arc, and a straight line:

[0077] S1. According to the pre-calibrated conversion relationship between the coordinate system of the object to be measured and the camera coordinate system, the theoretical information of feature A in the digital model is converted to the camera coordinate system;

[0078] The theoretical information of the circle or arc segment of feature A in the mathematical model includes the coordinates of the center, the normal vector (the normal vector of the plane where feature A is located) and the radius;

[0079] The theoretical information of the straight line segment of feature A in the mathematical model includes the starting point, length, direction vector (the extension direction vector of the straight line segment) and normal vector (the normal vector of the plane where feature A is located);

[0080] S2. In the camera coordinate system, using the converted theoretical information and a preset sampling step size, obtain multiple sampling points, where all of the sampling points are located on feature A;

[0081] Calculate the first-order derivative of each sampling point to obtain the tangential direction (first-order derivative direction) of each sampling point, and then obtain the gradient direction of each point by cross-multiplying the normal vector and each tangential direction;

[0082] S3. Search for points in the 3D point cloud within a preset search range, with the coordinates of a single sampling point as the center and the positive and negative directions of the corresponding gradient directions as the search directions. If a point is found, the first point found is recorded as the edge point coordinate; otherwise, it is considered that there is no edge point here, and the search is continued with the next sampling point as the center until all sampling points are traversed; that is, each sampling point is searched;

[0083] In this embodiment, Figure 2 As shown, the preset search range is a cube (a sphere can also be used in specific implementations), with a base length and width of 3 to 10 pixels and a height of 10 to 20 pixels. In specific implementations, the same cube search range is established for each sampling point. Within the cube, points in the point cloud are searched along both the positive and negative gradient directions. If a point is found, the first point found is recorded as the edge point coordinate.

[0084] The coordinates of all searched edge points are stored as the edge information of feature A.

[0085] Specifically, in step S2, for the circle or arc segment of feature A, the method of obtaining sampling points and calculating the gradient direction of each sampling point is as follows:

[0086] The converted theoretical information includes the center coordinates C(x0,y0,z0), the normal vector and radius R;

[0087] With the center coordinates C(x0,y0,z0) as the origin, the normal vector is the direction of one of the coordinate axes, and the other coordinate axis is The directions of the remaining coordinate axes are in, is a vector passing through the origin and not parallel to the normal;

[0088] Will Normalize to get the normalized vector

[0089] Coordinate axis The intersection point with the arc / circle is the first sampling point. Multiple sampling points are obtained by sampling the arc / circle multiple times in a counterclockwise direction with the central angle θ as the preset sampling step.

[0090] The first-order derivative of each sampling point is calculated to obtain the tangential direction of each point, and then the gradient direction of each point is obtained by cross-multiplying the normal vector and each tangential direction.

[0091] Among them, the coordinates of the kth sampling point (x k ,y k ,z k ), tangential direction (dx k ,dy k ,dz k ), gradient direction (gx k ,gy k ,gz k ) is calculated as follows:

[0092] x k =cos((k-1)×θ)×R×i1+sin((k-1)×θ)×R×i2+x0

[0093] y k =cos((k-1)×θ)×R×j1+sin((k-1)×θ)×R×j2+y0

[0094] z k =cos((k-1)×θ)×R×k1+sin((k-1)×θ)×R×k2+z0

[0095] dx k =cos((k-1)×θ)×R×i2-sin((k-1)×θ)×R×i1

[0096] dy k =cos((k-1)×θ)×R×j2-sin((k-1)×θ)×R×j1

[0097] dz k =cos((k-1)×θ)×R×k2-sin((k-1)×θ)×R×k1

[0098] gx k =j*dz k -k*dy k

[0099] gy k =k*dx k -i*dz k

[0100] gz k =i*dy k -j*dx k

[0101] Where θ is the central angle between adjacent sampling points; θ ranges from 0.5 to 5°; k = 1, 2, 3, ..., M, represents the number of the sampling point. When feature A is a circle, M = 360° / θ; when feature A is an arc, M = the central angle of the arc / θ.

[0102] Specifically, for the straight line segment of feature A, the method of obtaining sampling points and calculating the gradient direction of each sampling point is as follows:

[0103] The converted theoretical information includes the starting point P(x 0p ,y 0p ,z 0p ), length L, direction vector and normal vector

[0104] Taking the starting point P as the first adopted point, along the direction vector Multiple sampling points are obtained on the straight line with the preset sampling step size of the point spacing dL. The first-order derivative of each sampling point is calculated to obtain the tangential direction of each point. The gradient direction is then obtained by cross-producting the normal vector with each tangential direction.

[0105] In a straight line segment, the first-order derivative direction (tangential direction) of each sampling point is the direction vector (the extension direction of the straight line)

[0106] The coordinates of the kth sampling point (x k ,y k ,z k ), tangential direction (dx k ,dy k ,dz k ), gradient direction (gx k ,gy k ,gz k ) is calculated as follows:

[0107] x k =dL×(k-1)×i3+x 0p

[0108] y k =dL×(k-1)×j3+y 0p

[0109] z k =dL×(k-1)×k3+z 0p

[0110] dx k =i3

[0111] dy k =j3

[0112] dz k =k3

[0113] gx k =j*dz k -k*dy k

[0114] gy k =k*dx k -i*dz k

[0115] gz k =i*dy k -j*dx k

[0116] Where dL represents the distance between adjacent sampling points; dL ranges from 1 to 5 mm. k = 1, 2, 3, ..., M1 represents the number of the sampling point, M1 = L / dL.

[0117] Example 2

[0118] An edge extraction method based on theoretical information, this embodiment is suitable for obtaining the surface data of the object to be measured as a two-dimensional image (such as Figure 3 As shown), the following steps are used to extract edge information of a surface feature A of the object to be measured, where the feature A is composed of one or more of a circle, an arc, and a straight line:

[0119] Step 1: Based on the pre-calibrated conversion relationship between the coordinate system of the object to be measured and the camera coordinate system, the theoretical information of feature A in the digital model is converted to the camera coordinate system;

[0120] The theoretical information of the circle or arc segment of feature A in the mathematical model includes the coordinates of the center, the normal vector and the radius;

[0121] The theoretical information of the straight line segment of feature A in the mathematical model includes the starting point, length, direction vector and normal vector of the straight line;

[0122] Step 2: In the camera coordinate system, using the converted theoretical information and the preset sampling step size, obtain multiple sampling points, where all of the sampling points are located on feature A;

[0123] Calculate the first-order derivative of each sampling point to obtain the tangential direction of each point, and then obtain the gradient direction of each point by cross-multiplying the normal vector and each tangential direction;

[0124] Step 3: Convert the sampling points and tangential directions to the image coordinate system, and then add radial distortion and tangential distortion to each converted sampling point coordinate and tangential direction according to the camera distortion parameters to obtain the distorted sampling point coordinates (such as Figure 4 As shown) and the gradient direction (as Figure 5 shown);

[0125] Using the camera intrinsic parameter matrix, the coordinates of each distorted sampling point and the tangential direction are converted to the pixel coordinate system, and the gradient direction is calculated by perpendicularly comparing the gradient direction to the tangential direction.

[0126] Step 4: With the coordinates of a single converted sampling point as the center and the positive and negative directions of the corresponding gradient direction as the search directions, search for points in the two-dimensional image within the preset search range. If the point can be found, the point with the largest gradient change is recorded as the edge point coordinate; otherwise, it is considered that there is no edge point here, and the search is continued with the next sampling point as the center until all sampling points are traversed;

[0127] In this embodiment, Figure 6 As shown, the preset search range is a rectangular selection box (which can also be set to a circle or square in specific implementations) with a long side of 10 to 20 pixels and a short side of 3 to 10 pixels. In specific implementations, the same rectangular search range is established for each converted sampling point, and the point in the 2D image is searched within the rectangle.

[0128] The coordinates of all searched edge points are stored as the edge information of feature A.

[0129] In step 2, for the circle or arc segment of feature A, the method for obtaining sampling points and calculating the gradient direction of each sampling point is as follows:

[0130] The converted theoretical information includes the center coordinates C(x0, y0, z0) and the normal vector (the normal vector of the plane where feature A is located) and radius R;

[0131] With the center coordinates C(x0,y0,z0) as the origin, the normal vector is the direction of one of the coordinate axes, and the other coordinate axis is The directions of the remaining coordinate axes are in, is a vector passing through the origin and not parallel to the normal;

[0132] Will Normalize to get the normalized vector

[0133] Coordinate axis The intersection point with the arc / circle is the first sampling point. Multiple sampling points are obtained by sampling the arc / circle multiple times in a counterclockwise direction with the central angle θ as the preset sampling step.

[0134] The first-order derivative of each sampling point is calculated to obtain the tangential direction of each point, and then the gradient direction of each point is obtained by cross-multiplying the normal vector and each tangential direction.

[0135] The coordinates of the kth sampling point (x k ,y k ,z k ), tangential direction (dx k ,dy k ,dz k), gradient direction (gx k ,gy k ,gz k ) is calculated as follows:

[0136] x k =cos((k-1)×θ)×R×i1+sin((k-1)×θ)×R×i2+x0

[0137] y k =cos((k-1)×θ)×R×j1+sin((k-1)×θ)×R×j2+y0

[0138] z k =cos((k-1)×θ)×R×k1+sin((k-1)×θ)×R×k2+z0

[0139] dx k =cos((k-1)×θ)×R×i2-sin((k-1)×θ)×R×i1

[0140] dy k =cos((k-1)×θ)×R×j2-sin((k-1)×θ)×R×j1

[0141] dz k =cos((k-1)×θ)×R×k2-sin((k-1)×θ)×R×k1

[0142] gz k =j*dz k -k*dy k

[0143] gy k =k*dx k -i*dz k

[0144] gz k =i*dy k -j*dx k

[0145] Where θ is the central angle between adjacent sampling points; θ ranges from 0.5 to 5°. k = 1, 2, 3, ..., M represents the number of the sampling points. When feature A is a circle, M = 360° / θ. When feature A is an arc, M = the central angle of the arc / θ.

[0146] Specifically, for the straight line segment of feature A, the method of obtaining sampling points and calculating the gradient direction of each sampling point is as follows:

[0147] The converted theoretical information includes the starting point P(x 0p ,y0p ,z 0p ), length L, direction vector (direction of extension of the straight line) and normal vector (normal vector of the plane where feature A is located)

[0148] Taking the starting point P as the first adopted point, along the direction vector In the direction of the line, multiple samplings are performed with the point spacing dL as the preset sampling step to obtain multiple sampling points. The first-order derivative of each sampling point is calculated to obtain the tangential direction of each point. The gradient direction is then obtained by cross-producting the normal vector with each tangential direction.

[0149] The coordinates of the kth sampling point (x k ,y k ,z k ), tangential direction (dx k ,dy k ,dz k ), gradient direction (gx k ,gy k ,gz k ) is calculated as follows:

[0150] x k =dL×(k-1)×i3+x 0p

[0151] y k =dL×(k-1)×j3+y 0p

[0152] z k =dL×(k-1)×k3+z 0p

[0153] dx k =i3

[0154] dy k =j3

[0155] dz k =k3

[0156] gx k =j*dz k -k*dy k

[0157] gy k =k*dx k -i*dz k

[0158] gz k =i*dy k -j*dx k

[0159] Where dL represents the distance between adjacent sampling points; dL ranges from 1 to 5 mm. k = 1, 2, 3, ..., M1 represents the number of the sampling point, M1 = L / dL.

[0160] More specifically, when the camera distortion parameters are five parameters: radial distortion parameters k1, k2, k3, and tangential distortion parameters: p1, p2.

[0161] Step 3: Set the sampling point (x k ,y k ,z k ), and the gradient direction (gx k ,gy k ,gz k ) are converted to the image coordinate system, and then according to the camera distortion parameters, the coordinates of each converted sampling point (m k , n k ) and the gradient direction (dm k ,dn k ) Add radial distortion and tangential distortion to obtain the coordinates of the distorted sampling points (p k ,q k ) and the gradient direction is (dq k , -dp k ), as follows:

[0162] The sampling point (x k ,y k ,z k ), and the gradient direction (gx k ,gy k ,gz k ) are converted to the image coordinate system, and the converted sampling point coordinates (m k , n k ) and the gradient direction (dm k ,dn k ):

[0163] m k =x k / z k

[0164] n k =y k / z k

[0165]

[0166]

[0167] is the converted sampling point (m k , n k ) and the gradient direction (dmk ,dn k ) Add radial distortion and tangential distortion to obtain the coordinates of the distorted sampling points (p k ,q k ) and the gradient direction is (dq k , -dp k ), the formula is as follows:

[0168] p k =m k ×[1+k1×r+k2×r 2 +k3×r 3 ]+2×p1×m k ×n k +p2×[r+2×m k ×m k ]

[0169] q k =n k ×[1+k1×r+k2×r 2 +k3×r 3 ]+p1×[r+2×n k ×n k ]+2×p2×m k ×n k

[0170] dq k =n k ×[k1×d r +2×k2×r×d r +3×k3×r 2 ×d r ]+dn k ×[1+k1×r+k2×r 2 +k3×r 3 ]+2×p1×(m k ×dm k +3×n k ×dn k )+2×p2×(m k ×dn k +n k ×dm k )

[0171] dp k =m k ×[k1×d r +2×k2×r×d r +3×k3×r 2 ×d r ]+dm k ×[1+k1×r+k2×r 2 +k3×r3 ]+2×p1×(m k ×dn k +n k ×dm k )+2×p2×(3×m k ×dm k +n k ×dn k )

[0172] in, d r =2×(m k ×dm k +n k ×dn k ), k1, k2, k3 represent radial distortion parameters, and p1, p2 represent tangential distortion parameters.

[0173] The intrinsic parameter matrix of the camera obtained by pre-calibration is:

[0174]

[0175] Among them, f / dx represents the length of the focal length in the x-axis direction described by pixels, f / dy represents the length of the focal length in the y-axis direction described by pixels, and (u0, v0) represents the coordinates of the principal point.

[0176] Using the camera internal parameter matrix, the coordinates of each distorted sampling point (p k ,q k ) and the gradient direction (dq k , -dp k ) is converted to the pixel coordinate system to obtain the converted sampling point coordinates (u k ,v k ) and gradient direction (dv k ,-du k ), as follows:

[0177]

[0178]

[0179]

[0180]

[0181] Step 4: Search for points in the two-dimensional image within a preset search range, using the coordinates of a single converted sampling point as the center and the positive and negative directions of the corresponding gradient direction as the search directions. If a point is found, the point with the largest gradient change is recorded as the edge point coordinate; otherwise, it is considered that there is no edge point here, and the search is continued with the next sampling point as the center until all sampling points are traversed;

[0182] The coordinates of all searched edge points are stored as the edge information of feature A.

[0183] The edge extraction method proposed in this invention decomposes the feature A to be measured into circles and straight line segments. Based on theoretical information about circles, arcs, and straight line segments, it samples edge points and gradient directions on the theoretical feature A. Based on the sampled points and gradient directions, the true edge position is located, resulting in the whole-pixel edge. As a subsequent application of the method, sub-pixel edges can be obtained through interpolation. The edge information obtained using this method is more accurate, meeting the detection accuracy requirements of high-precision manufacturing.

[0184] The following is an illustrative example of different scenarios of Feature A:

[0185] Case 1: Feature A is a circular hole or semicircle, consisting only of circles or arcs. In this case, the sampling points are obtained and the gradient direction of each sampling point is calculated according to the processing method for circles or arc segments. The measured edge points are then found based on the sampling points and gradient directions.

[0186] Case 2. Feature A is a polygonal hole (rectangular hole, triangular hole, parallelogram hole, pentagonal hole, etc.) or an edge line; it consists only of straight line segments. In this case, the sampling points are obtained according to the processing method for straight line segments, the gradient direction of each sampling point is calculated, and then the measured edge points are found based on the sampling points and gradient directions.

[0187] Taking a rectangular hole as an example, when the known theoretical information is the hole center coordinates C(x0,y0,z0), the length direction and width direction Length value L, width value W. Based on the above information, find the four vertices of the rectangular hole (in counterclockwise order of lower right, upper right, upper left, and lower left):

[0188]

[0189]

[0190]

[0191]

[0192] Then the starting point of line 1 is P1 and the direction vector is The length is W; the starting point of line 2 is P2, and the direction vector is The length is L; the starting point of line 3 is P3, and the direction vector is The length is W; the starting point of line 4 is P4, and the direction vector is The length is L. The theoretical information of the four straight line segments is obtained. According to the processing method of the straight line segments, the sampling points are obtained, the gradient direction of each sampling point is obtained, and then the measured edge points are found based on the sampling points and the gradient direction.

[0193] Case 3. Feature A is a slot composed of an arc and a straight line. In this case, the edge of feature A is split into arc segments and straight line segments. Sampling points are obtained according to the arc segment and straight line segment processing methods, and the gradient direction of each sampling point is calculated. The measured edge points are then found based on the sampling points and gradient directions.

[0194] Specifically, the known theoretical information of the slot is the hole center point C (x0, y0, z0), the normal Length direction And length L and width W. Through the normal and length direction Cross product to get the width direction

[0195] Based on the above information, find the centers of the two semicircles of the slot (from right to left) and the four vertices (in the order of lower right, upper right, upper left, and lower left in counterclockwise order):

[0196]

[0197]

[0198]

[0199]

[0200]

[0201]

[0202] The theoretical information of the two line segments L1 and L2 are P2 and P4 as the starting points respectively. is the direction vector, and L is the length. It obtains sampling points according to the straight line segment processing method, calculates the gradient direction of each sampling point, and then finds the measured edge points based on the sampling points and gradient directions.

[0203] The two semicircles R1 and R2 have C1 and C2 as their center coordinates and 0.5*W as their radius. The normal vector is the starting point of the two semicircular arcs [P1, P2] and [P3, P4], respectively, with a central angle of 180°. Starting from P1 and proceeding counterclockwise, the arcs are L1, R1, L2, and R2. The arc segment processing method is used to obtain sampling points, calculate the gradient direction of each sampling point, and then find the measured edge point based on the sampling points and gradient direction.

[0204] For convenience in explanation and precise definition in the appended claims, the terms "upper," "lower," "inner," and "outer" are used to describe features of the exemplary embodiments with reference to the positions of such features as shown in the drawings.

[0205] The foregoing descriptions of specific exemplary embodiments of the present invention have been presented for purposes of illustration and description. The foregoing descriptions are not intended to be exhaustive or to limit the invention to the precise forms disclosed, and it is apparent that many variations and modifications are possible in light of the foregoing teachings. The exemplary embodiments have been chosen and described in order to explain the specific principles of the invention and their practical application, thereby enabling others skilled in the art to make and utilize the various exemplary embodiments of the invention and various alternatives and modifications thereof. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. An edge extraction method based on theoretical information, suitable for obtaining surface data of an object to be measured in the form of a three-dimensional point cloud, characterized in that: The following steps are used to extract edge information of a surface feature A of the object to be measured, where the feature A is composed of one or more of a circle, an arc, and a straight line: S1. According to the pre-calibrated conversion relationship between the coordinate system of the object to be measured and the camera coordinate system, the theoretical information of feature A in the digital model is converted to the camera coordinate system; The theoretical information of the circle or arc segment of feature A in the mathematical model includes the coordinates of the center, the normal vector and the radius; The theoretical information of the straight line segment of feature A in the mathematical model includes the starting point, length, direction vector and normal vector of the straight line; S2. In the camera coordinate system, using the converted theoretical information and a preset sampling step size, obtain multiple sampling points, where all of the sampling points are located on feature A; Calculate the first-order derivative of each sampling point to obtain the tangential direction of each sampling point, and then obtain the gradient direction of each point by cross-multiplying the normal vector and each tangential direction; S3. Search for points in the 3D point cloud within a preset search range, with the coordinates of a single sampling point as the center and the positive and negative directions of the corresponding gradient directions as the search directions. If a point is found, the first point found is recorded as the edge point coordinate; otherwise, it is considered that there is no edge point here, and the search is continued with the next sampling point as the center until all sampling points are traversed; The coordinates of all searched edge points are stored as the edge information of feature A.

2. A method for edge extraction based on theoretical information, suitable for obtaining surface data of an object to be measured as a two-dimensional image, characterized in that: The following steps are used to extract edge information of a surface feature A of the object to be measured, where the feature A is composed of one or more of a circle, an arc, and a straight line: Step 1: Based on the pre-calibrated conversion relationship between the coordinate system of the object to be measured and the camera coordinate system, the theoretical information of feature A in the digital model is converted to the camera coordinate system; The theoretical information of the circle or arc segment of feature A in the mathematical model includes the coordinates of the center, the normal vector and the radius; The theoretical information of the straight line segment of feature A in the mathematical model includes the starting point, length, direction vector and normal vector of the straight line; Step 2: In the camera coordinate system, using the converted theoretical information and the preset sampling step size, obtain multiple sampling points, where all of the sampling points are located on feature A; Calculate the first-order derivative of each sampling point to obtain the tangential direction of each point, and then obtain the gradient direction of each point by cross-multiplying the normal vector and each tangential direction; Step 3: Convert the sampling points and gradient directions to the image coordinate system. Then, according to the camera distortion parameters, add radial distortion and tangential distortion to each converted sampling point coordinate and gradient direction to obtain the distorted sampling point coordinate and gradient direction. Use the camera intrinsic parameter matrix to transform the coordinates of each distorted sampling point and the gradient direction into the pixel coordinate system; Step 4: Search for a point in the two-dimensional image within a preset search range, using the coordinates of a single converted sampling point as the center and the positive and negative directions of the corresponding gradient direction as the search directions. If a point is found, the point with the largest gradient change is recorded as the edge point coordinate; otherwise, it is considered that there is no edge point and the search is continued with the next sampling point as the center until all sampling points are traversed; The coordinates of all searched edge points are stored as the edge information of feature A.

3. The edge extraction method based on theoretical information according to claim 1 or 2, characterized in that: For the circle or arc segment of feature A, the method of obtaining sampling points and calculating the gradient direction of each sampling point is as follows: The converted theoretical information includes the center coordinates C(x0,y0,z0), the normal vector and radius R; With the center coordinates C(x0,y0,z0) as the origin, the normal vector is the direction of one of the coordinate axes, and the other coordinate axis is The directions of the remaining coordinate axes are in, is a vector passing through the origin and not parallel to the normal; Will Normalize to get the normalized vector Coordinate axis The intersection point with the arc / circle is the first sampling point. Multiple sampling points are obtained by sampling the arc / circle multiple times in a counterclockwise direction with the central angle θ as the preset sampling step. The first-order derivative of each sampling point is calculated to obtain the tangential direction of each point, and then the gradient direction of each point is obtained by cross-multiplying the normal vector and each tangential direction.

4. The edge extraction method based on theoretical information as claimed in claim 3, characterized in that: The coordinates of the kth sampling point (x k ,y k ,z k ), tangential direction (dx k ,dy k ,dz k ), gradient direction (gx k ,gy k ,gz k ) is calculated as follows: x k =cos((k-1)×θ)×R×i1+sin((k-1)×θ)×R×i2+x0 and k =cos((k-1)×θ)×R×j1+sin((k-1)×θ)×R×j2+y0 With k =cos((k-1)×θ)×R×k1+sin((k-1)×θ)×R×k2+z0 dx k =cos((k-1)×θ)×R×i2-sin((k-1)×θ)×R×i1 of k =cos((k-1)×θ)×R×j2-sin((k-1)×θ)×R×j1 dz k =cos((k-1)×θ)×R×k2-sin((k-1)×θ)×R×k1 gx k =j*dz k -k*dy k gy k =k*dx k -i*dz k gz k =i*dy k -j*dx k Where θ is the central angle between adjacent sampling points; k = 1, 2, 3, ..., M, represents the number of the sampling point. When feature A is a circle, M = 360° / θ; when feature A is an arc, M = the central angle of the arc / θ.

5. The edge extraction method based on theoretical information as claimed in claim 4, characterized in that: The value of θ is 0.5~5°.

6. The edge extraction method based on theoretical information according to claim 1 or 2, characterized in that: For the straight line segment of feature A, the method of obtaining sampling points and calculating the gradient direction of each sampling point is as follows: The converted theoretical information includes the starting point P(x 0p ,y 0p ,z 0p ), length L, direction vector and normal vector Taking the starting point P as the first adopted point, along the direction vector Multiple sampling points are obtained on the straight line with the preset sampling step size of the point spacing dL. The first-order derivative of each sampling point is calculated to obtain the tangential direction of each point. The gradient direction is then obtained by cross-producting the normal vector with each tangential direction.

7. The edge extraction method based on theoretical information according to claim 6, characterized in that: The coordinates of the kth sampling point (x k ,y k ,z k )Tangential direction (dx k ,dy k ,dz k ), gradient direction (gx k ,gy k ,gz k ) is calculated as follows: x k =dL×(k-1)×i3+x 0p and k =dL×(k-1)×j3+y 0p With k =dL×(k-1)×k3+z 0p dx k =i3 you k =j3 dz k =k3 gx k =j*dz k -k*dy k gy k =k*dx k -i*dz k gz k =i*dy k -i*dx k Wherein, dL represents the distance between adjacent sampling points; k=1, 2, 3...M1 represents the number of the sampling point, and M1=L / dL.

8. The edge extraction method based on theoretical information according to claim 6, characterized in that: The value of dL is 1 to 5 mm.

9. The edge extraction method based on theoretical information as claimed in claim 2, characterized in that: Step 3: Convert the sampling points and gradient directions to the image coordinate system, and then calculate the coordinates of each converted sampling point (m k , n k ) and the gradient direction (dm k ,dn k ) Add radial distortion and tangential distortion to obtain the coordinates of the distorted sampling points (p k ,q k ) and the gradient direction is (dq k , -dp k ), the formula is as follows: p k =m k ×[1+k1×r+k2×r 2 +k3×r 3 ]+2×p1×m k ×n k +p2×[r+2×m k ×m k ] q k =n k ×[1+k1×r+k2×r 2 +k3×r 3 ]+p1×[r+2×n k ×n k ] +2×p2×m k ×n k dq k =n k ×[k1×d r +2×k2×r×d r +3×k3×r 2 ×d r ] +dn k ×[1+k1×r+k2×r 2 +k3×r 3 ] +2×p1×(m k ×dm k +3×n k ×dn k ) +2×p2×(m k ×dn k +n k ×dm k ) dp k =m k ×[k1×d r +2×k2×r×d r +3×k3×r 2 ×d r ] +dm k ×[1+k1×r+k2×r 2 +k3×r 3 ] +2×p1×(m k ×dn k +n k ×dm k ) +2×p2×(3×m k ×dm k +n k ×dn k ) in, d r =2×(m k ×dm k +n k ×dn k ), k1, k2, k3 represent radial distortion parameters, and p1, p2 represent tangential distortion parameters.

Images