Round hole pose measurement method based on reprojection optimization

Abstract

The invention discloses a round hole pose measurement method based on reprojection optimization. The method comprises the steps that a binocular camera is used for collecting two target images containing a round hole, then initial pose parameters of the round hole are obtained, then a projection ellipse of the round hole on the plane of the binocular camera is generated through re-projection, the ellipse is detected in combination with the edge, the updating step length of the pose parameters of the round hole is obtained after processing is conducted through an ellipse error and a cosine attenuation strategy, and the pose parameters of the round hole are obtained. And the pose parameter of the circular hole is iteratively updated to realize the measurement of the pose of the circular hole. According to the method, the circular hole space pose parameters serve as variables to be optimized, the projection ellipse of the circular hole on the plane of the binocular camera is generated, the error between the projection ellipse and the ellipse obtained through the ellipse detection algorithm is minimum, the ellipse detection result is fully utilized, the defect that the ellipse detection result cannot meet the binocular vision epipolar constraint is overcome, and the binocular vision epipolar constraint requirement is met. Meanwhile, a cosine attenuation strategy is combined to calculate a learning rate, so that a circular hole space pose parameter calculation result is quickly converged, and the measurement precision is improved.

Description

Technical Field

[0001] The present invention relates to a circular hole posture measurement method, and in particular to a circular hole posture measurement method based on reprojection optimization. Background Art

[0002] Circular holes are common geometric features on parts. Estimating the pose of parts by measuring their pose is an important method for robotic grasping of parts containing holes. For example, during satellite assembly, the pose of circular holes on onboard equipment is often measured first, used to position the equipment and perform robotic grasping. However, existing circular hole pose measurement algorithms often use two-dimensional features that fail to fully satisfy spatial geometric constraints, resulting in low pose measurement accuracy.

[0003] Currently, mainstream methods for estimating the pose of circular apertures can be divided into two categories: monocular vision-based methods and binocular vision-based methods. Monocular vision-based methods rely solely on features from a single image to calculate the pose variables of the circular aperture. However, due to the geometric characteristics of the spatial oblique cone, the resulting solution is both true and false, leading to ambiguity. Prior information such as the radius is required to eliminate false solutions. Furthermore, single images are susceptible to complex lighting conditions and image quality, further limiting their application scenarios. Binocular vision-based methods use features from both images to calculate the pose variables of the circular aperture, eliminating false solutions obtained from monocular vision measurements and avoiding ambiguity. However, in practice, the features extracted from the two images used to calculate the pose variables often fail to meet the epipolar constraints of binocular vision, affecting measurement accuracy. While these studies have improved the performance of circular aperture pose measurement, they still face challenges in accuracy and robustness. Summary of the Invention

[0004] In order to solve the problems existing in the background technology, the present invention provides a circular hole pose measurement method based on reprojection optimization.

[0005] The technical solution adopted in the present invention is:

[0006] The circular hole pose measurement method based on reprojection optimization of the present invention comprises:

[0007] Step 1) Use a binocular camera to capture two target images containing a circular hole, use an ellipse detection algorithm to obtain the edge detection ellipse formed by the circular hole in the two target images and its ellipse parameters, and then obtain the initial pose parameters of the circular hole.

[0008] Step 2) Reproject the initial pose parameters of the circular hole to generate the projection ellipse of the circular hole on the binocular camera plane.

[0009] Step 3) According to the projected ellipse and the edge detection ellipse, the update step size of the circular hole's posture parameters is obtained after combining the ellipse error and the cosine decay strategy, and then the circular hole's posture parameters are updated.

[0010] Step 4) Repeat the same operation of the initial pose parameters in steps 2) to 3) according to the updated pose parameters of the circular hole until the preset number of iterations is reached, and the final updated pose parameters of the circular hole are obtained to achieve the measurement of the circular hole pose.

[0011] In the step 1), the circular holes in the two target images captured by the binocular camera are the same circular holes, and the pixels in the two target images correspond to each other one by one. The elliptical parameters of the edge detection ellipse include the major axis a, the minor axis b, the center O of the edge detection ellipse, and the center O of the edge detection ellipse. e The x-axis and y-axis coordinates in the image coordinate system ( e .x,O e .y) and inclination

[0012] In the step 1), for each target image, the initial position parameters of the circular hole corresponding to the edge detection ellipse are obtained using the circular hole binocular vision reconstruction method according to the elliptical parameters of the edge detection ellipse in the target image and the internal and external parameters of the binocular camera, including the x-axis, y-axis and z-axis coordinates of the center O of the circular hole in the camera coordinate system of the binocular camera (O c .x,O c .y,O c .z), radius r and normal vector n.

[0013] In the step 2), a world coordinate system is first established. The world coordinate system takes the center O of the circular hole as the origin, the normal vector of the plane where the circular hole is located as the z-axis, and the intersection of the plane where the circular hole is located and the xz plane of the camera coordinate system of the binocular camera as the x-axis. The y-axis is obtained according to the right-hand rule. The coordinates of any point on the edge of the circular hole in the world coordinate system are obtained, and then the x-axis, y-axis and z-axis coordinates (O) of the center O of the circular hole in the camera coordinate system of the binocular camera are obtained. c .x,O c .y,O c .z), radius r and normal vector n are used to project any point on the edge of the circular hole to the camera coordinate system to obtain the projection point; each projection point is processed by the least squares fitting method to obtain the projected ellipse.

[0014] The step 3) is as follows:

[0015] Step 3.1) Obtain the ellipse errors of the projected ellipse and the edge detection ellipse, then average them to obtain the average ellipse error. For each pose parameter of the circular hole, obtain the derivative of the error average with respect to the pose parameter.

[0016] In step 3.2), the cosine decay strategy is used to calculate the learning rate. The learning rate is multiplied by the derivative in step 3.1) to obtain the update step size of each pose parameter of the circular hole, and the pose parameters are updated according to their respective update step sizes.

[0017] In step 3.1), the elliptic error f is as follows:

[0018]

[0019] Among them, O represents the center of the circular hole; O e represents the center of the edge detection ellipse, a and b represent the lengths of the major and minor axes of the edge detection ellipse respectively; r represents the radius of the circular hole; n represents the normal vector of the circular hole; k represents the number of projection points of the normal vector of the circular hole; u i and v i They represent the u-axis and v-axis coordinates of the i-th pixel point of the projected ellipse on the camera plane.

[0020] The electronic device of the present invention comprises: a memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method described above.

[0021] The computer-readable storage medium of the present invention stores program data thereon, and when the program data is executed by a processor, the method described above is implemented.

[0022] The present invention uses a binocular vision reconstruction method to obtain the initial values of the circular hole pose parameters, including the normal vector, center coordinates, and radius. A projected ellipse of the circular hole on the binocular camera plane is generated. The error between the binocular camera projected ellipse and the ellipse obtained by edge detection is calculated. The derivative of this error with respect to the spatial pose parameters of the circular hole is obtained. A cosine decay strategy is used to calculate the learning rate, which is multiplied by the derivative to obtain the update step size for each spatial pose parameter of the circular hole, and the spatial pose parameters of the circular hole are updated. The algorithm terminates when a predetermined optimization cycle is reached; otherwise, the error solution, derivative calculation, and update of the circular hole pose parameters are repeated, offering the advantages of speed and high precision.

[0023] The beneficial effects of the present invention are:

[0024] 1) The present invention uses the spatial pose parameters of the circular hole as the variables to be optimized and generates a projected ellipse of the circular hole on the binocular camera plane, minimizing the error between the projected ellipse and the ellipse obtained by the ellipse detection algorithm. This fully utilizes the ellipse detection results and makes up for the deficiency that the ellipse detection results cannot meet the binocular vision epipolar constraint, thereby improving the measurement accuracy.

[0025] 2) The present invention adopts the cosine decay strategy to calculate the learning rate, so that the calculation results of the circular hole spatial posture parameters converge quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of the method of the present invention;

[0027] Figure 2 It is a schematic diagram of the elliptical loss of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] like Figure 1 As shown, the circular hole pose measurement method based on reprojection optimization of the present invention includes:

[0030] Step 1) Use a binocular camera to capture two target images containing a circular hole, use an ellipse detection algorithm to obtain the edge detection ellipse formed by the circular hole in the two target images and its ellipse parameters, and then obtain the initial pose parameters of the circular hole.

[0031] The circular holes in the two target images captured by the binocular camera are the same circular holes. The pixels in the two target images correspond to each other one by one. The elliptical parameters of the edge detection ellipse include the major axis a, minor axis b, and center O of the edge detection ellipse. e The x-axis and y-axis coordinates in the image coordinate system ( e .x,O e .y) and inclination The ellipse detection algorithm specifically adopts the ellipse method based on Hough transform and the least square method to fit the ellipse contour. For each target image, according to the ellipse parameters of the edge detection ellipse in the target image and the intrinsic and extrinsic parameters of the binocular camera, the circular hole binocular vision reconstruction method is used to obtain the initial pose parameters of the circular hole corresponding to the edge detection ellipse, including the x-axis, y-axis and z-axis coordinates of the center O of the circular hole in the camera coordinate system of the binocular camera (O c .x,O c .y,O c .z), radius r, and normal vector n. The circular hole binocular vision reconstruction method can specifically adopt the precise pose and radius estimation of circular target based on binocular vision.

[0032] Step 2) Generate the projection ellipse of the circular hole on the binocular camera plane based on the initial posture parameters of the circular hole. First, establish a world coordinate system. The world coordinate system takes the center O of the circular hole as the origin, the normal vector of the plane where the circular hole is located as the z-axis, and the intersection of the plane where the circular hole is located and the xz plane of the camera coordinate system of the binocular camera as the x-axis. According to the right-hand rule, the y-axis is obtained; the coordinates of any point on the edge of the circular hole in the world coordinate system are obtained, and then the x-axis, y-axis and z-axis coordinates (O) of the center O of the circular hole in the camera coordinate system of the binocular camera are obtained. c .x,O c .y,O c .z), radius r and normal vector n are used to project any point on the edge of the circular hole to the camera coordinate system to obtain the projection point; each projection point is processed by the least squares fitting method to obtain the projected ellipse.

[0033] The projection process is as follows:

[0034]

[0035] in, and Respectively represent the x-axis, y-axis, and z-axis coordinates of the i-th pixel in the camera coordinate system; R represents the rotation matrix of the world coordinate system relative to the camera coordinate system; and They represent the x-axis, y-axis, and z-axis coordinates of the i-th pixel in the world coordinate system respectively; r1 and r2 represent the first and second columns of the rotation matrix R respectively; and Represent the three elements of the first and second columns of the rotation matrix R respectively; θ i Represents the angle between the line connecting the i-th pixel and the center of the circle and the x-axis in the world coordinate system.

[0036] According to the projection relationship, the relationship between the projection point of the pixel point on the camera plane and the circular hole variable is as follows:

[0037]

[0038] Among them, u i and v i Respectively represent the u-axis and v-axis coordinates of the i-th pixel point on the camera plane; f x and f y Represents the focal length of the camera in the x-axis and y-axis directions respectively; c x and c y Respectively represent the coordinate values of the pixel point on the x-axis and y-axis of the image coordinate system; and They represent the x-axis, y-axis, and z-axis coordinates of the pixel point in the camera coordinate system respectively.

[0039] Step 3) Based on the projected ellipse and the edge detection ellipse, the update step size of the circular hole's pose parameters is obtained after combining the ellipse error and the cosine decay strategy, and then the circular hole's pose parameters are updated. The details are as follows:

[0040] Step 3.1) Obtain the elliptical errors of the projected ellipse and the edge detection ellipse, then average them to obtain the average elliptical error. For each pose parameter of the circular hole, obtain the derivative of the error average with respect to the pose parameter. The elliptical error f is as follows:

[0041]

[0042] Among them, O represents the center of the circular hole; O e represents the center of the edge detection ellipse, a and b represent the lengths of the major and minor axes of the edge detection ellipse respectively; r represents the radius of the circular hole; n represents the normal vector of the circular hole; k represents the number of projection points of the normal vector of the circular hole; u i and v i They represent the u-axis and v-axis coordinates of the i-th pixel point of the projected ellipse on the camera plane.

[0043] The partial derivatives of the error with respect to the circular hole pose parameters are as follows:

[0044]

[0045] Wherein, Δr represents the preset radius error.

[0046] Similarly, the partial derivatives of each posture parameter can be obtained separately.

[0047] In step 3.2), the cosine decay strategy is used to calculate the learning rate. The learning rate is multiplied by the derivative in step 3.1) to obtain the update step size of each pose parameter of the circular hole, and the pose parameters are updated according to their respective update step sizes.

[0048] Step 4) Repeat the same operation of the initial pose parameters in steps 2) to 3) according to the updated pose parameters of the circular hole until the preset number of iterations is reached, and the final updated pose parameters of the circular hole are obtained to achieve the measurement of the circular hole pose.

[0049] like Figure 2 As shown in the figure, the points on the space circle are connected to the optical center of the camera. The intersection of the connecting line and the camera plane is the projection point of the point on the space circle on the camera plane. The projection point does not coincide with the edge detection ellipse, so a loss function can be defined to measure the difference between them.

[0050] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the application can adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages. The application is described according to the flow chart of the method, system and computer program product of the embodiments of the present application.

[0051] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concepts. Therefore, the present invention is intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0052] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the equivalent technology of the present invention, the present application is intended to include these modifications and variations.

Claims

1. A circular hole pose measurement method based on reprojection optimization, characterized in that: include: Step 1) using a binocular camera to capture two target images containing a circular hole, using an ellipse detection algorithm to obtain the edge detection ellipse formed by the circular hole in the two target images and its ellipse parameters, thereby obtaining the initial pose parameters of the circular hole; Step 2) Reprojecting the initial pose parameters of the circular hole to generate a projection ellipse of the circular hole on the binocular camera plane; Step 3) According to the projected ellipse and the edge detection ellipse, the update step size of the circular hole's posture parameters is obtained after combining the ellipse error and the cosine decay strategy, and then the circular hole's posture parameters are updated; Step 4) Repeat the same operation of the initial pose parameters in steps 2) to 3) according to the updated pose parameters of the circular hole until the preset number of iterations is reached, and the final updated pose parameters of the circular hole are obtained to achieve the measurement of the circular hole pose.

2. The circular hole pose measurement method based on reprojection optimization according to claim 1, characterized in that: In the step 1), the circular holes in the two target images captured by the binocular camera are the same circular holes, and the ellipse parameters of the edge detection ellipse include the major axis a, the minor axis b, the center O e The x-axis and y-axis coordinates in the image coordinate system ( e .x,O e .y) and inclination 3. The circular hole pose measurement method based on reprojection optimization according to claim 1, characterized in that: In the step 1), for each target image, the initial position parameters of the circular hole corresponding to the edge detection ellipse are obtained using the circular hole binocular vision reconstruction method according to the elliptical parameters of the edge detection ellipse in the target image and the internal and external parameters of the binocular camera, including the x-axis, y-axis and z-axis coordinates of the center O of the circular hole in the camera coordinate system of the binocular camera (O c .x,O c .y,O c .z), radius r and normal vector n.

4. The circular hole pose measurement method based on reprojection optimization according to claim 3, characterized in that: In the step 2), a world coordinate system is first established. The world coordinate system takes the center O of the circular hole as the origin, the normal vector of the plane where the circular hole is located as the z-axis, and the intersection of the plane where the circular hole is located and the xz plane of the camera coordinate system of the binocular camera as the x-axis. The y-axis is obtained according to the right-hand rule. The coordinates of any point on the edge of the circular hole in the world coordinate system are obtained, and then the x-axis, y-axis and z-axis coordinates (O) of the center O of the circular hole in the camera coordinate system of the binocular camera are obtained. c .x,O c .y,O c .z), radius r and normal vector n are used to project any point on the edge of the circular hole to the camera coordinate system to obtain the projection point; each projection point is processed by the least squares fitting method to obtain the projected ellipse.

5. The circular hole pose measurement method based on reprojection optimization according to claim 1, characterized in that: The step 3) is as follows: Step 3.1) Obtain the ellipse errors of the projected ellipse and the edge detection ellipse, then average them to obtain the average ellipse error. For each pose parameter of the circular hole, obtain the derivative of the error average with respect to the pose parameter. In step 3.2), the cosine decay strategy is used to calculate the learning rate. The learning rate is multiplied by the derivative in step 3.1) to obtain the update step size of each pose parameter of the circular hole, and the pose parameters are updated according to their respective update step sizes.

6. The circular hole pose measurement method based on reprojection optimization according to claim 5, characterized in that: In step 3.1), the elliptic error f is as follows: Among them, O represents the center of the circular hole; O e represents the center of the edge detection ellipse, a and b represent the lengths of the major and minor axes of the edge detection ellipse respectively; r represents the radius of the circular hole; n represents the normal vector of the circular hole; k represents the number of projection points of the normal vector of the circular hole; u i and v i They represent the u-axis and v-axis coordinates of the i-th pixel point of the projected ellipse on the camera plane.

7. An electronic device, characterized in that: include: A memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images