A frustoconical object detection method based on monocular camera

Abstract

The application discloses a kind of methods for detecting circular truncated cone object based on monocular camera, based on monocular camera, the ellipse of the upper and lower surface of circular truncated cone object on camera image is obtained;Intermediate camera coordinate system is constructed, and the two groups of circle centers and normal vectors containing unknown radius parameters of upper and lower surface are calculated;According to the parallel relationship of the upper and lower surface of circular truncated cone object, the parallel geometric constraint is obtained, based on the parallel geometric constraint, the correct group of circle center and normal vector of each upper and lower surface is obtained, and the ambiguity in monocular circle pose estimation is eliminated;According to the height of circular truncated cone object and the normal vector of upper and lower surface, the radius of upper and lower surface is calculated;Based on the radius of upper and lower surface and the corresponding correct circle center, the circle center of upper and lower surface is obtained, so that circular truncated cone object can be detected more quickly and accurately by monocular camera.

Description

Technical Field

[0001] This invention belongs to the field of monocular vision and object pose estimation technology, specifically relating to a method for detecting frustum-shaped objects based on a monocular camera. Background Technology

[0002] Machine vision is a crucial research area in engineering, capable of replacing human eyes in various measurements and judgments. Many objects in daily life possess frustum-shaped features, such as nut-shaped workpieces, cups, roadblocks, and lampshades. In autonomous driving and robotic grasping processes, machine vision is essential for determining the positions of these objects. Traditional feature-matching methods, however, need to consider the correspondence of feature points during reconstruction, are not robust to noise and occlusion, and cannot fully utilize image information and elliptical features. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies and achieve the goal of rapid and accurate detection of frustum-shaped objects by a monocular camera, this invention adopts the following technical solution:

[0004] A method for detecting frustum-shaped objects based on a monocular camera includes the following steps:

[0005] Step S1: Based on the monocular camera, obtain the ellipses on the top and bottom surfaces of the frustum-shaped object in the camera image;

[0006] Step S2: Construct an intermediate camera coordinate system and calculate the center and normal vector of two sets of circles containing unknown radius parameters on both the top and bottom surfaces;

[0007] Step S3: Based on the parallel relationship between the top and bottom surfaces of the frustum-shaped object, obtain the parallel geometric constraints. Based on the parallel geometric constraints, obtain a correct set of center points and normal vectors for each of the top and bottom surfaces.

[0008] Step S4: Calculate the radii of the top and bottom surfaces based on the height of the frustum-shaped object and the normal vectors of the top and bottom surfaces;

[0009] Step S5: Based on the upper and lower radii and the corresponding correct circle centers, obtain the center of the upper and lower circles.

[0010] Furthermore, in step S3, since the upper and lower planes of the frustum are parallel, the correct upper and lower surface normal vectors can be obtained through the parallel geometric constraints of the upper and lower surfaces of the frustum. The calculation formula is as follows:

[0011]

[0012] in, The first normal vector of the upper plane is represented by the second normal vector. The solution and the first normal vector of the lower plane The angles between the solutions are taken, and the two solutions with the smallest angles are selected as the normal vectors of the upper and lower planes. The normal vector of the frustum is then calculated based on the normal vectors of the upper and lower planes. The calculation process is as follows:

[0013]

[0014] in Denotes the normal vector of the upper plane. This represents the normal vector of the lower plane. The final measured normal vector of the frustum. This represents a transformation into a unit vector; , This represents the two sets of normal vectors of the upper plane obtained in step S2; , These represent the two sets of normal vectors of the lower plane obtained in step S2.

[0015] Furthermore, in step S3, based on the correspondence between the circle center and the normal vector, the correct circle centers corresponding to the top and bottom are obtained. The calculation process is as follows:

[0016]

[0017] in, Indicates the center of the circle on the upper plane. Indicates the center of the circle in the lower plane; This represents the two sets of circle centers on the upper plane obtained in step S2; The two sets of circle centers on the lower plane obtained in step S2 are represented.

[0018] Furthermore, in step S3, the representation in the intermediate camera coordinate system is... , , , Transformed into camera coordinate system , , , The calculation formula is as follows:

[0019] .

[0020] Furthermore, in step S4, the center of the circle on the upper plane in the camera coordinate system calculated in step S3... The center of the lower plane , frustum normal vector According to the geometric constraints of the frustum, the direction vector of the line connecting the centers of the top and bottom circles is perpendicular to the normal vector of the frustum, and the distance between the lines connecting the centers is equal to the height of the frustum. We obtain the following system of equations:

[0021]

[0022] in, This indicates the distance between the centers of the top and bottom circles. , , They represent the center of the circle respectively. The x, y, z coordinates with respect to the radius The proportionality coefficient, , , Indicates the center of the circle. The x, y, z coordinates with respect to the radius The proportionality coefficient, , , Let x, y, and z represent the vector coordinates of the frustum normal vector along the x, y, and z axes, respectively. , This represents the radius of the upper and lower surfaces to be solved.

[0023] Furthermore, in step S5, the... , Bring into , In the middle, we obtain the coordinates of the center of the upper and lower circles.

[0024] Further, step S2 includes the following steps:

[0025] Step S2.1: Find two points on the ellipse such that the angle formed by the two points and the optical center of the camera is maximized;

[0026] Step S2.2: Based on the two points and the optical center, construct the intermediate camera coordinate system;

[0027] Step S2.3: Keep the camera optical center position unchanged, rotate the monocular camera so that the camera optical axis and the physical coordinate system of the camera image plane coincide with the corresponding intermediate camera coordinate system of the upper and lower planes, obtain the points in the intermediate camera coordinate system corresponding to the upper and lower planes, and calculate two sets of solutions for the center of the circle and the normal vector of the upper plane, and two sets of solutions for the center of the circle and the normal vector of the lower plane in the camera coordinate system.

[0028] Furthermore, in step S2.2, when observing the spatial target circular surface on the upper and lower planes using a camera, the optical center of the camera and two ellipses on the image plane form two observation cones, passing through the unit vectors Gp and Gq. and The coordinate system of the intermediate camera of the planar observation cone is calculated. , , Let p and q represent two points found on the ellipse. The optical center of the camera is represented by the following formula:

[0029]

[0030]

[0031] .

[0032] Furthermore, in step S2.3, the camera optical center is maintained. Position unchanged, , , Represent the three unit vectors of the intermediate camera coordinate system in the plane. Rotate the camera so that the camera optical axis is aligned with the unit vectors. The corresponding camera image plane physical coordinate system's x-axis and y-axis coincide with the unit vector. , They coincide, and p and q are two points found on the ellipse. For the optical center of the camera, Three points form a plane Passing through point and The perpendicular plane is L. ⊥ L ⊥ The intersection point with the ellipse is Two points, before the camera rotates. It is on the spatial circle and on the camera's imaging plane. At the corresponding point, after the camera rotates, the camera's imaging plane also rotates. At this time, the image plane intercepts the observation cone to obtain an ellipse. The point corresponding to the new imaging plane is ;

[0033] according to Seek Coordinates:

[0034]

[0035] in It is a scaling factor, the center of the target circle in space. yes The midpoint, yes The midpoint, therefore:

[0036]

[0037]

[0038] The formula for calculating the normal vector of the target circle in space is:

[0039]

[0040] Combining the properties of circles and the similarity relationship of triangles, the following equation can be written:

[0041]

[0042] By solving the above equations simultaneously, we obtain two sets of solutions for the center of the circle and the normal vector of the upper plane in the camera coordinate system:

[0043]

[0044] in, Point The coordinates in the x-axis direction, express Coordinates in the y-axis direction;

[0045] Similarly, two sets of solutions for the lower plane are obtained. and .

[0046] Furthermore, in step S1, the monocular camera is calibrated to obtain its intrinsic parameters and distortion coefficients. The image is then corrected based on the distortion coefficients, and edge detection is performed on the corrected image. The least squares method is used to detect two ellipses on the image.

[0047] The advantages and beneficial effects of this invention are as follows:

[0048] This invention utilizes the parallel relationship constraint between the spatial circles above and below a frustum. By constructing an intermediate camera coordinate system, it calculates two solutions for the center of the circles with unknown radius parameters and two solutions for the normal vectors of the upper and lower circular planes. Then, based on the parallel relationship between the upper and lower parts of the frustum, it obtains the correct center solution and the normal vectors of the upper and lower parts, eliminating the ambiguity in monocular circular pose estimation. Next, based on the known height of the frustum and the normal vectors of the upper and lower parts, it calculates the radii of the upper and lower parts. Finally, it substitutes the radii of the upper and lower circles into the correct center solution to obtain the coordinates of the center of the upper and lower parts, thereby enabling more accurate detection of frustum-shaped objects by a monocular camera. Attached Figure Description

[0049] Figure 1 This is a flowchart of the method in an embodiment of the present invention.

[0050] Figure 2 This is a schematic diagram of the coordinate system of the camera at the center of the spatial circle in an embodiment of the present invention. Detailed Implementation

[0051] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0052] like Figure 1 As shown, a method for detecting frustum-shaped objects based on a monocular camera includes the following steps:

[0053] Step S1: Calibrate the camera to obtain its intrinsic parameters and distortion coefficients. Correct the image based on the distortion coefficients. Then perform edge detection on the corrected image. Finally, use the least squares method to detect two ellipses in the image.

[0054] In this embodiment of the invention, the Zhang Zhengyou calibration method is used to calibrate the camera. The image is corrected based on the calibrated distortion coefficients. Edge detection is performed using the Canny edge detection algorithm. The least squares method is used to fit two ellipses on the image.

[0055] Step S2: Construct the intermediate camera coordinate system and calculate the two centers and two normal vectors of the upper and lower planes of the frustum-shaped object, each containing unknown radius parameters.

[0056] Step S2.1: As Figure 2 As shown, iterate through the pixels of the corresponding ellipse on the upper plane to find two points. , ,make maximum, Represent the optical center of the camera; traverse the pixels of the corresponding ellipse in the lower plane to find two points. , ,make maximum, These are all special points on the ellipse, obtained through calculation. Figure 2 The landing points shown are for illustrative purposes only.

[0057] Step S2.2: When observing the spatial target circular surface on the upper and lower planes using a camera, the optical center of the camera... Two ellipses on the image plane form two observation cones, which are traversed by vectors. and unit vector and The coordinate system of the intermediate camera on the upper plane observation cone is calculated. , , , through vector and unit vector and The unit vectors of the three coordinate axes of the intermediate camera coordinate system of the lower plane observation cone are calculated. , , The specific calculation formula is as follows:

[0058]

[0059]

[0060]

[0061] Step S2.3: Maintain the camera's optical center With the position unchanged, rotate the camera so that the optical axis is aligned with the unit vector. The corresponding camera image plane physical coordinate system's x-axis and y-axis coincide with the unit vector. , coincide, Three points form a plane Passing through point and The perpendicular plane is L. ⊥ L ⊥ The intersection point with the ellipse is Two points, before the camera rotates. It is on the spatial circle and on the camera's imaging plane. At the corresponding point, after the camera rotates, the camera's imaging plane also rotates. At this time, the image plane intercepts the observation cone to obtain an ellipse. The point corresponding to the new imaging plane is .

[0062] according to Seek Coordinates:

[0063]

[0064] in It is a scaling factor, the center of the target circle in space. yes The midpoint, yes The midpoint, therefore:

[0065]

[0066]

[0067] The formula for calculating the normal vector of the target circle in space is:

[0068]

[0069] Combining the properties of circles and the similarity relationship of triangles, the following equation can be written:

[0070]

[0071] By solving the above equations simultaneously, we obtain two sets of solutions for the center of the circle and the normal vector of the upper plane in the camera coordinate system:

[0072]

[0073] in, Point The coordinates in the x-axis direction, express Coordinates in the y-axis direction;

[0074] Similarly, two sets of solutions for the lower plane can be obtained. and .

[0075] Step S3: Based on the parallel relationship between the top and bottom surfaces of the frustum-shaped object, obtain a correct set of center points and normal vectors for each surface.

[0076] Because the top and bottom planes of a frustum-shaped object are parallel, the correct top and bottom normal vectors can be obtained through the parallel geometric constraints of the top and bottom of the frustum. The calculation formula is as follows:

[0077]

[0078] in, The first normal vector of the upper plane is represented by the second normal vector. The solution and the first normal vector of the lower plane The angles between the solutions are taken, and the two solutions with the smallest angles are selected as the normal vectors of the upper and lower planes. The normal vector of the frustum is then calculated based on the normal vectors of the upper and lower planes. The calculation process is as follows:

[0079]

[0080] in Denotes the normal vector of the upper plane. This represents the normal vector of the lower plane. The final measured normal vector of the frustum. This indicates that the vector is converted to a unit vector.

[0081] After obtaining the correct normal vector solution, based on the correspondence between the center solution and the normal vector solution, the correct center solutions corresponding to the top and bottom edges are obtained. The calculation process is as follows:

[0082]

[0083] in Indicates the center of the circle on the upper plane. Indicates the center of the circle in the lower plane.

[0084] , , , These are all representations in the intermediate camera coordinate system, and need to be transformed to represent them in the camera coordinate system. , , , The calculation formula is as follows:

[0085]

[0086] Step S4: Calculate the radii of the top and bottom surfaces based on the height of the frustum-shaped object and the normal vectors of the top and bottom surfaces.

[0087] The center of the upper plane is obtained through calculation in step S3. The center of the lower plane is The normal vector of the frustum is According to the geometric constraints of the frustum, the direction vector of the line connecting the centers of the top and bottom circles is perpendicular to the normal vector of the frustum, and the distance between the lines connecting the centers is equal to the height of the frustum. The following system of equations can be obtained:

[0088]

[0089] in It indicates the distance between the centers of the top and bottom circles. , This represents the radius of the top and bottom surfaces. The value can be determined by solving the equation. , .

[0090] Step S5: Substitute the radii of the upper and lower circles into the solution for the correct center to find the center of the upper and lower circles.

[0091] Will , Bring into , In the middle, we obtain the coordinates of the center of the upper and lower circles.

[0092] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting frustum-shaped objects based on a monocular camera, characterized in that... Includes the following steps: Step S1: Based on the monocular camera, obtain the ellipses on the top and bottom surfaces of the frustum-shaped object in the camera image; Step S2: Construct an intermediate camera coordinate system and calculate the center and normal vector of two sets of circles containing unknown radius parameters on both the top and bottom surfaces; Specifically, keeping the position of the camera's optical center unchanged, rotate the monocular camera so that the camera's optical axis and the physical coordinate system of the camera's image plane coincide with the corresponding intermediate camera coordinate system of the upper and lower planes. This yields two points in the intermediate camera coordinate system that correspond to the upper and lower planes. The camera's optical center and these two points form a first plane. Passing through a second plane perpendicular to the first plane, two more points are obtained on the upper and lower planes. Based on these four points, calculate the four points corresponding to the spatial target circle to obtain the center and normal vector of the spatial target circle. Combining the properties of the spatial target circle and the similarity relationship of the triangle formed by the four points and the camera's optical center, calculate two sets of solutions for the center and normal vector of the upper plane and the lower plane in the camera coordinate system. Step S3: Based on the parallel relationship between the top and bottom surfaces of the frustum-shaped object, obtain the parallel geometric constraints. Based on the parallel geometric constraints, obtain a correct set of center points and normal vectors for each of the top and bottom surfaces. Specifically, the angle between the normal vector of the upper plane and the normal vector of the lower plane is taken as the two solutions with the smallest angle as the normal vectors of the upper and lower planes, and the normal vector of the frustum is calculated based on the normal vectors of the upper and lower planes; based on the correspondence between the center of the circle and the normal vector, the correct center of the upper and lower planes is obtained. Step S4: Calculate the radii of the top and bottom surfaces based on the height of the frustum-shaped object and the normal vectors of the top and bottom surfaces; Specifically, the center of the circle on the upper plane in the camera coordinate system calculated through step S3. The center of the lower plane , frustum normal vector According to the geometric constraints of the frustum, the direction vector of the line connecting the centers of the top and bottom circles is perpendicular to the normal vector of the frustum, and the distance between the lines connecting the centers is equal to the height of the frustum. We obtain the following system of equations: in, This indicates the distance between the centers of the top and bottom circles. , , They represent the center of the circle respectively. The x, y, z coordinates with respect to the radius The proportionality coefficient, , , Indicates the center of the circle. The x, y, z coordinates with respect to the radius The proportionality coefficient, , , Let x, y, and z represent the vector coordinates of the frustum normal vector along the x, y, and z axes, respectively. , This represents the radius above and below the surface to be solved; Step S5: Based on the upper and lower radii and the corresponding correct circle centers, obtain the center of the upper and lower circles; Specifically, , Bring into , In the middle, we obtain the coordinates of the center of the upper and lower circles.

2. The method for detecting frustum-shaped objects based on a monocular camera according to claim 1, characterized in that: Step S2 includes the following steps: Step S2.1: Find two points on the ellipse such that the angle formed by the two points and the optical center of the camera is maximized; Step S2.2: Based on the two points and the optical center, construct the intermediate camera coordinate system; Step S2.3: Keep the camera optical center position unchanged, rotate the monocular camera so that the camera optical axis and the physical coordinate system of the camera image plane coincide with the corresponding intermediate camera coordinate system of the upper and lower planes, obtain the points in the intermediate camera coordinate system corresponding to the upper and lower planes, and calculate two sets of solutions for the center of the circle and the normal vector of the upper plane, and two sets of solutions for the center of the circle and the normal vector of the lower plane in the camera coordinate system.

3. The method for detecting frustum-shaped objects based on a monocular camera according to claim 2, characterized in that: In step S2.2, the unit vectors Gp and Gq are used. and The coordinate system of the intermediate camera of the planar observation cone is calculated. , , Let p and q represent two points found on the ellipse. The optical center of the camera is represented by the following formula: 。 4. The method for detecting frustum-shaped objects based on a monocular camera according to claim 2, characterized in that: In step S2.3, the camera optical center is maintained. Position unchanged, , , Represent the three unit vectors of the intermediate camera coordinate system in the plane. Rotate the camera so that the camera optical axis is aligned with the unit vectors. The corresponding camera image plane physical coordinate system's x-axis and y-axis coincide with the unit vector. , They coincide, and p and q are two points found on the ellipse. For the optical center of the camera, Three points form a plane Passing through point and The perpendicular plane is L. ⊥ L ⊥ The intersection point with the ellipse is Two points, before the camera rotates. It is on the spatial circle and on the camera's imaging plane. At the corresponding point, after the camera rotates, the camera's imaging plane also rotates. At this time, the image plane intercepts the observation cone to obtain an ellipse. The point corresponding to the new imaging plane is ; according to Seek Coordinates: in It is a scaling factor, the center of the target circle in space. yes The midpoint, yes The midpoint, therefore: The formula for calculating the normal vector of the target circle in space is: Combining the properties of circles and the similarity relationship of triangles, the following equation can be written: By solving the above equations simultaneously, we obtain two sets of solutions for the center of the circle and the normal vector of the upper plane in the camera coordinate system: in, Point The coordinates in the x-axis direction, express Coordinates in the y-axis direction; Similarly, two sets of solutions for the lower plane are obtained. and .

5. The method for detecting frustum-shaped objects based on a monocular camera according to claim 1, characterized in that: In step S3, the correct top and bottom surface normal vectors are obtained through the parallel geometric constraints of the frustum. The calculation formula is as follows: in, The first normal vector of the upper plane is represented by the second normal vector. The solution and the first normal vector of the lower plane The angles between the solutions are taken, and the two solutions with the smallest angles are selected as the normal vectors of the upper and lower planes. The normal vector of the frustum is then calculated based on the normal vectors of the upper and lower planes. The calculation process is as follows: in Denotes the normal vector of the upper plane. This represents the normal vector of the lower plane. The final measured normal vector of the frustum. This represents a transformation into a unit vector; , This represents the two sets of normal vectors of the upper plane obtained in step S2; , These represent the two sets of normal vectors of the lower plane obtained in step S2.

6. The method for detecting frustum-shaped objects based on a monocular camera according to claim 5, characterized in that: In step S3, the correct center of the circle corresponding to the top and bottom is obtained based on the correspondence between the circle center and the normal vector. The calculation process is as follows: in, Indicates the center of the circle on the upper plane. Indicates the center of the circle in the lower plane; This represents the two sets of circle centers on the upper plane obtained in step S2; The two sets of circle centers on the lower plane obtained in step S2 are represented.

7. The method for detecting frustum-shaped objects based on a monocular camera according to claim 6, characterized in that: In step S3, the representation in the intermediate camera coordinate system is... , , , Transformed into camera coordinate system , , , The calculation formula is as follows: 。 8. The method for detecting frustum-shaped objects based on a monocular camera according to claim 1, characterized in that: In step S1, the monocular camera is calibrated to obtain its intrinsic parameters and distortion coefficients. The image is then corrected based on the distortion coefficients, and edge detection is performed on the corrected image. The least squares method is used to detect two ellipses on the image.

Images