A camera pose estimation method, system and storage medium based on torus-like surface

Abstract

The present invention relates to the field of computer vision technology, and discloses a torus-like camera pose estimation method, system, and storage medium. The method comprises: establishing a test environment database; using a camera to navigate the test environment to obtain a real-time image, matching the real-time image with an image in the environment database to obtain a known image; establishing a homography matrix based on the real-time image and the known image; obtaining the pixel coordinates of the four corner points of the real-time image based on relevant information of the known image and the homography matrix; establishing a viewing cone with the real-time image as the base; establishing a torus-like surface based on the viewing cone and the pixel coordinates of the real-time image; establishing three torus-like parametric equations based on the pixel coordinates of the torus-like surface and the real-time image; solving the three torus-like parametric equations to determine the camera position; determining the camera orientation based on the camera position and the pixel coordinates of the real-time image, thereby determining the camera pose. The present invention improves the accuracy, efficiency, and generalization capability of camera pose estimation.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a torus-like-based camera pose estimation method, system, and storage medium. Background Art

[0002] Camera pose estimation is a key step in realizing self-driving cars and autonomous robots, and is also a prerequisite for augmented reality indoor navigation. With the continuous advancement of hardware technology and computing methods, camera positioning technology has made great progress. Currently, camera pose estimation is divided into traditional methods based on feature points and methods based on deep learning. Traditional methods include methods based on geometric structures and methods based on image retrieval. Geometric methods first extract feature points from the query image, then match the relationship between 2D feature points and 3D coordinate points in the scene model, and finally solve the camera pose. Image retrieval methods first feature encode the query image, retrieve neighboring images, and then match the relationship between 2D feature points on the image, and finally solve the camera pose. Deep learning-based methods for camera pose estimation mainly include two categories: one is to directly use neural networks to regress the camera pose or scene coordinates, and the other is to construct a 3D model by inputting a color image and its corresponding depth image, and finally obtain the camera pose corresponding to the image. Among them, when using deep learning methods to estimate camera pose, when the actual scene is very different from the training scene and the actual scene changes frequently, the estimation results are not accurate and the generalization ability is not strong. If the accuracy is to be improved, a new model can only be trained for the actual scene, which requires a lot of time and cost. The estimation efficiency is low and the cost is high.

[0003] The prior art discloses a method for estimating the pose of an object using a monocular camera based on deep learning. The method includes: generating a training set and a validation set based on the projection of a three-dimensional image of an object in a two-dimensional space, the object coordinates corresponding to the projection, and a label file of the object; using a cascaded convolutional neural network model to learn the training set and iterate hyperparameters; and using a test set to test the trained cascaded convolutional neural network model. When the accuracy of the trained cascaded convolutional neural network model is not less than a first preset threshold, the trained cascaded convolutional neural network model is used to estimate the pose of the object. The cascaded convolutional neural network model trained in the prior art has high camera pose estimation accuracy for scenes that are not much different from the training scene. However, for scenes that are significantly different from the training scene and the actual scene is constantly changing, the estimation result is not accurate and the ability to adapt to new environments is not strong, that is, the generalization ability is weak. To improve the estimation accuracy, new files can only be imported and a new model trained for the actual scene, which is inefficient and costly. Summary of the Invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a camera pose estimation method, system and storage medium based on a torus-like surface. The present invention can improve the accuracy, efficiency and generalization ability of camera pose estimation while reducing the cost of camera pose estimation.

[0005] In order to achieve the above object, the present invention provides a camera pose estimation method based on a torus-like surface, comprising:

[0006] Step S1: establishing a test environment database, wherein the test environment database includes a plurality of images and world coordinates and pixel coordinates corresponding to four corner points of each image;

[0007] Step S2: Obtaining real-time images , the real-time image Match the image with the image in the test environment database to obtain the real-time image Corresponding known image ;

[0008] Step S3: Create the real-time image and the known image The relevant homography matrix H;

[0009] Step S4: Based on the known image The pixel coordinates of the four corner points corresponding to the test environment database and the homography matrix H, obtain the real-time image The pixel coordinates of the four corresponding corner points ;

[0010] Step S5: Construct the real-time image is the bottom viewing cone;

[0011] Step S6: Using the Live Image The pixel coordinates of the four corresponding corner points Constructing a torus-like surface with the viewing cone;

[0012] Step S7: Construct the pixel coordinates of any position of the torus-like surface in the test environment according to the torus-like surface. The corresponding parametric equation of the first kind of torus;

[0013] Step S8: Select the live image The pixel coordinates of the four corresponding corner points and Repeat steps S6 to S7 to construct and The corresponding parametric equations of the second and third kinds of torus;

[0014] Step S9: Establish six functional optimization formulas to solve unknown variables based on the first, second, and third torus parametric equations, substitute the solved unknown variables into the first, second, and third torus parametric equations to obtain the world coordinates of the camera in the test environment. ;

[0015] Step S10: According to the world coordinates Calculating real-time images Pixel coordinates Corresponding real-time image World coordinates on , according to the world coordinates Calculate the camera's front vector;

[0016] Step S11: Calculate the camera's up vector based on the forward vector, and calculate the camera's up vector based on the world coordinates. , forward vector and up vector to obtain the position and orientation of the camera and complete the camera pose estimation.

[0017] Furthermore, step S1 specifically includes:

[0018] Step S1.1: Use a camera to capture a real scene of the test environment and obtain multiple images;

[0019] Step S1.2: Input each image into the program to obtain the four corner points A, B, C, and D corresponding to each image. The world coordinates corresponding to the four corner points A, B, C, and D are P, Q, R, and S, and the pixel coordinates corresponding to the four corner points A, B, C, and D are ;

[0020] Step S1.3: inputting the world coordinates and pixel coordinates corresponding to the multiple images and the four corner points of each image into a database, thereby establishing the test environment database.

[0021] Furthermore, step S3 specifically includes:

[0022] Step S3.1: extracting feature points of the real-time image and the known image F;

[0023] Step S3.2: Using a feature point matching detection algorithm, detect multiple feature points that match the real-time image and the known image F;

[0024] Step S3.3: Use The algorithm selects 4 or more pairs of matching feature points of the real-time image and the known image F, and uses the matching feature points of the real-time image and the known image F to establish the homography matrix H.

[0025] Furthermore, the feature point matching detection algorithm in step S3.2 is an image matching feature point detection algorithm based on HSV and Harris corner detection.

[0026] Furthermore, the specific process of step S5 includes:

[0027] Step S5.1: Live image The four vertices of ,in is the upper left corner vertex of the real-time image, is the lower left corner vertex of the real-time image, is the lower right corner vertex of the real-time image, The top right corner of the real-time image, with the camera position as the origin , calculate the horizontal field of view angle according to the camera internal parameters , vertical viewing angle and the physical focal length, which is the camera position to the real-time image distance;

[0028] Step S5.2: According to the horizontal viewing angle , vertical viewing angle and the physical focal length to construct the viewing frustum.

[0029] Furthermore, the viewing cone described in step S5 is constructed by the camera internal parameters to obtain the real-time image The device is an Android system, and the camera internal parameters are retrieved using the Android Camera2 API.

[0030] Furthermore, the forward vector in step S10 is determined by the following formula:

[0031]

[0032] in, Represents pixel coordinates The corresponding world coordinates; Represents pixel coordinates The corresponding world coordinates; Represents pixel coordinates The corresponding world coordinates.

[0033] Furthermore, the specific process of step S11 includes:

[0034] Step S11.1: In the real-time image Center line Find any point except the midpoint ,

[0035] and confirm The pixel coordinates of the center line With real-time image edge parallel;

[0036] Step S11.2: Construct a point A triangle with vertices, points is the midpoint of the center line, determined according to the Pythagorean theorem ;

[0037] Step S11.3: Convert the vector Forward vector Rotate the Find the vector ;

[0038] Step S11.4: Camera up vector Determined by the following formula:

[0039]

[0040] Furthermore, a torus-like camera pose estimation system includes:

[0041] A database establishment module is used to establish a test environment database, wherein the test environment database includes a plurality of images and the world coordinates and pixel coordinates corresponding to the four corner points of each image;

[0042] The first acquisition module: used to obtain real-time images , the real-time image Match the image with the image in the test environment database to obtain the real-time image Corresponding known image ;

[0043] The second acquisition module is used to acquire the real-time image and the known image The relevant homography matrix H;

[0044] The third acquisition module is used to obtain the known image The pixel coordinates of the four corner points corresponding to the test environment database and the homography matrix H, obtain the real-time image The pixel coordinates of the four corresponding corner points ;

[0045] Building a frustum module: used to build the real-time image is the bottom viewing cone;

[0046] Create a torus-like module: for using real-time images The pixel coordinates of the four corresponding corner points Constructing a torus-like surface with the viewing cone;

[0047] The first parameter equation module is used to construct the pixel coordinates of any position of the torus in the test environment according to the torus The corresponding parametric equation of the first kind of torus;

[0048] The second parameter equation module is used to select real-time images. The pixel coordinates of the four corresponding corner points and Repeat steps S6 to S7 to construct and the corresponding parametric equations of the second and third kinds of torus;

[0049] The fourth acquisition module is used to establish six functional optimization solutions to unknown variables based on the first-class torus parametric equation, the second-class torus equation, and the third-class torus equation, and substitute the solved unknown variables into the first-class torus parametric equation, the second-class torus equation, and the third-class torus equation to obtain the world coordinates of the camera in the test environment. ;

[0050] The first calculation module: according to the world coordinate Calculating real-time images Pixel coordinates Corresponding real-time image World coordinates on , according to the world coordinates Calculate the camera's front vector;

[0051] The second calculation module: calculates the camera's up vector according to the front vector, and calculates the camera's up vector according to the world coordinate , forward vector and up vector to obtain the position and orientation of the camera and complete the camera pose estimation.

[0052] In addition, the present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the torus-like-based camera pose estimation method is implemented.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] When testing the camera pose in a new environment or a changing environment, the present invention only needs to establish a new environment database, avoiding the need to re-import new files to train the neural network model when the deep learning method for determining the camera pose faces a new environment or a changing environment, thereby improving the efficiency and generalization ability of camera pose estimation and reducing the cost of camera pose estimation; the present invention also determines the position of the camera by solving the intersection of the torus-like equation, and determines the direction of the camera according to the position of the camera to determine the pose of the camera, thereby improving the accuracy of camera pose estimation by establishing and solving the torus-like equation. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flowchart of a method for estimating camera pose based on a torus-like surface according to an embodiment of the present invention;

[0056] Figure 2 This is a block diagram of a camera pose estimation system based on a torus-like surface according to an embodiment of the present invention;

[0057] Figure 3 It is a monocular vision imaging coordinate system of a camera pose estimation method based on a torus-like surface in an embodiment of the present invention;

[0058] Figure 4 This is a camera pose estimation method based on a torus-like surface according to an embodiment of the present invention. Figure 3 The imaging cone model after the cone is flipped;

[0059] Figure 5 It is a trajectory diagram of the vertices of the viewing cone on a plane in a camera pose estimation method based on a torus-like surface according to an embodiment of the present invention;

[0060] Figure 6 The present invention is a camera pose estimation method based on a torus-like surface, and the motion trajectory of the cone is around PQ The complete torus-like field obtained by string rotation;

[0061] Figure 7 The embodiment of the present invention is a camera pose estimation method based on a torus-like surface. Figure 6 the lower half of the torus-like field;

[0062] Figure 8 A method for estimating a camera pose based on a torus-like surface according to an embodiment of the present invention solves an upper vector graph based on a forward vector;

[0063] Figure 9 A sparse point cloud for three-dimensional reconstruction of a camera pose estimation method based on a torus-like surface according to an embodiment of the present invention;

[0064] Figure 10 This is a camera pose estimation method based on a torus-like surface according to an embodiment of the present invention. Figure 9Dense point cloud based on the foundation;

[0065] Figure 11 1 is an upper left perspective image and an upper right perspective image of a camera pose estimation method based on a torus-like surface according to an embodiment of the present invention;

[0066] Figure 12 1 is a side view and a front view of a camera pose estimation method based on a torus-like surface according to an embodiment of the present invention;

[0067] Figure 13 The present invention provides a bird's-eye view of an indoor positioning and navigation path using a torus-like camera pose estimation method. DETAILED DESCRIPTION

[0068] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0069] Example 1:

[0070] like Figure 1 As shown, a camera pose estimation method based on a torus-like surface according to a preferred embodiment of the present invention includes:

[0071] Step S1: Establish a test environment database, which includes multiple images and the world coordinates and pixel coordinates corresponding to the four corner points of each image;

[0072] Step S2: Obtaining real-time images , the real-time image Match the image with the test environment database to obtain the real-time image Corresponding known image ;

[0073] Step S3: Create and live image and known images The relevant homography matrix H;

[0074] Step S4: Based on the known image The pixel coordinates of the four corner points corresponding to the test environment database And homography matrix H, get real-time image The pixel coordinates of the four corresponding corner points ;

[0075] Step S5: Constructing a real-time image is the bottom viewing cone;

[0076] Step S6: Using the Live Image The pixel coordinates of the four corresponding corner points Construct a torus-like surface with the viewing cone;

[0077] Step S7: Construct the pixel coordinates of any position of the torus-like surface in the test environment according to the torus-like surface The corresponding parametric equation of the first kind of torus;

[0078] Step S8: Select the live image The pixel coordinates of the four corresponding corner points and Repeat steps S6 to S7 to construct and The corresponding parametric equations of the second and third kinds of torus;

[0079] Step S9: Establish six functional optimization formulas to solve unknown variables based on the first-class torus parametric equation, the second-class torus equation, and the third-class torus equation. Substitute the solved unknown variables into the first-class torus parametric equation, the second-class torus equation, and the third-class torus equation to obtain the world coordinates of the camera in the test environment. ;

[0080] Step S10: According to the world coordinates Calculating real-time images Pixel coordinates Corresponding real-time image World coordinates on , according to the world coordinates Calculate the camera's front vector;

[0081] Step S11: Calculate the camera's up vector based on the forward vector, and calculate the camera's up vector based on the world coordinates. , forward vector and up vector to obtain the position and orientation of the camera and complete the camera pose estimation.

[0082] When testing the camera pose in a new environment or a changing environment, the present invention only needs to establish a new environment database, avoiding the need to re-import new files to train the neural network model when the deep learning method for determining the camera pose faces a new environment or a changing environment, thereby improving the efficiency and generalization ability of camera pose estimation and reducing the cost of camera pose estimation; the present invention also determines the position of the camera by solving the intersection of the torus-like equation, and determines the direction of the camera according to the position of the camera to determine the pose of the camera, thereby improving the accuracy of camera pose estimation by establishing and solving the torus-like equation.

[0083] Example 2:

[0084] like Figure 1 As shown, a camera pose estimation method based on a torus-like surface according to an embodiment of the present invention includes:

[0085] include:

[0086] Step S1: Establish a test environment database, which includes multiple images and the world coordinates and pixel coordinates corresponding to the four corner points of each image;

[0087] In this embodiment, establishing a test environment database includes the following steps:

[0088] Step S1.1: Use a camera to capture a real scene of the test environment and obtain multiple images;

[0089] Step S1.2: Input each image into the program and obtain the four corner points A, B, C, and D corresponding to each image. The world coordinates corresponding to the four corner points A, B, C, and D are P, Q, R, and S, and the pixel coordinates corresponding to the four corner points A, B, C, and D are ;

[0090] Step S1.3: Input the multiple images and the world coordinates and pixel coordinates corresponding to the four corner points of each image into the database, thereby establishing a test environment database.

[0091] Step S2: Obtaining real-time images , the real-time image Match the image with the test environment database to obtain the real-time image Corresponding known image ;

[0092] In this embodiment, a professional camera or mobile phone or other device with a shooting function can be used to cruise in the test environment to obtain real-time images. When performing image matching, a fast image comparison and matching algorithm is used, prioritizing the search for previously used known images. If the previous image is no longer suitable, the previously used image set is searched and compared from near to far, combining the previous position. This significantly reduces search and matching time. This algorithm determines which known image F in the database the camera is currently capturing.

[0093] Step S3: Create and live image and known images The relevant homography matrix H;

[0094] In this embodiment, the specific process of step S3 includes:

[0095] Step S3.1: extracting feature points of the real-time image and the known image F;

[0096] Step S3.2: Using a feature point matching detection algorithm, detect multiple feature points that match the real-time image and the known image F. It should be noted that the feature point matching detection algorithm used in this step is an image matching feature point detection algorithm based on HSV and Harris corner detection;

[0097] Step S3.3: Use The algorithm selects 4 or more pairs of matching feature points between the real-time image and the known image F, and uses the matching feature points between the real-time image and the known image F to establish the homography matrix H.

[0098] Step S4: Based on the known image The pixel coordinates of the four corner points corresponding to the test environment database And homography matrix H, get real-time image The pixel coordinates of the four corresponding corner points ;

[0099] Step S5: Constructing a real-time image is the bottom viewing cone;

[0100] In an embodiment, step S5 specifically includes:

[0101] Step S5.1: Live image The four vertices of ,in is the upper left corner of the real-time image, is the lower left corner vertex of the real-time image, is the lower right corner of the real-time image. The upper right corner of the real-time image, with the camera position as the origin , calculate the horizontal field of view angle based on the camera internal parameters , vertical viewing angle and the physical focal length, which is the camera position to live image The above content can be obtained from the camera parameters to obtain real-time images. The device is an Android system, and the camera internal parameters are retrieved using the Android Camera2 API;

[0102] Step S5.2: Based on the horizontal viewing angle , vertical viewing angle and the physical focal length to construct the viewing frustum.

[0103] Step S6: Using the Live Image The pixel coordinates of the four corresponding corner points Construct a torus-like surface with the viewing cone;

[0104] In this embodiment, select the real-time image The pixel coordinates of the four corresponding corner points and the viewing cone to construct a torus-like surface, first solve ,Right now ξ , through the horizontal field of view angle ∠LOM, vertical field of view angle ∠NOM and physical focal length OC , calculate the physical height of the imaging surface KL and width KN :

[0105]

[0106]

[0107] in is ∠LOM, is ∠NOM.

[0108] The center pixel coordinates of the known imaging surface C , high resolution of imaging surface h img ,Width w img And the physical length of the imaging surface KL, KN, through the ratio t get , C Physical distance between:

[0109]

[0110] in, Cp img , Cq img , pq img is the Euclidean distance between pixel coordinates, It is the physical length in the real world. Combined with the physical focal length OC and the Pythagorean theorem, we can get The physical length of , finally according to the cosine theorem, can be calculated ξ :

[0111]

[0112] Step S7: Construct the pixel coordinates of any position of the torus-like surface in the test environment according to the torus-like surface The corresponding parametric equation of the first kind of torus;

[0113] In this embodiment, in order to obtain the parametric equation of the torus-like surface at any position in space, it is necessary to first establish the parametric equation of the space circle. PQ The parametric equation of a circle with the midpoint of the chord as the center and the PQ vector as the normal vector is:

[0114]

[0115] If a is 0, then the PQ vector is cross-producted with the vector (0,0,1) or the vector (1,0,0) to obtain a. The above formula is the center of the circle. O 1's trajectory.

[0116] Known P , Q The world coordinates of the points, and their O Angle at point ξ , we can find O 1 is the center of the circle, PQ The radius of the space circle whose chord is R With the center O 1 to string PQ distance h :

[0117]

[0118] Then point O 1 is the center of the circle, R As the radius, construct the parametric equation of the outer circle, and then you can get the parametric equation of the first type of torus:

[0119]

[0120] Step S8: Select the live image The pixel coordinates of the four corresponding corner points and Repeat steps S6 to S7 to construct and The corresponding parametric equations of the second and third kinds of torus;

[0121] In this embodiment, it is assumed that The constructed torus-like parametric equation has the subscript 1, The constructed torus-like parametric equation has the subscript 2, The subscript of the constructed torus-like parametric equation is 3. The three torus-like parametric equations at different positions can be iteratively solved for the intersection coordinates using numerical optimization methods. Six functional formulas are established to optimize the six unknown variables in the three torus-like parametric equations. :

[0122]

[0123] Step S9: Establish six functional optimization formulas to solve unknown variables based on the first-class torus parametric equation, the second-class torus equation, and the third-class torus equation. Substitute the solved unknown variables into the first-class torus parametric equation, the second-class torus equation, and the third-class torus equation to obtain the world coordinates of the camera in the test environment. ;

[0124] In this embodiment, the quasi-Newton method (Broyden-Fletcher-Goldfarb-Shanno, BFGS) is used to solve the intersection point, that is, the position O of the camera in the world coordinate system.

[0125] Step S10: According to the world coordinates Calculating real-time images Pixel coordinates Corresponding real-time image World coordinates on , according to the world coordinates Calculate the camera's front vector;

[0126] In this embodiment, the distance between OP and OQ can be obtained by using the world coordinates of the point O. The proportional relationship is used to calculate the world coordinates S0 and S1 of the pixel corner points p and q projected on the imaging plane;

[0127]

[0128] According to the above formula, we can find the world coordinates of another corner point projected on the image plane. Assume that the world coordinates of the corner point corresponding to r are S2. To ensure that the direction of the front vector points forward, S0, S1, and S2 correspond to the upper left corner, upper right corner, and lower right corner of the poster respectively. The front vector X is obtained by vector cross product:

[0129]

[0130] Step S11: Calculate the camera's up vector based on the forward vector, and calculate the camera's up vector based on the world coordinates. , forward vector and up vector to obtain the position and orientation of the camera and complete the camera pose estimation.

[0131] In this embodiment, to solve the up vector based on the front vector, it is also necessary to find a point T on the imaging plane and parallel to the center line JI of the KN side that is different from the pixel position of point C, and select the pixel coordinate point , and determine the pixel coordinates of point T, whose horizontal coordinate is The horizontal coordinate is the vertical coordinate of C, and the vertical coordinate is the vertical coordinate of C. , according to the Pythagorean theorem, we can find , that is β, then the OT vector can be obtained by rotating the OS0 vector around the front vector f by an angle of β, and the up vector u of the camera position point O can be obtained by cross-producting the unit vector of OT with the front vector f.

[0132] Example 3:

[0133] like Figure 2 As shown in FIG, a block diagram of a camera pose estimation system based on a torus-like surface according to an embodiment of the present invention includes:

[0134] Database establishment module: used to establish a test environment database, which includes multiple images and the world coordinates and pixel coordinates corresponding to the four corner points of each image;

[0135] The first acquisition module: used to obtain real-time images , the real-time image Match the image with the test environment database to obtain the real-time image Corresponding known image ;

[0136] The second acquisition module: used to acquire established and real-time images and known images The relevant homography matrix H;

[0137] The third acquisition module: used to obtain the image The pixel coordinates of the four corner points corresponding to the test environment database And homography matrix H, get real-time image The pixel coordinates of the four corresponding corner points ;

[0138] Building a frustum module: used to build real-time images is the bottom viewing cone;

[0139] Create a torus-like module: for using real-time images The pixel coordinates of the four corresponding corner points Construct a torus-like surface with the viewing cone;

[0140] The first parameter equation module is used to construct the pixel coordinates of any position of the torus in the test environment based on the torus. The corresponding parametric equation of the first kind of torus;

[0141] The second parameter equation module is used to select real-time images. The pixel coordinates of the four corresponding corner points and Repeat steps S6 to S7 to construct and the corresponding parametric equations of the second and third kinds of torus;

[0142] The fourth acquisition module is used to establish six functional optimization solutions to unknown variables based on the first-class torus parametric equation, the second-class torus equation, and the third-class torus equation, and substitute the solved unknown variables into the first-class torus parametric equation, the second-class torus equation, and the third-class torus equation to obtain the world coordinates of the camera in the test environment. ;

[0143] The first calculation module: according to the world coordinates Calculating real-time images Pixel coordinates Corresponding real-time image World coordinates on , according to the world coordinates Calculate the camera's front vector;

[0144] The second calculation module: calculate the camera's up vector based on the front vector, and the world coordinates , forward vector and up vector to obtain the position and orientation of the camera and complete the camera pose estimation.

[0145] When testing the camera pose in a new environment or a changing environment, the present invention only needs to establish a new environment database, avoiding the need to re-import new files to train the neural network model when the deep learning method for determining the camera pose faces a new environment or a changing environment, thereby improving the efficiency and generalization ability of camera pose estimation and reducing the cost of camera pose estimation; the present invention also determines the position of the camera by solving the intersection of the torus-like equation, and determines the direction of the camera according to the position of the camera to determine the pose of the camera, thereby improving the accuracy of camera pose estimation by establishing and solving the torus-like equation.

[0146] Example 4:

[0147] Finally, the present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a camera pose estimation method based on a torus-like surface is implemented.

[0148] In summary, the embodiments of the present invention provide a camera pose estimation method, system and storage medium based on a torus-like surface. When testing the camera pose in a new environment or a changing environment, the present invention only needs to establish a new environment database, avoiding the need for deep learning to determine the camera pose method to re-import new files to train the neural network model when facing a new environment or a changing environment, thereby improving the efficiency and generalization ability of camera pose estimation and reducing the cost of camera pose estimation. The present invention also determines the position of the camera by solving the intersection of the torus-like surface equation, and determines the orientation of the camera according to the position of the camera to determine the pose of the camera. The accuracy of camera pose estimation is improved by establishing and solving the torus-like surface equation.

[0149] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention. These improvements and substitutions should also be regarded as the scope of protection of the present invention.

Claims

1. A camera pose estimation method based on a torus-like surface, characterized in that: include: Step S1: establishing a test environment database, wherein the test environment database includes a plurality of images and world coordinates and pixel coordinates corresponding to four corner points of each image; Step S2: Obtaining real-time images , the real-time image Match the image with the image in the test environment database to obtain the real-time image Corresponding known image ; Step S3: Create the real-time image and the known image The relevant homography matrix H; Step S4: Based on the known image The pixel coordinates of the four corner points corresponding to the test environment database and the homography matrix H, obtain the real-time image The pixel coordinates of the four corresponding corner points ; Step S5: Construct the real-time image is the bottom viewing cone; Step S6: Using the Live Image The pixel coordinates of the four corresponding corner points Constructing a torus-like surface with the viewing cone; Step S7: Construct the pixel coordinates of any position of the torus-like surface in the test environment according to the torus-like surface. The corresponding parametric equation of the first kind of torus; Step S8: Select the live image The pixel coordinates of the four corresponding corner points and Repeat steps S6 to S7 to construct and The corresponding parametric equations of the second and third kinds of torus; Step S9: Establish six functional optimization formulas to solve unknown variables based on the first, second, and third torus parametric equations, substitute the solved unknown variables into the first, second, and third torus parametric equations to obtain the world coordinates of the camera in the test environment. ; Step S10: According to the world coordinates Calculating real-time images Pixel coordinates Corresponding real-time image World coordinates on , according to the world coordinates Calculate the camera's front vector; Step S11: Calculate the camera's up vector based on the forward vector, and calculate the camera's up vector based on the world coordinates. , forward vector and up vector to obtain the position and orientation of the camera and complete the camera pose estimation.

2. A camera pose estimation method based on a torus-like surface according to claim 1, characterized in that: Step S1 specifically includes: Step S1.1: Use a camera to capture a real scene of the test environment and obtain multiple images; Step S1.2: Input each image into the program to obtain the four corner points A, B, C, and D corresponding to each image. The world coordinates corresponding to the four corner points A, B, C, and D are P, Q, R, and S, and the pixel coordinates corresponding to the four corner points A, B, C, and D are ; Step S1.3: inputting the world coordinates and pixel coordinates corresponding to the multiple images and the four corner points of each image into a database, thereby establishing the test environment database.

3. A camera pose estimation method based on a torus-like surface according to claim 2, characterized in that: Step S3 specifically includes: Step S3.1: extracting feature points of the real-time image and the known image F; Step S3.2: Using a feature point matching detection algorithm, detect multiple feature points that match the real-time image and the known image F; Step S3.3: Use The algorithm selects 4 or more pairs of matching feature points of the real-time image and the known image F, and uses the matching feature points of the real-time image and the known image F to establish the homography matrix H.

4. A camera pose estimation method based on a torus-like surface according to claim 3, characterized in that: The feature point matching detection algorithm described in step S3.2 is an image matching feature point detection algorithm based on HSV and Harris corner detection.

5. A method for estimating camera pose based on a torus-like surface according to claim 4, characterized in that: The specific process of step S5 includes: Step S5.1: Live image The four vertices of ,in is the upper left corner vertex of the real-time image, is the lower left corner vertex of the real-time image, is the lower right corner vertex of the real-time image, The top right corner of the real-time image, with the camera position as the origin , calculate the horizontal field of view angle according to the camera internal parameters , vertical viewing angle and the physical focal length, which is the camera position to the real-time image distance; Step S5.2: According to the horizontal viewing angle , vertical viewing angle and the physical focal length to construct the viewing frustum.

6. A method for estimating camera pose based on a torus-like surface according to claim 5, characterized in that: The viewing cone described in step S5 is constructed by the camera internal parameters to obtain real-time images The device is an Android system, and the camera internal parameters are retrieved using the AndroidCamera2 API.

7. A method for estimating camera pose based on a torus-like surface according to claim 6, characterized in that: The forward vector in step S10 is determined by the following formula: in, Represents pixel coordinates The corresponding world coordinates; Represents pixel coordinates The corresponding world coordinates; Represents pixel coordinates The corresponding world coordinates.

8. A method for estimating camera pose based on a torus-like surface according to claim 7, characterized in that: The specific process of step S11 includes: Step S11.1: In the real-time image Center line Find any point except the midpoint , and confirm The pixel coordinates of the center line With real-time image edge parallel; Step S11.2: Construct a point A triangle with vertices, points is the midpoint of the center line, determined according to the Pythagorean theorem ; Step S11.3: Convert the vector Forward vector Rotate the Find the vector ; Step S11.4: Camera up vector Determined by the following formula: 。 9. A camera pose estimation system based on a torus-like surface, characterized in that: include: A database establishment module is used to establish a test environment database, wherein the test environment database includes a plurality of images and the world coordinates and pixel coordinates corresponding to the four corner points of each image; The first acquisition module: used to obtain real-time images , the real-time image Match the image with the image in the test environment database to obtain the real-time image Corresponding known image ; The second acquisition module is used to acquire the real-time image and the known image The relevant homography matrix H; The third acquisition module is used to obtain the known image The pixel coordinates of the four corner points corresponding to the test environment database and the homography matrix H, obtain the real-time image The pixel coordinates of the four corresponding corner points ; Building a frustum module: used to build the real-time image is the bottom viewing cone; Create a torus-like module: for using real-time images The pixel coordinates of the four corresponding corner points Constructing a torus-like surface with the viewing cone; The first parameter equation module is used to construct the pixel coordinates of any position of the torus in the test environment according to the torus The corresponding parametric equation of the first kind of torus; The second parameter equation module is used to select real-time images. The pixel coordinates of the four corresponding corner points and Repeat steps S6 to S7 to construct and the corresponding parametric equations of the second and third kinds of torus; The fourth acquisition module is used to establish six functional optimization solutions to unknown variables based on the first-class torus parametric equation, the second-class torus equation, and the third-class torus equation, and substitute the solved unknown variables into the first-class torus parametric equation, the second-class torus equation, and the third-class torus equation to obtain the world coordinates of the camera in the test environment. ; The first calculation module: according to the world coordinate Calculating real-time images Pixel coordinates Corresponding real-time image World coordinates on , according to the world coordinates Calculate the camera's front vector; The second calculation module: calculates the camera's up vector according to the front vector, and calculates the camera's up vector according to the world coordinate , forward vector and up vector to obtain the position and orientation of the camera and complete the camera pose estimation.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the camera pose estimation method based on a torus-like surface according to any one of claims 1 to 8 is implemented.

Images