A map coordinate optimization method, system, and storage medium

Abstract

This application proposes a map coordinate optimization method, system, and storage medium. In the map coordinate optimization process, GPS is incorporated as a constraint. The application divides the global map into multiple local maps, using the initial pose of the starting frame within each local map as the starting point. Then, the initial poses of keyframes within each local map are obtained. Subsequently, the zenith poses of the starting frames and keyframes within each local map are obtained, and a transformation matrix is ​​calculated as a global constraint for subsequent optimization. The initial poses of the starting frames within each local map are optimized using GPS global constraints, resulting in the final poses of the starting frames within each local map. Based on the final poses of the starting frames within each local map, the final poses of the keyframes within each local map are obtained. Based on the final poses of the starting frames and keyframes within each local map, the map point coordinates of the starting frames and keyframes within each local map are obtained. Finally, the map point coordinates of all starting frames and keyframes are integrated to obtain the global map.

Description

Technical Field

[0001] This application relates to the field of visual real-time positioning and mapping technology, specifically to a map coordinate optimization method, system, and storage medium. Background Technology

[0002] In Visual Simultaneous Localization and Mapping (VSLAM), the pose optimization of keyframes in the map is often performed by the backend localmapping thread. In the backend pose optimization, visual matching information and IMU inertial navigation information are usually used to construct the residual of the pose nonlinear optimization problem.

[0003] In the current VSLAM, the visual constraints and IMU constraints on which the backend optimization thread is based can estimate the relative pose in the local path relatively well, and then transform it into the global coordinate system of the map through the pose transformation relationship. However, both visual constraints and IMU constraints will introduce cumulative errors during the camera's movement, which will lead to errors in the pose estimation of subsequent key frames.

[0004] Furthermore, relying solely on visual and IMU constraints is insufficient to eliminate these accumulated errors. When a path cannot be looped back, it is even more difficult to repair the accumulated errors through loop closure detection. Due to the cumulative pose error, the map points generated during the mapping process will also deviate significantly from the actual scene. Summary of the Invention

[0005] To address the aforementioned issues, this application proposes a map coordinate optimization method, system, and storage medium, incorporating GPS constraints during the map coordinate optimization process.

[0006] This method results in more accurate pose estimation for the start frame and keyframes, greater robustness, and more accurate map creation.

[0007] Firstly, this application proposes a map coordinate optimization method, which specifically includes the following steps:

[0008] Step 1: Use the first pose of the starting frame in each local map as the local reference point.

[0009] Step 2: Based on the first pose of the starting frame in each local map, obtain the first pose of the key frame in each local map.

[0010] Step 3: Obtain GPS information for the starting frame and key frame in each local map, and calculate the second pose of the starting frame and the second pose of the key frame in each local map based on the GPS information.

[0011] Step 4: Establish GPS global constraints based on the first pose of the starting frame, the first pose of the key frame, the second pose of the starting frame, and the second pose of the key frame in each local map.

[0012] Step 5: Optimize the first pose of the starting frame in each local map using GPS global constraints to obtain the final pose of the starting frame in each local map. Based on the final pose of the starting frame in each local map, obtain the final pose of the key frame in each local map.

[0013] Step 6: Based on the final pose of the starting frame and the final pose of the key frame in each local map, obtain the map point coordinates of the starting frame and key frame in each local map, and then obtain the global map.

[0014] This method incorporates GPS global constraints during map coordinate optimization, which can significantly reduce the impact of cumulative errors on the map creation process. Furthermore, the GPS global constraints only apply to the starting frame within each local map, preventing them from being used as a strong constraint that affects the entire map coordinate optimization process. As a result, the map obtained through this method is more accurate.

[0015] Furthermore, step 2 specifically includes:

[0016] Step 21: Obtain the relative pose of keyframes within each local map to the starting frame within each local map, and perform initial optimization by combining visual constraints and IMU constraints to obtain the result after initial optimization.

[0017] Step 22: After filtering the results of the initial optimization through reprojection error, perform a second optimization to obtain the first pose of the keyframes in each local map.

[0018] The first pose of the keyframe in each local map is obtained after visual and IMU constraints. It should be noted that the first pose of the starting frame in each local map is the starting point of each local map and also the standard point for visual and IMU constraints.

[0019] Furthermore, the GPS information in the step specifically includes: the longitude, latitude, and altitude of the starting frame or key frame within each local map.

[0020] In the second pose coordinate system, the x-axis points east, the y-axis points north, and the z-axis is perpendicular to the ground and pointing upwards. To obtain the second pose of the starting frame or the second pose of the key frame, the required information is the longitude, latitude, and height of the starting frame or the key frame.

[0021] Furthermore, the construction of the GPS global constraints is specifically as follows:

[0022] Step 31: Calculate the transformation matrix between the second pose and the first pose within the first local map, and use this as the global transformation matrix from the second pose to the first pose.

[0023] Step 32: After obtaining the transformation matrix, the starting frame and key frame in the subsequent local map are transformed from the second pose to the first pose through the transformation matrix, which serves as the subsequent GPS global constraint.

[0024] Furthermore, the calculation of the transformation matrix between the first pose and the second pose within the first local map is specifically as follows:

[0025] Calculate multiple residual matrices between the first pose and the second pose of the starting frame in the first local map, as well as between the first pose and the second pose of the preset number of keyframes. Based on these multiple residual matrices, calculate the transformation matrix.

[0026] In the global map, the transformation relationship between the first pose and the second pose is fixed. By calculating the transformation matrix between the first pose and the second pose at a preset number of frames, the transformation matrix between the first pose and the second pose in the global map can be obtained.

[0027] Furthermore, step 5 specifically includes:

[0028] Step 51: Based on the first pose of the starting frame in each local map, the first pose of the preset frame key frame and GPS global constraints, the correction matrix of each local map is obtained.

[0029] Step 52: Correct the first pose of the starting frame in each local map using the correction matrix to obtain the final pose of the starting frame in each local map.

[0030] Step 53: Based on the final pose of the starting frame in each local map and the first pose of the keyframe in each local map, obtain the final pose of the keyframe in each local map.

[0031] Based on the first pose of the starting frame, the first pose of the preset keyframe, and GPS global constraints, the third pose of the starting frame and the third pose of the preset keyframe can be obtained. The correction matrix is ​​the result of nonlinear optimization calculation of the first pose of the starting frame, the first pose of the preset keyframe, the third pose of the starting frame, and the third pose of the preset keyframe.

[0032] Furthermore, step 5 also includes:

[0033] When the number of keyframes in the local map is less than the preset value, the first pose of the starting frame in the local map is taken as the final pose of the starting frame in the local map.

[0034] When the number of keyframes in a local map is less than a preset value, the first pose of the keyframe in the local map is used as the final pose of the keyframe in the local map.

[0035] There is a deviation between the first and third poses of the starting frame and the keyframe in each local map. The first and third poses of the starting frame and the preset number of keyframes in each local map are taken, and the average deviation is calculated to obtain the correction matrix of each local map. Then, the first pose of the starting frame of each local map is corrected with the correction matrix of each local map to obtain the final pose of the starting frame in each local map. Based on the final pose of the starting frame in each local map and the relative pose of the keyframes in each local map and the starting frame in each local map, the final pose of the keyframe in each local map is obtained.

[0036] It should be noted that GPS global constraints only apply to the starting frame within a local map. This is to prevent GPS global constraints from acting as strong constraints and affecting the entire map coordinate optimization process.

[0037] Secondly, this application proposes a system for map coordinate optimization, specifically including:

[0038] Start Module: Used to take the first pose of the starting frame in each local map as a local reference point.

[0039] First pose acquisition module: used to obtain the first pose of key frames in each local map based on the first pose of the starting frame in each local map.

[0040] The second pose acquisition module acquires GPS information of the starting frame and key frame in each local map, and calculates the second pose of the starting frame and the second pose of the key frame in each local map based on the GPS information.

[0041] GPS global constraint establishment module: Establishes GPS global constraints based on the first pose of the starting frame, the first pose of the key frame, the second pose of the starting frame, and the second pose of the key frame in each local map.

[0042] Final pose calculation module: Optimizes the first pose of the starting frame in each local map using GPS global constraints to obtain the final pose of the starting frame in each local map. Based on the final pose of the starting frame in each local map, the final pose of the key frame in each local map is obtained.

[0043] Integration module: Optimizes the first pose of the starting frame in each local map using GPS global constraints to obtain the final pose of the starting frame in each local map, and obtains the final pose of the key frame in each local map based on the final pose of the starting frame in each local map.

[0044] Furthermore, the final pose calculation module also includes:

[0045] Correction matrix acquisition unit: Based on the first pose of the starting frame in each local map, the first pose of the preset number of key frames, and GPS global constraints, the correction matrix of each local map is obtained.

[0046] Starting frame correction unit: Corrects the first pose of the starting frame in each local map with the correction matrix to obtain the final pose of the starting frame in each local map.

[0047] Keyframe Correction Unit: Based on the final pose of the starting frame in each local map and the first pose of the keyframe in each local map, the final pose of the keyframe in each local map is obtained.

[0048] Thirdly, based on the same inventive concept, this application also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a control processor, implement the map coordinate optimization method as described above.

[0049] It should be noted that since the computer-readable storage medium in this embodiment is based on the same inventive concept as the map coordinate optimization method in the above embodiments, the corresponding content in the method embodiments is also applicable to this system embodiment, and will not be described in detail here.

[0050] In summary, this application proposes a map coordinate optimization method that incorporates GPS global constraints during the optimization process, resulting in more accurate optimization results. First, the global map is divided into multiple local maps, with the first pose of the starting frame in each local map serving as the starting point. Then, the first pose of the keyframes within each local map is obtained. Subsequently, the second poses of the starting frames and keyframes within each local map are obtained, and a transformation matrix is ​​calculated as the GPS global constraint for subsequent optimization. The first pose of the starting frames in each local map is optimized using the GPS global constraint to obtain the final pose of the starting frames within each local map. Based on the final poses of the starting frames in each local map, the final poses of the keyframes within each local map are obtained. Based on the final poses of the starting frames and keyframes within each local map, the map point coordinates of the starting frames and keyframes within each local map are obtained. Finally, the map point coordinates of all starting frames and keyframes are integrated to obtain the global map.

[0051] Compared with the prior art, this application has at least the following beneficial effects:

[0052] In the process of map coordinate optimization, this application adds GPS global constraints in addition to visual constraints and IMU constraints. Compared with the existing technology, the pose estimation of the starting frame and key frame is more accurate and robust, thus the map coordinates are more accurate.

[0053] The map coordinate optimization method proposed in this application can greatly reduce the impact of cumulative error on the map building process. Furthermore, the GPS global constraints only apply to the starting frame within each local map, thus preventing the GPS global constraints from being used as a strong constraint that affects the entire map coordinate optimization process. Attached Figure Description

[0054] Appendix Figure 1 This is a flowchart illustrating the map coordinate optimization method according to an embodiment of the present invention.

[0055] Appendix Figure 2 This is a flowchart illustrating the VIO optimization process in an embodiment of the present invention.

[0056] Appendix Figure 3 This is a flowchart illustrating the GPS optimization process according to an embodiment of the present invention.

[0057] Appendix Figure 4 This is a schematic diagram of a map coordinate optimization system according to an embodiment of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0059] Example 1: As Figure 1 As shown, this application proposes a map coordinate optimization method, which specifically includes the following steps:

[0060] Step 1: Use the first pose of the starting frame in each local map as the local reference point.

[0061] Step 2: Based on the first pose of the starting frame in each local map, obtain the first pose of the key frame in each local map.

[0062] Step 3: Obtain GPS information for the starting frame and key frame in each local map, and calculate the second pose of the starting frame and the second pose of the key frame in each local map based on the GPS information.

[0063] Step 4: Establish GPS global constraints based on the first pose of the starting frame, the first pose of the key frame, the second pose of the starting frame, and the second pose of the key frame in each local map.

[0064] Step 5: Optimize the first pose of the starting frame in each local map using GPS global constraints to obtain the final pose of the starting frame in each local map. Based on the final pose of the starting frame in each local map, obtain the final pose of the key frame in each local map.

[0065] Step 6: Based on the final pose of the starting frame and the final pose of the key frame in each local map, obtain the map point coordinates of the starting frame and key frame in each local map, and then obtain the global map.

[0066] In an embodiment of the present invention, optionally, the first pose is the VIO pose and the second pose is the Northeast Sky pose.

[0067] In this embodiment of the invention, VIO optionally stands for Visual-Inertial Odometry, which is used to estimate the camera's pose (position and orientation) from data from the camera and inertial sensors (such as accelerometers and gyroscopes). VIO pose typically refers to the camera's attitude and position relative to an initial reference frame.

[0068] In an embodiment of the present invention, optionally, the northeast celestial pose is the pose information obtained after acquiring the GPS information of the starting frame or key frame, wherein the northeast celestial pose coordinate system takes the north as the positive half-axis of the Y-axis, the east as the positive half-axis of the X-axis, and the direction perpendicular to the ground surface upward as the positive half-axis of the Z-axis.

[0069] Furthermore, step 2 specifically includes:

[0070] Step 21: Obtain the relative pose of keyframes within each local map to the starting frame within each local map, and perform initial optimization by combining visual constraints and IMU constraints to obtain the result after initial optimization.

[0071] Step 22: After filtering the results of the initial optimization through reprojection error, perform a second optimization to obtain the first pose of the keyframes in each local map.

[0072] In this embodiment of the invention, optionally, step 2 is a VIO optimization process, specifically:

[0073] VIO visual constraints: The form of visual constraints is the reprojection error form, that is:

[0074] e visual =obs-unproject(R cl P l +t cl (2)

[0075] In equation (2), obs is the pixel plane coordinate of the associated feature point in the frame, Rcl and tcl are the rotation matrix and translation vector of the frame in the local map coordinate system, Pl is the coordinate of the map point associated with the feature point in the local map coordinate system, and unproject(·) is the reprojection process.

[0076] The IMU constraint form is:

[0077] In equation (3), Tcl and Tcl_IMU are the local map coordinate system pose of the current keyframe and the local map pose given by the IMU, Trl and Trl_IMU are the local map coordinate system pose of the reference keyframe and the local map pose given by the IMU. For any current frame, the previous keyframe is taken as the reference frame for this step, and I is the identity matrix.

[0078] This optimization process employs a secondary optimization method to filter the reprojection error of the visual constraints. The reprojection error selected in this invention is 5.991. The detailed process can be found in the appendix. Figure 2 .

[0079] Furthermore, the GPS information specifically includes: the longitude, latitude, and altitude of the starting frame or key frame within each local map.

[0080] In the Northeast Celestial Pose Coordinate System, the x-axis points east, the y-axis points north, and the z-axis is perpendicular to the ground and pointing upwards. To obtain the second pose of the starting frame or the second pose of the key frame, the required information is the longitude, latitude, and altitude of the starting frame or the key frame.

[0081] Furthermore, the construction of the GPS global constraints is specifically as follows:

[0082] Step 31: Calculate the transformation matrix between the second pose and the first pose within the first local map, and use this as the global transformation matrix from the second pose to the first pose.

[0083] Step 32: After obtaining the transformation matrix, the starting frame and key frame in the subsequent local map are transformed from the second pose to the first pose through the transformation matrix, which serves as the subsequent GPS global constraint.

[0084] Furthermore, the calculation of the transformation matrix between the first pose and the second pose within the first local map is specifically as follows:

[0085] Calculate multiple residual matrices between the first pose and the second pose of the starting frame in the first local map, as well as between the first pose and the second pose of the preset number of keyframes. Based on these multiple residual matrices, calculate the transformation matrix.

[0086] In an embodiment of the present invention, optionally, assuming that the northeast-sky pose of a certain frame corresponding to the GPS signal is TGPS, the VIO pose of the frame is TVIO, and the transformation matrix from the GPS northeast-sky coordinate system to the VIO coordinate system is TGPS2VIO, then:

[0087] T VIO =T GPS2VIO T GPS (1)

[0088] When performing the initial local map backend optimization, the transformation matrix TGPS2VIO is not yet available, so the transformation cannot be performed. When the number of frames within the local map is greater than or equal to 5 for the first time, TGPS2VIO is calculated. The calculation method adopts a non-linear optimization method, that is, each frame is constructed based on the residual of equation (1), and the optimization variable is TGPS2VIO.

[0089] After the transformation matrix TGPS2VIO is given, the pose of each subsequent keyframe participating in the backend optimization will be transformed from the GPS north-south sky pose to the VIO pose through this matrix, denoted as TVIOByGPS, as the basis for subsequent global constraints.

[0090] Nonlinear optimization is a class of optimization problems that solve problems with nonlinear objective functions. The goal is to find the variable values ​​that minimize or maximize the objective function. This type of problem has wide applications in various fields, including machine learning, computer vision, and engineering optimization.

[0091] Here are some common nonlinear optimization methods:

[0092] Gradient Descent: Gradient descent is a fundamental optimization method that gradually decreases the objective function by updating parameters along the negative gradient direction. Stochastic gradient descent (SGD) and batch gradient descent are common variants used to handle large-scale datasets.

[0093] Newton's Method: Newton's method uses the second derivative of the objective function to update parameters, and it typically converges faster than gradient descent. However, computing and storing the Hessian matrix of the objective function can present computational and storage challenges.

[0094] Quasi-Newton methods: Quasi-Newton methods avoid directly calculating the Hessian matrix by estimating the inverse of the Hessian matrix of the objective function. Among them, BFGS (Broyden-Fletcher-Goldfarb-Shanno) and L-BFGS (Limited-memory BFGS) are commonly used quasi-Newton methods.

[0095] Conjugate Gradient: The conjugate gradient method is an iterative method for solving specific types of quadratic function optimization problems, and it has a relatively fast convergence speed.

[0096] The Levenberg-Marquardt algorithm is commonly used for nonlinear least squares problems, such as parameter estimation and curve fitting. It combines the advantages of gradient descent and Newton's method, and can effectively handle nonlinear optimization problems.

[0097] Particle Swarm Optimization (PSO): PSO is an optimization algorithm based on swarm intelligence that searches for the optimal solution by simulating the behavior of flocks of birds or schools of fish.

[0098] Genetic Algorithm (GA): A genetic algorithm is an optimization algorithm inspired by natural selection and genetic mechanisms. It searches for the optimal solution by simulating the biological evolution process.

[0099] Simulated Annealing: The simulated annealing algorithm simulates the annealing process of solid materials. During the search process, it accepts a certain probability of inferior solutions in order to avoid getting trapped in local optima.

[0100] Furthermore, step 5 specifically includes:

[0101] Step 51: Based on the first pose of the starting frame in each local map, the first pose of the preset frame key frame, and GPS global constraints, the correction matrix of each local map is obtained.

[0102] Step 52: Correct the first pose of the starting frame in each local map using the correction matrix to obtain the final pose of the starting frame in each local map.

[0103] Step 53: Based on the final pose of the starting frame in each local map and the first pose of the keyframe in each local map, obtain the final pose of the keyframe in each local map.

[0104] Furthermore, step 5 also includes:

[0105] When the number of keyframes in the local map is less than the preset value, the first pose of the starting frame in the local map is taken as the final pose of the starting frame in the local map.

[0106] When the number of keyframes in a local map is less than a preset value, the first pose of the keyframe in the local map is used as the final pose of the keyframe in the local map.

[0107] In this embodiment of the invention, optionally, the GPS global optimization constraint applies to the VIO pose Tlw of the starting frame within the current local map coordinate system. The VIO pose of the starting frame before GPS global correction is denoted as Tlw_ini. In any complete GPS global optimization process, five frame images starting from the starting frame are involved. Before optimization, the VIO pose of each frame is Tcw_init = TclTlw_init, and the coordinates of the GPS information in the VIO coordinate system for each frame are TVIOByGPS. Typically, there is a deviation between Tcw_init and TVIOByGPS, and this deviation is generally consistent across all frames within the same local map. Therefore:

[0108] T cw_init T correct =T VIOByGPS (4)

[0109] Where Tcorrect is the correction matrix. This method constructs a nonlinear optimization problem based on equation (4) to solve this matrix. The global optimization constraint flowchart of PS is attached. Figure 3 .

[0110] After solving for the correction matrix, the final pose of the starting frame is Tlw_final = TlwTcorrect, and the final pose of the remaining keyframes is Tcl_final = Tcl Tlw_final.

[0111] It should be noted that GPS global constraints only apply to the starting frame within a local map. This is to prevent GPS global constraints from acting as strong constraints and affecting the entire map coordinate optimization process.

[0112] In this embodiment of the invention, the step of obtaining the map point coordinates of the starting frame and keyframe within each local map based on the final pose of the starting frame and the final pose of the keyframe within each local map, and then obtaining the global map, specifically involves:

[0113] The VIO global coordinates of map points within the local map are:

[0114] Example 2:

[0115] This application proposes a system for map coordinate optimization, such as... Figure 4 As shown, it specifically includes:

[0116] Start Module: Used to take the first pose of the starting frame in each local map as a local reference point.

[0117] First pose acquisition module: used to obtain the first pose of key frames in each local map based on the first pose of the starting frame in each local map.

[0118] The second pose acquisition module acquires GPS information of the starting frame and key frame in each local map, and calculates the second pose of the starting frame and the second pose of the key frame in each local map based on the GPS information.

[0119] GPS global constraint establishment module: Establishes GPS global constraints based on the first pose of the starting frame, the first pose of the key frame, the second pose of the starting frame, and the second pose of the key frame in each local map.

[0120] Final pose calculation module: Optimizes the first pose of the starting frame in each local map using GPS global constraints to obtain the final pose of the starting frame in each local map. Based on the final pose of the starting frame in each local map, the final pose of the key frame in each local map is obtained.

[0121] Integration module: Optimizes the first pose of the starting frame in each local map using GPS global constraints to obtain the final pose of the starting frame in each local map, and obtains the final pose of the key frame in each local map based on the final pose of the starting frame in each local map.

[0122] Furthermore, the final pose calculation module also includes:

[0123] Correction matrix acquisition unit: Based on the first pose of the starting frame in each local map, the first pose of the preset number of key frames, and GPS global constraints, the correction matrix of each local map is obtained.

[0124] Starting frame correction unit: Corrects the first pose of the starting frame in each local map with the correction matrix to obtain the final pose of the starting frame in each local map.

[0125] Keyframe Correction Unit: Based on the final pose of the starting frame in each local map and the first pose of the keyframe in each local map, the final pose of the keyframe in each local map is obtained.

[0126] Example 3:

[0127] Based on the same inventive concept, this application also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a control processor, implement the map coordinate optimization method as described above.

[0128] It should be noted that since the computer-readable storage medium in this embodiment is based on the same inventive concept as the map coordinate optimization method in the above embodiments, the corresponding content in the method embodiments is also applicable to this system embodiment, and will not be described in detail here.

[0129] In summary, this application proposes a map coordinate optimization method that incorporates GPS global constraints during the optimization process, resulting in more accurate optimization results. First, the global map is divided into multiple local maps, with the first pose of the starting frame in each local map serving as the starting point. Then, the first pose of the keyframes within each local map is obtained. Subsequently, the second poses of the starting frames and keyframes within each local map are obtained, and a transformation matrix is ​​calculated as the GPS global constraint for subsequent optimization. The first pose of the starting frames in each local map is optimized using the GPS global constraint to obtain the final pose of the starting frames within each local map. Based on the final poses of the starting frames in each local map, the final poses of the keyframes within each local map are obtained. Based on the final poses of the starting frames and keyframes within each local map, the map point coordinates of the starting frames and keyframes within each local map are obtained. Finally, the map point coordinates of all starting frames and keyframes are integrated to obtain the global map.

[0130] In the process of map coordinate optimization, this application adds GPS global constraints in addition to visual constraints and IMU constraints. Compared with the existing technology, the pose estimation of the starting frame and key frame is more accurate and robust, thus the map coordinates are more accurate.

[0131] The map coordinate optimization method proposed in this application can greatly reduce the impact of cumulative error on the map building process. Furthermore, the GPS global constraints only apply to the starting frame within each local map, thus preventing the GPS global constraints from being used as a strong constraint that affects the entire map coordinate optimization process.

[0132] In the several embodiments provided in this application, it will be understood that each block in the flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0133] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0134] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application for those skilled in the art.

Claims

1. A map coordinate optimization method, characterized in that, Specifically, it includes: S1: Use the first pose of the starting frame in each local map as the local reference point; S2: Based on the first pose of the starting frame in each local map, obtain the relative pose of the key frame in each local map relative to the starting frame in each local map, and perform initial optimization by combining visual constraints and IMU constraints to obtain the result after initial optimization. After filtering the result after initial optimization through reprojection error, perform secondary optimization to obtain the first pose of the key frame in each local map. S3: Obtain GPS information of the starting frame and key frame in each local map, and calculate the second pose of the starting frame and the second pose of the key frame in each local map based on the GPS information. S4: Establish GPS global constraints based on the first pose of the starting frame, the first pose of the key frame, the second pose of the starting frame, and the second pose of the key frame in each local map. S5: Optimize the first pose of the starting frame in each local map using GPS global constraints to obtain the final pose of the starting frame in each local map. Based on the final pose of the starting frame in each local map, obtain the final pose of the key frame in each local map. S6: Based on the final pose of the starting frame and the final pose of the key frame in each local map, obtain the map point coordinates of the starting frame and key frame in each local map, and then obtain the global map.

2. The map coordinate optimization method according to claim 1, characterized in that, The GPS information specifically includes: The longitude, latitude, and height of the starting frame and keyframe within each local map.

3. The map coordinate optimization method according to claim 2, characterized in that, The construction of the GPS global constraints is specifically as follows: S31: Calculate the transformation matrix between the second pose and the first pose within the first local map, and use this as the global transformation matrix from the second pose to the first pose; S32: After obtaining the transformation matrix, the starting frame and key frame in the subsequent local map are transformed from the second pose to the first pose through the transformation matrix, which serves as the subsequent GPS global constraint.

4. The map coordinate optimization method according to claim 3, characterized in that, The calculation of the transformation matrix between the first pose and the second pose within the first local map is specifically as follows: Calculate multiple residual matrices between the first pose of the starting frame and the second pose of the starting frame within the first local map, as well as between the first pose of the preset number of key frames and the second pose of the preset number of key frames. Based on these multiple residual matrices, calculate the transformation matrix.

5. The map coordinate optimization method according to claim 3, characterized in that, Step S4 specifically includes: S41: Based on the first pose of the starting frame in each local map, the first pose of the preset frame key frame and GPS global constraints, the correction matrix of each local map is obtained. S42: Correct the first pose of the starting frame in each local map with the correction matrix to obtain the final pose of the starting frame in each local map. S43: Based on the final pose of the starting frame in each local map and the first pose of the key frame in each local map, obtain the final pose of the key frame in each local map.

6. The map coordinate optimization method according to claim 4, characterized in that, Step S5 further includes: When the number of keyframes in the local map is less than the preset value, the first pose of the starting frame in the local map is taken as the final pose of the starting frame in the local map. When the number of keyframes in a local map is less than a preset value, the first pose of the keyframe in the local map is used as the final pose of the keyframe in the local map.

7. A system for a map coordinate optimization method as described in any one of claims 1-6, characterized in that, The system includes: The starting module is used to take the first pose of the starting frame in each local map as a local reference point. First pose acquisition module: used to obtain the first pose of key frames in each local map based on the first pose of the starting frame in each local map; Second pose acquisition module: acquires GPS information of the starting frame and key frame in each local map, and calculates the second pose of the starting frame and the second pose of the key frame in each local map based on the GPS information; GPS constraint establishment module: Establishes global GPS constraints based on the first pose of the starting frame, the first pose of the key frame, the second pose of the starting frame, and the second pose of the key frame within each local map. Final pose calculation module: Optimize the first pose of the starting frame in each local map using GPS global constraints to obtain the final pose of the starting frame in each local map. Based on the final pose of the starting frame in each local map, obtain the final pose of the key frame in each local map. And the integration module: Utilizes GPS global constraints to optimize the first pose of the starting frame in each local map, obtaining the starting pose of each local map. The final pose of the initial frame is obtained by calculating the final pose of the keyframes in each local map based on the final pose of the initial frame in each local map.

8. The system according to claim 7, characterized in that, The final pose calculation module also includes: Correction matrix acquisition unit: Based on the first pose of the starting frame in each local map, the first pose of the preset number of key frames and GPS global constraints, the correction matrix of each local map is obtained; Starting frame correction unit: Corrects the first pose of the starting frame in each local map with the correction matrix to obtain the final pose of the starting frame in each local map; And the keyframe correction unit: based on the final pose of the starting frame in each local map and the first pose of the keyframe in each local map, the final pose of the keyframe in each local map is obtained.

9. A computer-readable storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed by the control processor, they implement the map coordinate optimization method as described in any one of claims 1-6.

Images