Visual Inertial Odometry, Pose Temporal Deviation Estimation Method and Localization Method

Abstract

Embodiments of the present disclosure disclose a visual inertial odometer, a method for estimating pose time-series deviation, and a positioning method. The method for estimating the pose time-series deviation of the visual inertial odometer includes: extracting visual features from an image and establishing a data association between the visual features and a high-precision map, where the image is captured by a camera integrated with the visual odometer; obtaining a prior pose of the visual inertial odometer; based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, establishing an estimation model for the pose time-series deviation of the visual inertial odometer, and estimating the pose of the camera and the pose time-series deviation of the visual inertial odometer, which can continuously estimate the pose deviation of the visual inertial odometer, and moreover, using the estimation model for the pose time-series deviation of the visual inertial odometer to jointly estimate the pose of the camera and the pose time-series deviation of the visual inertial odometer, thereby accurately estimating the pose of the visual inertial odometer.

Description

Technical Field

[0001] The present disclosure relates to the field of positioning technologies, and specifically relates to a visual inertial odometer, a method for estimating pose time series deviation, and a positioning method. Background Art

[0002] Currently, high-precision maps are usually collected using high-precision integrated inertial navigation and lidar devices. Due to the high cost, high-precision devices cannot meet the requirements of high-frequency and large-scale high-precision map collection, making it difficult to ensure high-frequency updates of the maps. Inexpensive visual inertial (also known as visual-inertial) devices can be deployed on a large scale, and through VIO (Visual Inertial Odometry), GPS, and three-dimensional reconstruction, the collection and high-frequency updates of high-precision map elements can be completed.

[0003] In the fields of robotics and computer vision, visual odometry is defined as the process of calculating the position and pose (pose) from images captured by a camera, and is widely used in mobile device fields such as robots, for example, Mars rovers. If a visual odometry system uses an inertial measurement unit at the same time, such a system is usually called a visual inertial odometer (VIO). To ensure the accuracy of a high-precision map collection system based on VIO, it is necessary to correct the time series deviation of the VIO pose when estimating the VIO pose. The time series deviation refers to the time series formed by the deviation between the measured value and the true value. However, it is difficult to obtain the time series deviation of the VIO pose. Summary of the Invention

[0004] To solve the problems in the related technologies, embodiments of the present disclosure provide a method for estimating the pose time series deviation of a visual inertial odometer, a positioning method for a visual inertial odometer, a device, an electronic device, a readable storage medium, a computer program product, and a visual inertial odometer.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for estimating the pose time series deviation of a visual inertial odometer, the method including:

[0006] Extract visual features from an image, and establish a data association between the visual features and a high-precision map, where the image is captured by a camera integrated with a visual odometer;

[0007] Obtain a prior pose of the visual inertial odometer;

[0008] Based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, establish an estimation model for the pose time series deviation of the visual inertial odometer, and estimate the pose of the camera and the pose time series deviation of the visual inertial odometer.

[0009] In combination with the first aspect, in the first implementation manner of the first aspect of the present disclosure, establishing an estimation model for the pose time-series deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer includes:

[0010] Based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, respectively determine the visual feature reprojection error, the prior pose error, and the pose deviation error of the visual inertial odometer as the constraint terms in the estimation model for the pose time-series deviation of the visual inertial odometer.

[0011] In combination with the first implementation manner of the first aspect, in the second implementation manner of the first aspect of the present disclosure, determining the visual feature reprojection error of the visual inertial odometer as the constraint term in the estimation model for the pose time-series deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer includes:

[0012] According to the established data association between the visual features and the high-precision map, to obtain the Figure 3 dimensional features that match the visual features;

[0013] Obtain the pixel coordinates of the visual features that match the Figure 3 dimensional features on the image;

[0014] Based on the time-series pose variables, the Figure 3 dimensional features, and the pixel coordinates of the visual features that match the Figure 3 dimensional features on the image, calculate the visual feature reprojection error of the visual inertial odometer as the constraint of the pose variables of the visual features on the camera in the estimation model for the pose time-series deviation of the visual inertial odometer.

[0015] In combination with the first implementation manner of the first aspect, in the third implementation manner of the first aspect of the present disclosure, determining the prior pose error of the visual inertial odometer as the constraint term in the estimation model for the pose time-series deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer includes:

[0016] Based on the time-series pose variables of the camera at a specified moment and the pose deviation of the visual inertial odometer, use a preset function to calculate the first predicted value at the specified moment;

[0017] Based on the prior pose of the visual inertial odometer at the specified moment obtained and the first prediction value at the specified moment, calculate the prior pose error as the constraint on the pose variable of the camera and the pose deviation of the visual inertial odometer in the estimation model of the pose temporal deviation of the visual inertial odometer.

[0018] Combined with an implementation manner of the first aspect, in the fourth implementation manner of the first aspect of the present disclosure, based on the established data association between the visual feature and the high-precision map and the prior pose of the visual inertial odometer, determining the pose deviation error of the visual inertial odometer as a constraint term in the estimation model of the pose temporal deviation of the visual inertial odometer includes:

[0019] Based on a preset autoregressive pose deviation estimation model, calculate the pose deviation of the visual inertial odometer at the specified moment by using the pose deviations at a preset number of moments of the visual inertial odometer before the specified moment;

[0020] Based on the calculated pose deviation of the visual inertial odometer at the specified moment and the temporal deviations at a preset number of moments before the specified moment, calculate the pose deviation error of the visual inertial odometer as the constraint on the calculation process of the autoregressive pose deviation estimation model in the estimation model of the pose temporal deviation of the visual inertial odometer.

[0021] Combined with any one of the first aspect, the first implementation manner to the fourth implementation manner of the first aspect, in the fifth implementation manner of the first aspect of the present disclosure, the estimating the pose of the camera and the pose temporal deviation of the visual inertial odometer includes:

[0022] Based on the visual feature reprojection error, prior pose error, and pose deviation error of the visual inertial odometer, calculate the total error of the visual inertial odometer;

[0023] Using a preset nonlinear optimization algorithm, estimate the pose of the camera and the pose temporal deviation of the visual inertial odometer by minimizing the total error of the pose of the visual inertial odometer.

[0024] Combined with the first aspect, in the sixth implementation manner of the first aspect of the present disclosure, the establishing an estimation model of the pose temporal deviation of the visual inertial odometer based on the established data association between the visual feature and the high-precision map and the prior pose of the visual inertial odometer includes:

[0025] Based on the established data association between the visual feature and the high-precision map, calculate the visual feature reprojection error of the visual inertial odometer as the constraint of the visual feature on the pose variable of the camera in the estimation model of the pose temporal deviation of the visual inertial odometer;

[0026] Calculate the prior pose error of the visual inertial odometer based on the temporal pose variables of the camera at a specified moment, as a constraint on the pose variables of the camera and the pose deviation of the visual inertial odometer in the estimation model of the temporal pose deviation of the visual inertial odometer.

[0027] Based on a preset autoregressive pose deviation estimation model, calculate the pose deviation of the visual inertial odometer at a specified moment using the deviation of the prior pose of the visual inertial odometer, and calculate the pose deviation error of the visual inertial odometer as a constraint on the calculation process of the autoregressive pose deviation estimation model in the estimation model of the temporal pose deviation of the visual inertial odometer.

[0028] In a second aspect, an embodiment of the present disclosure provides a visual inertial odometer, which includes a visual inertial odometer temporal pose deviation estimation device, and the visual inertial odometer temporal pose deviation estimation device includes:

[0029] An extraction and establishment module, configured to extract visual features from an image and establish a data association between the visual features and a high-precision map, where the image is collected by a camera integrated with a visual odometer.

[0030] An acquisition module, configured to acquire the prior pose of the visual inertial odometer.

[0031] An estimation module, configured to establish an estimation model of the temporal pose deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, and estimate the pose of the camera and the temporal pose deviation of the visual inertial odometer.

[0032] In a third aspect, an embodiment of the present disclosure provides a visual inertial odometer positioning method, and the method includes:

[0033] Extract visual features from an image and establish a data association between the visual features and a high-precision map, where the image is collected by a camera integrated with a visual odometer.

[0034] Acquire the prior pose of the visual inertial odometer.

[0035] Based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, establish an estimation model of the temporal pose deviation of the visual inertial odometer, and estimate the pose of the camera and the temporal pose deviation of the visual inertial odometer.

[0036] Perform positioning on the visual inertial odometer based on the estimated pose of the camera and the temporal pose deviation of the visual inertial odometer.

[0037] Fourthly, an embodiment of the present disclosure provides a computer program product, including computer instructions, which when executed by a processor, implement the methods described in the first aspect, the first to sixth implementation manners of the first aspect, and the third aspect.

[0038] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0039] According to the technical solutions provided by the embodiments of the present disclosure, by extracting visual features from an image and establishing a data association between the visual features and a high-precision map, where the image is captured by a camera integrated with a visual odometer; obtaining the prior pose of the visual inertial odometer; and based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, establishing an estimation model for the temporal sequence deviation of the visual inertial odometer pose, and estimating the pose of the camera and the temporal sequence deviation of the visual inertial odometer pose, the pose deviation of the visual inertial odometer can be continuously estimated, and by using the estimation model for the temporal sequence deviation of the visual inertial odometer pose, the pose of the camera and the temporal sequence deviation of the visual inertial odometer pose are estimated together, thereby accurately estimating the pose of the visual inertial odometer.

[0040] According to the technical solutions provided by the embodiments of the present disclosure, by establishing an estimation model for the temporal sequence deviation of the visual inertial odometer pose based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, it includes: based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, respectively determining the visual feature reprojection error, the prior pose error, and the pose deviation error of the visual inertial odometer as constraint terms in the estimation model for the temporal sequence deviation of the visual inertial odometer pose, so as to perform holistic error optimization of the temporal sequence deviation of the visual inertial odometer pose, and by using the estimation model for the temporal sequence deviation of the visual inertial odometer pose, the pose of the camera and the temporal sequence deviation of the visual inertial odometer pose are estimated together, thereby accurately estimating the pose of the visual inertial odometer.

[0041] According to the technical solutions provided by the embodiments of the present disclosure, by determining the visual feature reprojection error of the visual inertial odometer as a constraint term in the estimation model for the temporal sequence deviation of the visual inertial odometer pose based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, it includes: according to the established data association between the visual features and the high-precision map, to obtain the Figure 3 dimensional features; obtaining the pixel coordinates of the visual features matching the Figure 3 dimensional features on the image; based on the temporal pose variables, the Figure 3 dimensional features and the Figure 3The pixel coordinates of the visual features of the dimension feature matching on the image are used to calculate the visual feature reprojection error of the visual inertial odometer as the constraint of the pose variable of the camera by the visual feature in the estimation model of the pose time series deviation of the visual inertial odometer. This can be used for the overall error optimization of the pose time series deviation of the visual inertial odometer. Moreover, the pose of the camera and the pose time series deviation of the visual inertial odometer are estimated together using the estimation model of the pose time series deviation of the visual inertial odometer, and then the pose of the visual inertial odometer is accurately estimated.

[0042] According to the technical solution provided by the embodiment of the present disclosure, by based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, determining the prior pose error of the visual inertial odometer as a constraint term in the estimation model of the pose time series deviation of the visual inertial odometer, includes: calculating a first predicted value at a specified time based on the time series pose variable of the camera at the specified time and the pose deviation of the visual inertial odometer; calculating the prior pose error as the constraint on the pose variable of the camera and the pose deviation of the visual inertial odometer in the estimation model of the pose time series deviation of the visual inertial odometer based on the obtained prior pose of the visual inertial odometer at the specified time and the first predicted value at the specified time. This can be used for the overall error optimization of the pose time series deviation of the visual inertial odometer. Moreover, the pose of the camera and the pose time series deviation of the visual inertial odometer are estimated together using the estimation model of the pose time series deviation of the visual inertial odometer, and then the pose of the visual inertial odometer is accurately estimated.

[0043] According to the technical solution provided by the embodiment of the present disclosure, by based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, determining the pose deviation error of the visual inertial odometer as a constraint term in the estimation model of the pose time series deviation of the visual inertial odometer, includes: calculating the pose deviation of the visual inertial odometer at a specified time based on the preset autoregressive pose deviation estimation model and the pose deviations at a preset number of times before the specified time of the visual inertial odometer; calculating the pose deviation error of the visual inertial odometer as the constraint on the calculation process of the autoregressive pose deviation estimation model in the estimation model of the pose time series deviation of the visual inertial odometer based on the calculated pose deviation of the visual inertial odometer at the specified time and the time series deviations at a preset number of times before the specified time. This can be used for the overall error optimization of the pose time series deviation of the visual inertial odometer, and then the autoregressive process can be used to model and estimate the pose time series deviation of the visual inertial odometer. Moreover, the pose of the camera and the pose time series deviation of the visual inertial odometer are estimated together using the estimation model of the pose time series deviation of the visual inertial odometer, and then the pose of the visual inertial odometer is accurately estimated.

[0044] According to the technical solution provided by the embodiments of the present disclosure, estimating the pose of the camera and the temporal deviation of the visual inertial odometry pose includes: calculating the total error of the visual inertial odometry based on the visual feature reprojection error, the prior pose error, and the pose deviation error of the visual inertial odometry; using a preset non - linear optimization algorithm to estimate the pose of the camera and the temporal deviation of the visual inertial odometry pose by minimizing the total error of the visual inertial odometry pose. This can be used for the overall error optimization of the temporal deviation of the visual inertial odometry pose. Moreover, by using the estimation model of the temporal deviation of the visual inertial odometry pose, the pose of the camera and the temporal deviation of the visual inertial odometry pose are estimated together, and then the pose of the visual inertial odometry is accurately estimated.

[0045] According to the technical solution provided by the embodiments of the present disclosure, establishing an estimation model of the temporal deviation of the visual inertial odometry pose based on the established data association between the visual features and the high - definition map and the prior pose of the visual inertial odometry includes: calculating the visual feature reprojection error of the visual inertial odometry as the constraint of the visual feature on the pose variable of the camera in the estimation model of the temporal deviation of the visual inertial odometry pose based on the established data association between the visual features and the high - definition map; calculating the prior pose error of the visual inertial odometry as the constraint on the pose variable of the camera and the pose deviation of the visual inertial odometry in the estimation model of the temporal deviation of the visual inertial odometry pose based on the temporal pose variable of the camera at a specified moment; based on a preset autoregressive pose deviation estimation model, calculating the pose deviation of the visual inertial odometry at a specified moment using the deviation of the prior pose of the visual inertial odometry, and calculating the pose deviation error of the visual inertial odometry as the constraint on the calculation process of the autoregressive pose deviation estimation model in the estimation model of the temporal deviation of the visual inertial odometry pose. This can be used for the overall error optimization of the temporal deviation of the visual inertial odometry pose, and then the autoregressive process can be used to model and estimate the temporal deviation of the visual inertial odometry pose. Moreover, by using the estimation model of the temporal deviation of the visual inertial odometry pose, the pose of the camera and the temporal deviation of the visual inertial odometry pose are estimated together, and then the pose of the visual inertial odometry is accurately estimated.

[0046] According to the technical solution provided by the embodiments of the present disclosure, the visual inertial odometer includes a device for estimating the pose time-series deviation of the visual inertial odometer. The device for estimating the pose time-series deviation of the visual inertial odometer includes: an extraction and establishment module configured to extract visual features from an image and establish a data association between the visual features and a high-precision map, where the image is captured by a camera integrated with the visual odometer; an acquisition module configured to acquire the prior pose of the visual inertial odometer; and an estimation module configured to establish an estimation model for the pose time-series deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, and estimate the pose of the camera and the pose time-series deviation of the visual inertial odometer, so as to continuously estimate the pose deviation of the visual inertial odometer, and use the estimation model of the pose time-series deviation of the visual inertial odometer to jointly estimate the pose of the camera and the pose time-series deviation of the visual inertial odometer, and further accurately estimate the pose of the visual inertial odometer.

[0047] According to the technical solution provided by the embodiments of the present disclosure, by extracting visual features from an image and establishing a data association between the visual features and a high-precision map, where the image is captured by a camera integrated with the visual odometer; acquiring the prior pose of the visual inertial odometer; establishing an estimation model for the pose time-series deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, and estimating the pose of the camera and the pose time-series deviation of the visual inertial odometer; and positioning the visual inertial odometer based on the estimated pose of the camera and the pose time-series deviation of the visual inertial odometer, the pose deviation of the visual inertial odometer can be continuously estimated, and the estimation model of the pose time-series deviation of the visual inertial odometer is used to jointly estimate the pose of the camera and the pose time-series deviation of the visual inertial odometer, and further accurately estimate the pose of the visual inertial odometer.

[0048] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In combination with the accompanying drawings, through the following detailed description of non-limiting embodiments, other features, objects, and advantages of the present disclosure will become more apparent. In the drawings:

[0050] Figure 1 A flowchart showing a method for estimating the pose time-series deviation of a visual inertial odometer according to an embodiment of the present disclosure is shown;

[0051] Figure 2 An exemplary diagram showing the implementation process of a method for estimating the pose time-series deviation of a visual inertial odometer according to an embodiment of the present disclosure is shown;

[0052] Figure 3 An exemplary factor graph showing the algorithm of a visual inertial odometry pose time-series deviation estimation method according to an embodiment of the present disclosure;

[0053] Figure 4 Another exemplary factor graph showing the algorithm of a visual inertial odometry pose time-series deviation estimation method according to an embodiment of the present disclosure;

[0054] Figure 5 A structural block diagram showing a visual inertial odometry pose time-series deviation estimation device according to an embodiment of the present disclosure;

[0055] Figure 6 A structural block diagram showing a visual inertial odometry according to an embodiment of the present disclosure;

[0056] Figure 7 A flowchart showing a visual inertial odometry positioning method according to an embodiment of the present disclosure;

[0057] Figure 8 A structural block diagram showing a visual inertial odometry positioning device according to an embodiment of the present disclosure;

[0058] Figure 9 A structural block diagram showing an electronic device according to an embodiment of the present disclosure;

[0059] Figure 10 It is a schematic structural diagram of a computer system suitable for implementing the method according to an embodiment of the present disclosure. Detailed implementation manners

[0060] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for clarity, parts unrelated to the description of the exemplary embodiments are omitted in the drawings.

[0061] In the present disclosure, it should be understood that terms such as "including" or "having" are intended to indicate the presence of the labels, numbers, steps, actions, components, parts, or combinations thereof disclosed in this specification, and are not intended to exclude the possibility of the presence or addition of one or more other labels, numbers, steps, actions, components, parts, or combinations thereof.

[0062] It should be further noted that, without conflict, the embodiments and the labels in the embodiments in the present disclosure can be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0063] To estimate the VIO pose, the inventors of the present disclosure considered the following several solutions.

[0064] Solution 1: In the state space of pose estimation, only consider the pose variables {x1, x2, …, x n} of the camera in the Visual-Inertial Odometry (VIO), without taking the pose biases {bias1, bias2, …, bias n} of the Visual-Inertial Odometry as variables, and do not use the relative motion constraint between image frames. Disadvantages of this solution: If there is a bias in the prior pose of VIO, this solution can only localize the image frames with visual feature constraints by matching with the high-precision map, and the image frames without visual features cannot be localized.

[0065] Solution 2: In the state space of pose estimation, only consider the pose variables {x1, x2, …, x n} of the camera in the Visual-Inertial Odometry (VIO), without taking the pose biases {bias1, bias2, …, bias n} of the Visual-Inertial Odometry as variables, and use the relative motion constraint between image frames. Disadvantages of this solution: Compared with Solution 1, this solution can localize the image frames without visual features through the relative motion constraint between image frames. However, if there is a bias in the relative motion between image frames itself, the VIO pose calculated through the relative motion between image frames is also biased.

[0066] Solution 3: In the state space of pose estimation, consider the pose variables {x1, x2, …, x n} of the camera in the Visual-Inertial Odometry (VIO), and simultaneously consider the pose biases of the Visual-Inertial Odometry in segments. By assuming that the pose bias remains unchanged within a certain period of time or distance, calibrate the pose bias of the Visual-Inertial Odometry within this period of time or distance. Disadvantages of this solution: The pose bias of the Visual-Inertial Odometry changes continuously in the time series, and it is not easy to divide the segments where the bias remains unchanged in time or space.

[0067] Considering the disadvantages of the above solutions, the inventor of the present disclosure proposes a new solution: In the state space of pose estimation, simultaneously consider the pose variables {x1, x2, …, x n} of the camera, and the pose biases {bias1, bias2, …, bias n} of the Visual-Inertial Odometry as variables. Compared with Solution 1, it can solve the localization problem of image frames lacking visual feature (image matching with the high-precision prior map) constraints. Compared with Solution 2, it can avoid the problem of localization bias caused by the bias in the relative motion of image frames. Compared with Solution 3, it can continuously estimate the bias of the Visual-Inertial Odometry.

[0068] In the solution of the embodiment of the present disclosure, the process of correcting the pose time sequence deviation of the visual inertial odometer needs to add visual constraints formed by associating with high-precision prior map data, and model and estimate the pose time sequence error of the visual inertial odometer. Moreover, through the modeling and estimation of the pose time sequence error of the visual inertial odometer in the embodiment of the present disclosure, not only can the pose time sequence error of the visual inertial odometer be estimated, but also the camera pose variables of the visual inertial odometer can be estimated, and thus the pose estimation of the visual inertial odometer can be realized.

[0069] For example, when the high-precision map for the current map application in the high-speed scenario has been basically collected, a map update process is required. Usually, customers have a high requirement for the map update frequency, and a high-precision map update system that can be deployed on a large scale is needed. Through the solution of the embodiment of the present disclosure, when updating the high-precision map, a monocular camera, a consumer-grade inertial navigation system, and GPS can be used for high-frequency update of the high-precision map. For example, a map update device including VIO can be installed on various vehicles to collect data.

[0070] According to the technical solution provided by the embodiment of the present disclosure, by extracting visual features from an image and establishing a data association between the visual features and the high-precision map, where the image is collected by a camera integrated with a visual odometer; obtaining the prior pose of the visual inertial odometer; based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, establishing an estimation model for the pose time sequence deviation of the visual inertial odometer, and estimating the pose of the camera and the pose time sequence deviation of the visual inertial odometer, the pose deviation of the visual inertial odometer can be continuously estimated, and by using the estimation model of the pose time sequence deviation of the visual inertial odometer, the pose of the camera and the pose time sequence deviation of the visual inertial odometer can be estimated together, and thus the pose of the visual inertial odometer can be accurately estimated.

[0071] To solve the above problems, the present disclosure proposes a method for estimating the pose time sequence deviation of a visual inertial odometer, a method for positioning a visual inertial odometer, a device, an electronic device, a readable storage medium, a computer program product, and a visual inertial odometer.

[0072] Figure 1 The flowchart of a method for estimating the pose time sequence deviation of a visual inertial odometer according to an embodiment of the present disclosure is shown. As Figure 1 shown, the method for estimating the pose time sequence deviation of a visual inertial odometer includes steps S101, S102, and S103.

[0073] In step S101, visual features are extracted from the image, and a data association between the visual features and the high-precision map is established. The image is captured by a camera integrated with visual odometry. In step S102, the prior pose of the visual inertial odometer is obtained. In step S103, based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, an estimation model for the temporal deviation of the visual inertial odometer pose is established, and the pose of the camera and the temporal deviation of the visual inertial odometer pose are estimated.

[0074] In an embodiment of the present disclosure, the extraction of visual features can be completed by means known in the related art, and the present disclosure does not limit the specific visual feature extraction method. Those skilled in the art can understand that an important characteristic of visual feature extraction is "repeatability": the visual features extracted from different images of the same scene should be the same. In an embodiment of the present disclosure, extracting visual features from an image does not mean that visual features can be extracted from every image that can be obtained by the camera, but that visual features may only be extracted from some images due to reasons such as scene changes. Therefore, in an embodiment of the present disclosure, the problem of localizing image frames lacking visual feature (image matching with high-precision prior map) constraints can be solved.

[0075] In an embodiment of the present disclosure, the process of correcting the temporal error of the visual inertial odometer pose needs to add a visual constraint formed by the data association between the visual features and the high-precision (prior) map. The method of establishing the data association between the visual features and the high-precision map can be obtained from the related art, and the present disclosure does not limit this.

[0076] In an embodiment of the present disclosure, the prior pose of the visual inertial odometer refers to the pose and the deviation of the pose of the visual inertial odometer obtained as prior values in a preset manner when estimating the temporal deviation of the visual inertial odometer pose. The specific acquisition method can be obtained from the related art, and the present disclosure does not limit this.

[0077] In an embodiment of the present disclosure, step S103 includes: based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, respectively determining the visual feature reprojection error, the prior pose error, and the pose deviation error of the visual inertial odometer as constraint terms in the estimation model of the temporal deviation of the visual inertial odometer pose.

[0078] In an embodiment of the present disclosure, in the estimation model of the temporal deviation of the visual inertial odometer pose, the visual feature reprojection error, the prior pose error, and the pose deviation error are components of the total error that need to be optimized to complete the continuous estimation of the temporal error of the visual inertial odometer.

[0079] According to the technical solution provided by the embodiments of the present disclosure, an estimation model for the pose time-series deviation of a visual inertial odometer is established by means of the data association between the established visual features and the high-precision map and the prior pose of the visual inertial odometer, including: based on the data association between the established visual features and the high-precision map and the prior pose of the visual inertial odometer, respectively determining the visual feature reprojection error, the prior pose error, and the pose deviation error of the visual inertial odometer as constraint terms in the estimation model for the pose time-series deviation of the visual inertial odometer, so as to perform global error optimization of the pose time-series deviation of the visual inertial odometer, and moreover, using the estimation model for the pose time-series deviation of the visual inertial odometer to jointly estimate the pose of the camera and the pose time-series deviation of the visual inertial odometer, thereby accurately estimating the pose of the visual inertial odometer.

[0080] The following refers to Figure 2 Describe the implementation process of the method for estimating the pose time-series deviation of a visual inertial odometer according to an embodiment of the present disclosure. Figure 2 An exemplary schematic diagram showing the implementation process of the method for estimating the pose time-series deviation of a visual inertial odometer according to an embodiment of the present disclosure.

[0081] As Figure 2 shown, steps 201, 202, and 203 are the processes of constructing the visual feature reprojection error. In step 201, image visual features are extracted from the images (image frames) obtained by the camera. For example, 2D visual features are extracted from the images. In step 202, the data association between the visual features and the high-precision map is performed. For example, the matching relationship between the 2D visual features of the image and the Figure 3 D elements of the high-precision map is generated. The Figure 3 D elements of the high-precision map can also be referred to as 3D visual features, and may include points, lines, planes, edge features, etc. In step 203, the reprojection error is constructed.

[0082] In an embodiment of the present disclosure, based on the data association between the established visual features and the high-precision map and the prior pose of the visual inertial odometer, determining the visual feature reprojection error of the visual inertial odometer as a constraint term in the estimation model for the pose time-series deviation of the visual inertial odometer includes: according to the established data association between the visual features and the high-precision map, to obtain the Figure 3 dimensional features matching the visual features; obtaining the pixel coordinates of the visual features matching the Figure 3 dimensional features on the image; based on the time-series pose variables, the Figure 3 dimensional features, and the Figure 3The pixel coordinates of the visual feature matched by the dimensional feature on the image are calculated, and the visual feature reprojection error of the visual inertial navigation odometer is used as the constraint of the visual feature on the pose variable of the camera in the estimation model of the visual inertial navigation odometer pose timing deviation.

[0083] In one embodiment of the present disclosure, the following formula (1) can be used to construct the visual feature reprojection error error reprojection :

[0084] error reprojection =proj(x k ,landmarks k )-pixel map,k (1)

[0085] Among them, x k is the temporal pose variable at time k; landmarks k The visual features (two-dimensional visual features) at time k are associated with the high-precision map data, and the matched map Figure 3 Dimensional features (high precision Figure 3 D element); proj is a reprojection function that can be obtained from related technologies; pixel map,k is the image at time k and the ground Figure 3 Dimensional landmarks k The pixel coordinates of the matched visual features.

[0086] According to the technical solution provided by the embodiment of the present disclosure, based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial navigation odometer, the visual feature reprojection error of the visual inertial navigation odometer is determined as a constraint item in the estimation model of the temporal deviation of the visual inertial navigation odometer pose, including: according to the established data association between the visual features and the high-precision map, a map matching the visual features is obtained. Figure 3 dimensional features; obtain the Figure 3 The pixel coordinates of the visual features matched by the dimensional features on the image; based on the temporal pose variables, the geo Figure 3 dimensional features and Figure 3 The pixel coordinates of the visual features matched with the dimensional features on the image are calculated, and the visual feature reprojection error of the visual inertial odometry is calculated as the constraint of the visual features on the pose variables of the camera in the estimation model of the visual inertial odometry pose timing deviation, which can be used for the overall error optimization of the visual inertial odometry pose timing deviation, and the estimation model of the visual inertial odometry pose timing deviation is used to estimate the camera pose and the visual inertial odometry pose timing deviation together, so as to accurately estimate the pose of the visual inertial odometry.

[0087] likeFigure 2 As shown, steps 204 and 205 are the processes of constructing the prior pose error of the visual inertial odometer. In step 204, the prior pose of the visual inertial odometer VIO is obtained. In step 205, the prior pose error of the visual inertial odometer VIO is constructed.

[0088] In an embodiment of the present disclosure, based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, determining the prior pose error of the visual inertial odometer as a constraint term in the estimation model of the pose time series deviation of the visual inertial odometer includes: calculating a first predicted value at a specified time based on the time series pose variable of the camera at the specified time and the pose deviation of the visual inertial odometer; calculating the prior pose error as a constraint on the pose variable of the camera and the pose deviation of the visual inertial odometer in the estimation model of the pose time series deviation of the visual inertial odometer based on the obtained prior pose of the visual inertial odometer at the specified time and the first predicted value at the specified time.

[0089] In an embodiment of the present disclosure, the prior pose error error of the visual inertial odometer is constructed using the following formula (2) VIO :

[0090] error VIO = x vio,k - compose(x k , bias k ) (2)

[0091] Wherein, x vio,k is the prior pose of the visual inertial odometer at time k; x k is the pose variable of the camera at time k; bias k is the pose deviation of the visual inertial odometer at time k; compose is a composite operation known in the related art. For example, the composite operation of SE(3) group elements can obtain the predicted value of the SE(3) state, that is, the aforementioned first predicted value. In an embodiment of the present disclosure, the pose deviation bias k can be calculated through an autoregressive process using the pose deviations of a preset number of consecutive times before time k, and its specific calculation method can refer to the following calculation process for the pose deviation error.

[0092] According to the technical solution provided by the embodiments of the present disclosure, by determining the prior pose error of the visual inertial odometer as a constraint term in the estimation model of the pose time series deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, it includes: calculating a first predicted value at a specified moment based on the time series pose variable of the camera at the specified moment and the pose deviation of the visual inertial odometer by using a preset function; calculating the prior pose error as a constraint on the pose variable of the camera and the pose deviation of the visual inertial odometer in the estimation model of the pose time series deviation of the visual inertial odometer based on the obtained prior pose of the visual inertial odometer at the specified moment and the first predicted value at the specified moment. This can be used for the overall error optimization of the pose time series deviation of the visual inertial odometer, and the pose of the camera and the pose time series deviation of the visual inertial odometer are estimated together by using the estimation model of the pose time series deviation of the visual inertial odometer, so as to accurately estimate the pose of the visual inertial odometer.

[0093] As Figure 2 shown, steps 206 and 207 are the processes of constructing the pose deviation error of the visual inertial odometer. Specifically, the pose deviation error of the visual inertial odometer is constructed by using an autoregressive process. In step 206, the parameters of the autoregressive process are identified. In step 207, the pose deviation error of the visual inertial odometer is constructed.

[0094] In an embodiment of the present disclosure, by determining the pose deviation error of the visual inertial odometer as a constraint term in the estimation model of the pose time series deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, it includes: calculating the pose deviation of the visual inertial odometer at a specified moment based on the pose deviations of the visual inertial odometer at a preset number of moments before the specified moment by using a preset autoregressive pose deviation estimation model; calculating the pose deviation error of the visual inertial odometer as a constraint on the calculation process of the autoregressive pose deviation estimation model in the estimation model of the pose time series deviation of the visual inertial odometer based on the calculated pose deviation of the visual inertial odometer at the specified moment and the time series deviations at a preset number of moments before the specified moment.

[0095] In an embodiment of the present disclosure, based on the preset autoregressive pose deviation estimation model shown in the following formula (3), the pose deviation bias of the visual inertial odometer at a specified moment k can be estimated based on the pose deviations bias k-i of the visual inertial odometer at a preset number of moments before the specified moment. k :

[0096]

[0097] Among them, the autoregressive process in formula (3) is a discretized p-order Gaussian Markov process. Among them, a i is the correlation coefficient; b0·β k is the driving noise of the autoregressive process, b0 is the coefficient of the driving noise, and β k is the noise, which follows N(0,1) and is independently and identically distributed. The Markov property of order p makes the pose deviation bias k at time k only depend on the states from time k-1 to k-p in the past. Therefore, this Gaussian Markov process is also called an autoregressive process, that is, the state at time k can be predicted by a regression equation through the p states before time k. If bias k the state itself is a vector, the regression equation can also be written as a vectorized regression process, and its specific form can be obtained from related technologies, which will not be elaborated in this disclosure.

[0098] In an embodiment of the present disclosure, based on the pose deviation bias k at the specified time k of the calculated visual inertial odometer and the pose deviation bias k-i at a preset number of times before the specified time, the pose deviation error error AR of the visual inertial odometer can be calculated as an autoregressive pose deviation estimation model, that is, the constraint of the calculation process of formula (3):

[0099]

[0100] Among them, a i is the correlation coefficient.

[0101] As can be seen from the above description, the autoregressive process for estimating the pose deviation bias k at the specified time k using formula (3) needs to satisfy the constraint of the pose deviation error error AR of the visual inertial odometer. From a certain perspective, the driving noise b0·β k of the autoregressive process is an embodiment of the uncertainty of the pose deviation error error AR of the visual inertial odometer. In an embodiment of the present disclosure, this uncertainty can be not considered first to calculate the pose deviation error error AR , and then this uncertainty is used to measure this pose deviation error error AR .

[0102] According to the technical solution provided by the embodiments of the present disclosure, by determining the pose deviation error of the visual inertial odometer as a constraint term in the estimation model of the pose time series deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, it includes: based on a preset autoregressive pose deviation estimation model, calculating the pose deviation of the visual inertial odometer at a specified moment by using the pose deviations at a preset number of moments of the visual inertial odometer before the specified moment; based on the calculated pose deviation of the visual inertial odometer at the specified moment and the time series deviation at a preset number of moments before the specified moment, calculating the pose deviation error of the visual inertial odometer as a constraint in the calculation process of the autoregressive pose deviation estimation model in the estimation model of the pose time series deviation of the visual inertial odometer, which can be used for the overall error optimization of the pose time series deviation of the visual inertial odometer, and then the autoregressive process can be used to model and estimate the pose time series deviation of the visual inertial odometer. Moreover, the pose of the camera and the pose time series deviation of the visual inertial odometer are estimated together by using the estimation model of the pose time series deviation of the visual inertial odometer, and then the pose of the visual inertial odometer is accurately estimated.

[0103] In an embodiment of the present disclosure, the process of calculating the visual feature reprojection error, the process of calculating the prior pose error, and the process of calculating the pose deviation error can be executed in parallel (as Figure 2 shown) when running the estimation model of the pose time series deviation of the visual inertial odometer, or can be executed in a specific order according to the actual situation, and the present disclosure does not limit this.

[0104] In an embodiment of the present disclosure, the calculation process of the autoregressive pose deviation estimation model is a Gaussian Markov process.

[0105] According to the technical solution provided by the embodiments of the present disclosure, since the calculation process of the autoregressive pose deviation estimation model is a Gaussian Markov process, the Gaussian Markov process can be used to model and estimate the pose time series deviation of the visual inertial odometer. Moreover, the pose of the camera and the pose time series deviation of the visual inertial odometer are estimated together by using the estimation model of the pose time series deviation of the visual inertial odometer, and then the pose of the visual inertial odometer is accurately estimated.

[0106] As Figure 2 shown, an optimization equation can be constructed based on the calculated visual feature reprojection error, prior pose error, and pose deviation error of the visual inertial odometer, and this optimization equation optimizes the visual feature reprojection error, prior pose error, and pose deviation error of the visual inertial odometer as a whole.

[0107] In one embodiment of the present disclosure, step S103 includes: calculating the total error of the visual inertial odometer based on the visual feature reprojection error, prior pose error, and pose deviation error of the visual inertial odometer; using a preset non-linear optimization algorithm to estimate the pose of the camera and the pose time series deviation of the visual inertial odometer by minimizing the total error of the pose of the visual inertial odometer.

[0108] According to the technical solution provided by the embodiment of the present disclosure, the estimation of the pose of the camera and the pose time series deviation of the visual inertial odometer includes: calculating the total error of the visual inertial odometer based on the visual feature reprojection error, prior pose error, and pose deviation error of the visual inertial odometer; using a preset non-linear optimization algorithm to estimate the pose of the camera and the pose time series deviation of the visual inertial odometer by minimizing the total error of the pose of the visual inertial odometer, which can be used for the overall error optimization of the pose time series deviation of the visual inertial odometer, and estimating the pose of the camera and the pose time series deviation of the visual inertial odometer together using the estimation model of the pose time series deviation of the visual inertial odometer, thereby accurately estimating the pose of the visual inertial odometer.

[0109] In one embodiment of the present disclosure, using the following formula (5), based on the visual feature reprojection error error reprojection of the visual inertial odometer, prior pose error error VIO , and pose deviation error error AR , calculate the total error error of the visual inertial odometer:

[0110]

[0111] In one embodiment of the present disclosure, using the following formula (6), by performing calculations to minimize the total error error of the visual inertial odometer, an estimation model of the pose time series deviation of the visual inertial odometer is established, and the pose of the camera and the pose time series deviation {bias1, bias2,..., bias n} of the visual inertial odometer are estimated:

[0112] {x1, x2,..., x n , bias1, bias2,..., bias n} = argmin x,bias∈se(3) error (6)

[0113] Among them, argmin means to minimize (optimize) the total error error of the visual inertial odometer when x, bias ∈ se(3). In this case, the pose variables {x1, x2, …, x n} of the camera of the visual inertial odometer and the pose timing deviation {bias1, bias2, …, bias n} that minimize the total error error of the visual inertial odometer are the required pose variables {x1, x2, …, x n} of the camera and the pose timing deviation {bias1, bias2, …, bias n} of the visual inertial odometer. That is, the pose variables of the camera of the visual inertial odometer and the pose timing deviation of the visual inertial odometer that minimize (optimize) the total error error are calculated. Although the estimation model of the pose timing deviation of the visual inertial odometer mentioned in the embodiments of the present disclosure is for obtaining the pose timing deviation of the visual inertial odometer, the pose variables of the camera will also be solved during the solution.

[0114] In an embodiment of the present disclosure, Equation (6) can be considered to estimate two variables, namely, the pose variables of the camera and the pose timing deviation of the visual inertial odometer, when the constraints of the visual features on the pose variables of the camera, the constraints of the prior pose of the visual inertial odometer, and the calculation process of the autoregressive pose deviation estimation model are overall optimized. Among them, modeling the deviation of the pose of the visual inertial odometer and estimating (using the aforementioned autoregressive process) the pose timing deviation are the key points for the solution of the embodiments of the present disclosure to be able to estimate the pose timing deviation of the visual inertial odometer.

[0115] As Figure 2 shown, using Equation (6), step 209 can estimate the pose of the camera of the visual inertial odometer and the pose timing deviation of the visual inertial odometer.

[0116] In an embodiment of the present disclosure, Equation (6) constructs a non - linear optimization problem, and the solution methods include but are not limited to: optimization algorithms such as the Gauss - Newton method, the Dogleg optimization method, the conjugate gradient descent method, etc.

[0117] It should be understood that the estimation model of the pose time-series deviation of the visual inertial odometer mentioned in the embodiments of the present disclosure has not only the pose deviation of the visual inertial odometer as a variable, but also the pose variable of the camera as a variable. In an embodiment of the present disclosure, the estimation model of the pose time-series deviation of the visual inertial odometer can be considered as a system of equations constructed by formulas (1) to (6). Formula (1) constructs the visual feature reprojection error of the visual inertial odometer as the constraint of the pose variable of the visual feature on the camera in the estimation model of the pose time-series deviation of the visual inertial odometer, including the pose variable of the camera as a variable. Formula (2) calculates the prior pose error as the constraint on the pose variable of the camera and the pose deviation of the visual inertial odometer in the estimation model of the pose time-series deviation of the visual inertial odometer, including the pose variable of the camera and the pose deviation of the visual inertial odometer as variables. Formula (4) calculates the pose deviation error of the visual inertial odometer as the constraint on the calculation process of the autoregressive pose deviation estimation model shown in formula (3) in the estimation model of the pose time-series deviation of the visual inertial odometer, including the pose deviation of the visual inertial odometer as a variable. When estimating the two variables of the pose variable of the camera and the pose time-series deviation of the visual inertial odometer using formulas (5) and (6), the solution should be obtained by minimizing these errors as constraints as a whole. It should be understood that the foregoing specific formulas (1) to (6) are merely exemplary formulas for constructing the estimation model of the pose time-series deviation of the visual inertial odometer. According to the teachings of the embodiments of the present disclosure, those skilled in the art can adopt other formulas to calculate the foregoing errors related to the pose of the visual inertial odometer as constraint terms. In this case, formulas (1) to (6) can be replaced by other formulas. It should also be understood that according to the teachings of the embodiments of the present disclosure, those skilled in the art can adopt other formulas to calculate other errors related to the pose of the visual inertial odometer as constraint terms, as long as an estimation model of the pose time-series deviation of the visual inertial odometer with the two variables of the pose variable of the camera and the pose time-series deviation of the visual inertial odometer as variables is established.

[0118] In one embodiment of the present disclosure, step S103 includes: based on the established data association between the visual features and the high-precision map, calculating the visual feature reprojection error of the visual inertial odometer as the constraint of the visual features on the pose variables of the camera in the estimation model of the pose time series deviation of the visual inertial odometer; based on the time series pose variables of the camera at a specified moment, calculating the prior pose error of the visual inertial odometer as the constraint on the pose variables of the camera and the pose deviation of the visual inertial odometer in the estimation model of the pose time series deviation of the visual inertial odometer; based on a preset autoregressive pose deviation estimation model, using the deviation of the prior pose of the visual inertial odometer to calculate the pose deviation of the visual inertial odometer at the specified moment, and calculating the pose deviation error of the visual inertial odometer as the constraint on the calculation process of the autoregressive pose deviation estimation model in the estimation model of the pose time series deviation of the visual inertial odometer; estimating the pose time series deviation of the visual inertial odometer.

[0119] In one embodiment of the present disclosure, although it is mentioned that the need is to estimate the pose time series deviation of the visual inertial odometer, however, the important purpose is to achieve the pose estimation or positioning of the visual inertial odometer. Therefore, the established estimation model of the pose time series deviation of the visual inertial odometer can not only estimate the pose time series deviation of the visual inertial odometer, but also estimate the time series pose of the camera, and further estimate the pose estimation of the visual inertial odometer, that is, to position the visual inertial odometer.

[0120] The following refers to Figure 3 Describe the factor graph of the algorithm in one embodiment of the present disclosure. Figure 3 Fig. 300 shows an exemplary factor graph of the algorithm of the method for estimating the pose time series deviation of a visual inertial odometer according to one embodiment of the present disclosure.

[0121] As Figure 3 shown, module 301 is the visual feature reprojection error, which is used as the constraint of the visual features on the pose variables of the camera. Module 302 is the pose variable of the camera. Module 303 is the prior pose error of the visual inertial odometer, which is used as the constraint on the pose variables of the camera and the pose deviation of the visual inertial odometer in the estimation model of the pose time series deviation of the visual inertial odometer. Module 304 represents the time series error of the visual inertial odometer. Module 305 represents the pose deviation error of the visual inertial odometer, which is used as the autoregressive constraint on the calculation process of the autoregressive pose deviation estimation model.

[0122] According to the technical solution provided by the embodiment of the present disclosure, an estimation model of the pose time-series deviation of the visual inertial odometer is established by using the data association between the established visual features and the high-precision map and the prior pose of the visual inertial odometer, including: calculating the visual feature reprojection error of the visual inertial odometer as the constraint of the pose variable of the camera by the visual features in the estimation model of the pose time-series deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map; calculating the prior pose error of the visual inertial odometer as the constraint of the pose variable of the camera and the pose deviation of the visual inertial odometer in the estimation model of the pose time-series deviation of the visual inertial odometer based on the time-series pose variable of the camera at a specified moment; calculating the pose deviation of the visual inertial odometer at the specified moment by using the deviation of the prior pose of the visual inertial odometer based on a preset autoregressive pose deviation estimation model, and calculating the pose deviation error of the visual inertial odometer as the constraint of the calculation process of the autoregressive pose deviation estimation model in the estimation model of the pose time-series deviation of the visual inertial odometer; estimating the pose time-series deviation of the visual inertial odometer, which can be used for the overall error optimization of the pose time-series deviation of the visual inertial odometer, and then the autoregressive process can be used to model and estimate the pose time-series deviation of the visual inertial odometer. Moreover, the pose of the camera and the pose time-series deviation of the visual inertial odometer are estimated together by using the estimation model of the pose time-series deviation of the visual inertial odometer, and then the pose of the visual inertial odometer is accurately estimated.

[0123] In an embodiment of the present disclosure, step S103 includes: setting a time window for the pose time-series deviation of the visual inertial odometer, so that the pose variables of the camera within a specific time window correspond to the deviation of the same prior pose. According to the technical solution provided by the embodiment of the present disclosure, by setting a time window for the pose time-series deviation of the visual inertial odometer, so that the pose variables of the camera within a specific time window correspond to the deviation of the same prior pose, the calculation amount of estimating the pose of the camera and the pose time-series deviation of the visual inertial odometer by using the estimation model of the pose time-series deviation of the visual inertial odometer can be reduced.

[0124] The following refers to Figure 4 Describe the factor graph of the algorithm in another embodiment of the present disclosure. Figure 4 FIG. 400 shows another exemplary factor graph of the algorithm of the method for estimating the pose time-series deviation of the visual inertial odometer according to an embodiment of the present disclosure.

[0125] As Figure 4As shown, module 401 is the visual feature reprojection error, which serves as a constraint on the pose variables of the camera by the visual features. Module 402 is the pose variable of the camera. Module 403 is the prior pose error of the visual-inertial odometer, which serves as a constraint on the pose variable of the camera and the pose deviation of the visual-inertial odometer in the estimation model of the pose time series deviation of the visual-inertial odometer. Module 404 represents the time series error of the visual-inertial odometer. Module 405 represents the pose deviation error of the visual-inertial odometer, which serves as an autoregressive constraint in the calculation process of the autoregressive pose deviation estimation model. Figure 4 The algorithm shown in Figure 3 is an improvement of the algorithm shown in Figure 4 The algorithm in

[0126] The solution of the embodiment of the present disclosure can handle but is not limited to handling the time series deviation of the pose of the visual-inertial odometer, and can also be used to handle the time series deviation of other pose estimation algorithms. The modeling of the pose time series error in the solution of the embodiment of the present disclosure can be extended from an autoregressive process to an autoregressive moving average process, or a deep neural network model based on a Recurrent neural network and other mathematical methods for modeling time series processes. The pose estimation algorithm of the solution of the embodiment of the present disclosure can be used but is not limited to the estimation of the camera pose, and can also be used for pose estimation problems related to other sensors such as Lidar and radar.

[0127] The solution of the embodiment of the present disclosure adopts a time series model for the pose time series deviation of the visual-inertial odometer, combined with a high-precision prior map and visual features, to achieve high-precision continuous estimation of the camera pose.

[0128] The following refers to Figure 5 Describe a visual-inertial odometer pose time series deviation estimation device according to an embodiment of the present disclosure. Figure 5 The structural block diagram of a visual-inertial odometer pose time series deviation estimation device 500 according to an embodiment of the present disclosure is shown.

[0129] As Figure 5As shown in the figure, the visual inertial odometry pose time series deviation estimation device 500 includes: an extraction and establishment module 501, an acquisition module 502, and an estimation module 503. The extraction and establishment module 501 is configured to extract visual features from an image and establish a data association between the visual features and a high-precision map, where the image is captured by a camera integrated with visual odometry. The acquisition module 502 is configured to acquire the prior pose of the visual inertial odometry. The estimation module 503 is configured to establish an estimation model for the visual inertial odometry pose time series deviation based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometry, and estimate the pose of the camera and the visual inertial odometry pose time series deviation. According to the technical solution provided by the embodiments of the present disclosure, through the extraction and establishment module, which is configured to extract visual features from an image and establish a data association between the visual features and a high-precision map, where the image is captured by a camera integrated with visual odometry; the acquisition module, which is configured to acquire the prior pose of the visual inertial odometry; and the estimation module, which is configured to establish an estimation model for the visual inertial odometry pose time series deviation based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometry, and estimate the pose of the camera and the visual inertial odometry pose time series deviation, the pose deviation of the visual inertial odometry can be continuously estimated, and the pose of the camera and the visual inertial odometry pose time series deviation are estimated together using the estimation model of the visual inertial odometry pose time series deviation, thereby accurately estimating the pose of the visual inertial odometry.

[0130] Those skilled in the art can understand that the technical solutions described with reference to Figure 5 can be combined with the embodiments described with reference to Figures 1 to 4 so as to have the technical effects achieved by the embodiments described with reference to Figures 1 to 4 The specific content can be referred to the description made above according to Figures 1 to 4 and will not be elaborated herein.

[0131] The following refers to Figure 6 to describe a visual inertial odometry according to an embodiment of the present disclosure. Figure 6 FIG. shows a structural block diagram of a visual inertial odometry 600 according to an embodiment of the present disclosure.

[0132] As Figure 6 shown, the visual inertial odometry 600 includes a camera 601 for acquiring an image and a visual inertial odometry pose time series deviation estimation device 500, and the visual inertial odometry pose time series deviation estimation device 500 is the same as the visual inertial odometry pose time series deviation estimation device 500 shown in Figure 5 .

[0133] According to the technical solution provided by the embodiments of the present disclosure, the visual inertial odometer includes a visual inertial odometer pose timing deviation estimation device, and the visual inertial odometer pose timing deviation estimation device includes: an extraction and establishment module configured to extract visual features from an image and establish a data association between the visual features and a high-precision map, where the image is captured by a camera integrated with a visual odometer; an acquisition module configured to acquire a prior pose of the visual inertial odometer; an estimation module configured to establish an estimation model for the visual inertial odometer pose timing deviation based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, and estimate the pose of the camera and the visual inertial odometer pose timing deviation, which can continuously estimate the pose deviation of the visual inertial odometer, and use the estimation model of the visual inertial odometer pose timing deviation to jointly estimate the pose of the camera and the visual inertial odometer pose timing deviation, and then accurately estimate the pose of the visual inertial odometer.

[0134] Those skilled in the art can understand that the technical solutions described with reference to Figure 6 can be combined with the embodiments described with reference to Figures 1 to 5 so as to have the technical effects achieved by the embodiments described with reference to Figures 1 to 5 The specific content can be referred to the above description according to Figures 1 to 5 which will not be elaborated herein.

[0135] The following refers to Figure 7 to describe a visual inertial odometer positioning method according to an embodiment of the present disclosure. Figure 7 FIG. shows a flowchart of a visual inertial odometer positioning method according to an embodiment of the present disclosure. In an embodiment of the present disclosure, the visual inertial odometer includes a camera for acquiring images. As Figure 7 shown, the visual inertial odometer positioning method includes steps S701, S702, S703, and S704.

[0136] In step S701, visual features are extracted from the image, and a data association between the visual features and a high-precision map is established, where the image is captured by a camera integrated with a visual odometer. In step S702, a prior pose of the visual inertial odometer is acquired. In step S703, an estimation model for the visual inertial odometer pose timing deviation is established based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, and the pose of the camera and the visual inertial odometer pose timing deviation are estimated. In step S704, the visual inertial odometer is positioned based on the estimated pose of the camera and the visual inertial odometer pose timing deviation.

[0137] When estimating the pose of the camera and the time-series deviation of the pose of the visual inertial odometer according to an embodiment of the present disclosure, the visual inertial odometer is positioned based on the estimated pose of the camera and the time-series deviation of the pose of the visual inertial odometer.

[0138] According to the technical solution provided by the embodiment of the present disclosure, by extracting visual features from an image and establishing a data association between the visual features and a high-precision map, the image is collected by a camera integrated with a visual odometer; obtaining a prior pose of the visual inertial odometer; based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, establishing an estimation model for the time-series deviation of the pose of the visual inertial odometer, and estimating the pose of the camera and the time-series deviation of the pose of the visual inertial odometer; positioning the visual inertial odometer based on the estimated pose of the camera and the time-series deviation of the pose of the visual inertial odometer, the pose deviation of the visual inertial odometer can be continuously estimated, and the pose of the camera and the time-series deviation of the pose of the visual inertial odometer are estimated together by using the estimation model of the time-series deviation of the pose of the visual inertial odometer, thereby accurately estimating the pose of the visual inertial odometer.

[0139] When solving the problem of visual inertial odometer positioning, the solution of the embodiment of the present disclosure can isolate the deviation of the pose of the pose estimation system (visual inertial odometer) as a variable and establish an autoregressive model in time series, so as to complete the positioning of the visual inertial odometer even when visual features are missing. This positioning solution can be used in the visual inertial odometer positioning algorithm for high-precision map update.

[0140] Those skilled in the art can understand that the technical solution described with reference to Figure 7 can be combined with the embodiment described with reference to Figures 1 to 6 so as to have the technical effects achieved by the embodiment described with reference to Figures 1 to 6 The specific content can refer to the description according to Figures 1 to 6 above, and the specific content will not be elaborated here.

[0141] The following refers to Figure 8 to describe a visual inertial odometer positioning device according to an embodiment of the present disclosure. Figure 8 The structural block diagram of a visual inertial odometer positioning device 800 according to an embodiment of the present disclosure is shown.

[0142] As Figure 8As shown, the visual inertial odometry positioning device 800 includes: an extraction and establishment module 801, configured to extract visual features from an image and establish a data association between the visual features and a high-precision map, where the image is captured by a camera integrated with visual odometry; an acquisition module 802, configured to acquire a prior pose of the visual inertial odometry; an estimation module 803, configured to establish an estimation model of the pose time-series deviation of the visual inertial odometry based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometry, and estimate the pose of the camera and the pose time-series deviation of the visual inertial odometry; and a positioning module 804, configured to perform positioning on the visual inertial odometry based on the estimated pose of the camera and the pose time-series deviation of the visual inertial odometry.

[0143] According to the technical solution provided by the embodiments of the present disclosure, through the extraction and establishment module, which is configured to extract visual features from an image and establish a data association between the visual features and a high-precision map, where the image is captured by a camera integrated with visual odometry; the acquisition module, which is configured to acquire a prior pose of the visual inertial odometry; the estimation module, which is configured to establish an estimation model of the pose time-series deviation of the visual inertial odometry based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometry, and estimate the pose of the camera and the pose time-series deviation of the visual inertial odometry; and the positioning module, which is configured to perform positioning on the visual inertial odometry based on the estimated pose of the camera and the pose time-series deviation of the visual inertial odometry, the pose deviation of the visual inertial odometry can be continuously estimated, and the pose of the camera and the pose time-series deviation of the visual inertial odometry are estimated together using the estimation model of the pose time-series deviation of the visual inertial odometry, thereby accurately estimating the pose of the visual inertial odometry.

[0144] Figure 9 The structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0145] The embodiments of the present disclosure also provide an electronic device, such as Figure 9 shown, including at least one processor 901; and a memory 902 communicatively connected to the at least one processor 901; wherein the memory 902 stores instructions executable by the at least one processor 901, and the instructions are executed by the at least one processor 901 to implement the following steps:

[0146] Extract visual features from the image and establish a data association between the visual features and the high-precision map, where the image is captured by a camera integrated with visual odometry; obtain the prior pose of the visual inertial odometry; based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometry, establish an estimation model for the temporal deviation of the visual inertial odometry pose, and estimate the pose of the camera and the temporal deviation of the visual inertial odometry pose.

[0147] In an embodiment of the present disclosure, establishing an estimation model for the temporal deviation of the visual inertial odometry pose based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometry includes: based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometry, respectively determine the visual feature reprojection error, prior pose error, and pose deviation error of the visual inertial odometry as constraint terms in the estimation model for the temporal deviation of the visual inertial odometry pose.

[0148] In an embodiment of the present disclosure, determining the visual feature reprojection error of the visual inertial odometry as a constraint term in the estimation model for the temporal deviation of the visual inertial odometry pose based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometry includes: according to the established data association between the visual features and the high-precision map, to obtain the Figure 3 geometric features that match the visual features; obtain the pixel coordinates of the visual features that match the Figure 3 geometric features on the image; based on the temporal pose variables, the Figure 3 geometric features, and the pixel coordinates of the visual features that match the Figure 3 geometric features on the image, calculate the visual feature reprojection error of the visual inertial odometry as the constraint of the visual features on the pose variables of the camera in the estimation model for the temporal deviation of the visual inertial odometry pose.

[0149] In an embodiment of the present disclosure, determining the prior pose error of the visual inertial odometry as a constraint term in the estimation model for the temporal deviation of the visual inertial odometry pose based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometry includes:

[0150] Based on the temporal pose variables of the camera and the pose deviation of the visual inertial odometer at a specified moment, calculate the first prediction value at the specified moment using a preset function; based on the prior pose of the visual inertial odometer at the obtained specified moment and the first prediction value at the specified moment, calculate the prior pose error as a constraint on the pose variables of the camera and the pose deviation of the visual inertial odometer in the estimation model of the pose temporal deviation of the visual inertial odometer.

[0151] In an embodiment of the present disclosure, determining the pose deviation error of the visual inertial odometer as a constraint term in the estimation model of the pose temporal deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer includes: based on a preset autoregressive pose deviation estimation model, calculating the pose deviation of the visual inertial odometer at a specified moment using the pose deviations of a preset number of moments of the visual inertial odometer before the specified moment; based on the calculated pose deviation of the visual inertial odometer at the specified moment and the temporal deviations of a preset number of moments before the specified moment, calculating the pose deviation error of the visual inertial odometer as a constraint on the calculation process of the autoregressive pose deviation estimation model in the estimation model of the pose temporal deviation of the visual inertial odometer.

[0152] In an embodiment of the present disclosure, estimating the pose of the camera and the pose temporal deviation of the visual inertial odometer includes: calculating the total error of the visual inertial odometer based on the visual feature reprojection error, prior pose error, and pose deviation error of the visual inertial odometer; using a preset nonlinear optimization algorithm to estimate the pose of the camera and the pose temporal deviation of the visual inertial odometer by minimizing the total error of the pose of the visual inertial odometer.

[0153] In an embodiment of the present disclosure, establishing an estimation model for the pose time - series deviation of the visual - inertial odometer based on the established data association between the visual features and the high - definition map and the prior pose of the visual - inertial odometer includes: calculating the visual - feature reprojection error of the visual - inertial odometer as the constraint of the pose variable of the camera by the visual features in the estimation model for the pose time - series deviation of the visual - inertial odometer based on the established data association between the visual features and the high - definition map; calculating the prior pose error of the visual - inertial odometer as the constraint on the pose variable of the camera and the pose deviation of the visual - inertial odometer in the estimation model for the pose time - series deviation of the visual - inertial odometer based on the time - series pose variable of the camera at a specified moment; based on a preset autoregressive pose - deviation estimation model, calculating the pose deviation of the visual - inertial odometer at a specified moment using the deviation of the prior pose of the visual - inertial odometer, and calculating the pose - deviation error of the visual - inertial odometer as the constraint on the calculation process of the autoregressive pose - deviation estimation model in the estimation model for the pose time - series deviation of the visual - inertial odometer.

[0154] The embodiments of the present disclosure also provide an electronic device, as Figure 9 shown, including at least one processor 901; and a memory 902 communicatively connected to at least one processor 901; wherein, the memory 902 stores instructions executable by at least one processor 901, and the instructions are executed by at least one processor 901 to implement the following steps: extracting visual features from an image and establishing a data association between the visual features and a high - definition map, where the image is captured by a camera integrated with a visual odometer; obtaining the prior pose of the visual - inertial odometer; establishing an estimation model for the pose time - series deviation of the visual - inertial odometer based on the established data association between the visual features and the high - definition map and the prior pose of the visual - inertial odometer, and estimating the pose of the camera and the pose time - series deviation of the visual - inertial odometer; and positioning the visual - inertial odometer based on the estimated pose of the camera and the pose time - series deviation of the visual - inertial odometer.

[0155] As Figure 10 shown, the computer system 1000 includes a processing unit 1001, which can execute various processes in the embodiments shown in the above - mentioned figures according to a program stored in a read - only memory (ROM) 1002 or a program loaded from a storage section 1008 into a random - access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the system 1000 are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0156] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as required. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is installed on the drive 1010 as required so that a computer program read from it can be installed into the storage section 1008 as required. Among them, the processing unit 1001 can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0157] In particular, according to an embodiment of the present disclosure, the method described above with reference to the accompanying drawings can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a computer-readable medium, and the computer program includes program code for executing the method in the accompanying drawings. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 1009, and / or installed from the removable medium 1011. For example, an embodiment of the present disclosure includes a readable storage medium on which computer instructions are stored, and when the computer instructions are executed by a processor, program code for executing the method in the accompanying drawings is implemented.

[0158] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0159] The units or modules involved in the embodiments of the present disclosure can be implemented in software or in hardware. The described units or modules can also be provided in a processor, and the names of these units or modules do not, in some cases, constitute a limitation on the units or modules themselves.

[0160] As another aspect, the present disclosure also provides a computer-readable storage medium, which can be the computer-readable storage medium included in the nodes described in the above embodiments; or can exist alone and be a computer-readable storage medium not assembled into a device. The computer-readable storage medium stores one or more programs, and the one or more programs are used by one or more processors to execute the methods described in the present disclosure.

[0161] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

Claims

1. A method for estimating the pose time-series deviation of visual inertial odometry, wherein, The method includes: extracting visual features from an image and establishing a data association between the visual features and a high-precision map, where the image is captured by a camera integrated with visual odometry; obtaining a prior pose of the visual inertial odometer; establishing an estimation model for the temporal pose deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, and estimating the pose of the camera and the temporal pose deviation of the visual inertial odometer; wherein, the estimating the pose of the camera and the temporal pose deviation of the visual inertial odometer includes: calculating a total error of the visual inertial odometer based on a visual feature reprojection error, a prior pose error, and a pose deviation error of the visual inertial odometer; using a preset non-linear optimization algorithm to estimate the pose of the camera and the temporal pose deviation of the visual inertial odometer by minimizing the total error of the pose of the visual inertial odometer.

2. The method according to claim 1, wherein the establishing an estimation model for the temporal pose deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer includes: respectively determining a visual feature reprojection error, a prior pose error, and a pose deviation error of the visual inertial odometer as constraint terms in the estimation model for the temporal pose deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer.

3. The method according to claim 2, wherein the determining a visual feature reprojection error of the visual inertial odometer as a constraint term in the estimation model for the temporal pose deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer includes: obtaining a three-dimensional map feature matching the visual feature according to the established data association between the visual features and the high-precision map; obtaining pixel coordinates of the visual feature matching the three-dimensional map feature in the image; calculating a visual feature reprojection error of the visual inertial odometer as a constraint of the pose variable of the camera by the visual feature based on the temporal pose variable, the three-dimensional map feature, and the pixel coordinates of the visual feature matching the three-dimensional map feature in the image in the estimation model for the temporal pose deviation of the visual inertial odometer.

4. The method according to claim 2, wherein the determining a prior pose error of the visual inertial odometer as a constraint term in the estimation model for the temporal pose deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer includes: calculating a first predicted value at a specified time based on the temporal pose variable of the camera at the specified time and the pose deviation of the visual inertial odometer by using a preset function; Based on the prior pose of the visual inertial odometer at the specified moment and the first prediction value at the specified moment, calculate the prior pose error as the constraint on the pose variable of the camera and the pose deviation of the visual inertial odometer in the estimation model of the pose time series deviation of the visual inertial odometer.

5. The method according to claim 2, based on the established data association between the visual feature and the high-precision map and the prior pose of the visual inertial odometer, determining the pose deviation error of the visual inertial odometer as a constraint term in the estimation model of the pose time series deviation of the visual inertial odometer, includes: Based on a preset autoregressive pose deviation estimation model, calculate the pose deviation of the visual inertial odometer at the specified moment by using the pose deviations at a preset number of moments of the visual inertial odometer before the specified moment; Based on the calculated pose deviation of the visual inertial odometer at the specified moment and the time series deviation at a preset number of moments before the specified moment, calculate the pose deviation error of the visual inertial odometer as the constraint on the calculation process of the autoregressive pose deviation estimation model in the estimation model of the pose time series deviation of the visual inertial odometer.

6. The method according to claim 1, the establishing an estimation model of the pose time series deviation of the visual inertial odometer based on the established data association between the visual feature and the high-precision map and the prior pose of the visual inertial odometer, includes: Based on the established data association between the visual feature and the high-precision map, calculate the visual feature reprojection error of the visual inertial odometer as the constraint of the visual feature on the pose variable of the camera in the estimation model of the pose time series deviation of the visual inertial odometer; Based on the time series pose variable of the camera at the specified moment, calculate the prior pose error of the visual inertial odometer as the constraint on the pose variable of the camera and the pose deviation of the visual inertial odometer in the estimation model of the pose time series deviation of the visual inertial odometer; Based on a preset autoregressive pose deviation estimation model, calculate the pose deviation of the visual inertial odometer at the specified moment by using the deviation of the prior pose of the visual inertial odometer, and calculate the pose deviation error of the visual inertial odometer as the constraint on the calculation process of the autoregressive pose deviation estimation model in the estimation model of the pose time series deviation of the visual inertial odometer.

7. A visual inertial odometer, wherein, The visual inertial odometer includes a visual inertial odometer pose time series deviation estimation device, and the visual inertial odometer pose time series deviation estimation device includes: An extraction and establishment module, configured to extract visual features from an image and establish a data association between the visual features and a high-precision map, where the image is collected by a camera integrated with a visual odometer; an acquisition module, configured to acquire the prior pose of the visual inertial odometer; An estimation module, configured to establish an estimation model of the pose time-series deviation of the visual inertial odometer based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, and estimate the pose of the camera and the pose time-series deviation of the visual inertial odometer; wherein, the estimating the pose of the camera and the pose time-series deviation of the visual inertial odometer includes: Calculating the total error of the visual inertial odometer based on the visual feature reprojection error, the prior pose error, and the pose deviation error of the visual inertial odometer; Using a preset non-linear optimization algorithm, estimating the pose of the camera and the pose time-series deviation of the visual inertial odometer by minimizing the total error of the pose of the visual inertial odometer.

8. A visual inertial odometry positioning method, wherein, The method includes: Extracting visual features from an image and establishing a data association between the visual features and a high-precision map, where the image is collected by a camera integrated with a visual odometer; Obtaining the prior pose of the visual inertial odometer; Based on the established data association between the visual features and the high-precision map and the prior pose of the visual inertial odometer, establishing an estimation model of the pose time-series deviation of the visual inertial odometer, and estimating the pose of the camera and the pose time-series deviation of the visual inertial odometer; Positioning the visual inertial odometer based on the estimated pose of the camera and the pose time-series deviation of the visual inertial odometer; wherein, the estimating the pose of the camera and the pose time-series deviation of the visual inertial odometer includes: Calculating the total error of the visual inertial odometer based on the visual feature reprojection error, the prior pose error, and the pose deviation error of the visual inertial odometer; Using a preset non-linear optimization algorithm, estimating the pose of the camera and the pose time-series deviation of the visual inertial odometer by minimizing the total error of the pose of the visual inertial odometer.

9. A computer program product, including computer instructions, which when executed by a processor implement the method according to any one of claims 1-6, 8.

Images