Method for simultaneously performing positioning and map creation based on image information and computer

Abstract

This invention provides a method for minimizing the imbalance between accuracy and cost in image-based positioning and mapping (SLAM), while simultaneously ensuring low system latency during positioning. [Solution] The method includes receiving an image taken on a dynamic vehicle; performing an unsupervised depth feature matching step on the image to extract a plurality of visual features; performing a depth estimation step on the image to generate depth values ​​for the plurality of visual features; receiving odometer information of the dynamic vehicle; performing a map creation program to generate a result map based on the plurality of visual features, the depth values ​​of the plurality of visual features, and the odometer information; and performing a positioning program to generate a position based on the plurality of visual features, the depth values ​​of the plurality of visual features, the odometer information, and the result map.

Claims

1. Receiving images taken on a moving vehicle, Performing an unsupervised depth feature matching step on the aforementioned image to extract multiple visual features, Performing a depth estimation step on the aforementioned image to generate depth values ​​for the multiple visual features, Receiving odometer information of the aforementioned dynamic vehicle, Based on the aforementioned plurality of visual features, the depth values ​​of the plurality of visual features, and the odometer information, a map creation program is executed to generate a result map which is a map of the plurality of visual features. A method for simultaneously performing image-based positioning and map creation, comprising: executing a positioning program to generate the position of the dynamic beagle based on the plurality of visual features, the depth values ​​of the plurality of visual features, the odometer information, and the result map.

2. The unsupervised depth feature matching step further comprises processing the image using a first encoder-decoder frame, wherein the first encoder-decoder frame includes a common encoder for reducing the size of the image, a first decoder head for extracting a plurality of feature points from the output of the encoder, and a second decoder head for generating a plurality of feature descriptions corresponding to the extracted plurality of feature points, wherein the plurality of feature descriptions are used to indicate that the corresponding plurality of feature points are static or dynamic, the simultaneous method of positioning and mapping based on image information according to claim 1.

3. The size of the aforementioned image is I Ｗ ×I Ｈ The image is divided into multiple cells, and a corresponding intermediate tensor is generated for each cell, and the intermediate tensor is, [Math 1] Satisfying the condition, where F Ｃ indicates the channel depth, C indicates the intermediate tensor, IW(n) indicates the lateral dimension of the image, and IH(n) indicates the vertical dimension of the image. [Math 2] This represents a C-dimensional Euclidean space of IW(n) × IH(n) × F, and the size of the tensor output from the first decoder head is [Math 3] Satisfying the conditions, here, [Math 4] is the number of channels corresponding to the plurality of cells, and O I indicates the size of the tensor output from the first decoder head. [Math 5] teeth, [Math 6] The size of the tensor output from the second decoder head is [Number 7] Satisfying the conditions, here, [Number 8] is the number of channels corresponding to the multiple cells, and OD indicates the size of the tensor output from the second decoder head. [Number 9] teeth, [Number 10] A method for simultaneously performing positioning and map creation based on image information as described in claim 2.

4. The method for simultaneously performing positioning and map creation based on image information according to claim 2, wherein the second decoder head includes bicubic interpolation and L2 normalization of the plurality of feature descriptions.

5. The method for simultaneously performing positioning and map creation based on image information according to claim 2, wherein the first encoder-decoder frame is trained by distilled information from a teacher network that performs supervised training by markings in a dataset.

6. The simultaneous method for positioning and mapping based on image information according to claim 1, wherein the depth estimation step further comprises processing the image using a second encoder-decoder frame, wherein the second encoder-decoder frame includes a transformer-based backbone network for extracting multiple multiscale features and a decoder program for processing the multiple multiscale features to generate a disparity-log volume.

7. The method for simultaneously performing positioning and map creation based on image information according to claim 6, wherein the decoder program includes, in order, a plurality of adaptive feature aggregation modules for processing the plurality of multiscale features, a convolution module including upsampling and a plurality of convolutional layers, and an output layer.

8. The method for simultaneously performing positioning and mapping based on image information according to claim 6, wherein the second encoder-decoder frame is trained by single-network self-distillation for refining the parallax-log space.

9. The method for simultaneously performing positioning and mapping based on image information according to claim 6, wherein the second encoder-decoder frame is trained by a loss function that includes a weighted sum of three main loss terms, including luminance reconstruction loss, smoothing normalization loss, and self-distillation consistency loss.

10. The method for simultaneously performing positioning and map creation based on image information according to claim 1, wherein the positioning program includes obtaining an initial estimated point on the result map based on the odometer information.

11. The method for simultaneously performing positioning and map creation based on image information according to claim 1, wherein the positioning program includes a progressive non-convexity (GNC) nonlinear optimization program for matching the plurality of visual features to feature points of an existing map.

12. The simultaneous method of positioning and map creation based on image information according to claim 1, wherein the odometer information is obtained from one or any combination thereof of information provided by the odometer of the rim count on the dynamic vehicle, information provided by the receiver of the global navigation satellite system on the dynamic vehicle, and information provided by the inertial measurement unit on the dynamic vehicle.

13. A computer used for simultaneous positioning and map creation, The computer is used to execute a plurality of instructions stored in non-volatile memory to realize the simultaneous execution method of positioning and map creation described in any one of claims 1 to 12, wherein the computer is a computer located on the dynamic vehicle.

14. A computer used for simultaneous positioning and mapping according to claim 13, further comprising one or any combination thereof of a rim count odometer on the dynamic vehicle, a receiver for a global navigation satellite system on the dynamic vehicle, and an inertial measurement unit on the dynamic vehicle.

Images