Visual inertia fusion navigation positioning method based on visual information selection

Abstract

The invention relates to a visual inertial fusion navigation positioning method based on visual information selection, which comprises the following steps of: acquiring IMU (Inertial Measurement Unit) data and two adjacent frames of image data, inputting a visual inertial fusion navigation positioning model based on visual information selection, and outputting a navigation positioning result; the model comprises an IMU feature extraction network, a visual feature extraction network, a strategy network and a fusion network, and the IMU feature extraction network is constructed based on LSTM introducing expansion convolution and is used for extracting inertial features from IMU data; the visual feature extraction network is constructed based on a CNN (Convolutional Neural Network) introducing a channel-space attention mechanism, and is used for extracting image features from two adjacent frames of image data; the strategy network is used for judging whether the image features are forbidden or not; and the fusion network is constructed based on LSTM (Long Short Term Memory) introducing an efficient local attention mechanism, and is used for fusing the inertial features and the image features output by the strategy network. Compared with the prior art, efficient and accurate navigation can be achieved without complex calibration modeling or a large number of true value data sets.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent navigation, and in particular relates to a visual-inertial fusion navigation and positioning method based on visual information selection. Background Art

[0002] With the development of artificial intelligence technology and the engineering application of smart chips, autonomous intelligent precision navigation methods have become a major research hotspot. Traditional visual navigation methods usually rely on image data obtained by cameras. Cameras have the advantages of low cost, small size and the ability to achieve positioning in complex environments, so they are widely used in various engineering fields. By using the rich visual information collected by the camera and analyzing the spatial information of adjacent frame images, the position and posture information of the device can be calculated, so that the motion trajectory of the device in a continuous time period can be obtained. This process is called visual odometry (VO).

[0003] However, since traditional visual navigation methods are difficult to achieve high-precision positioning tasks under harsh conditions such as blurred images or insufficient light, other sensors are usually required for supplementary measurements. Inertial Measurement Unit (IMU) is widely used in the field of navigation due to its small size, low cost and measurement results that are not easily affected by external interference. However, the data obtained by the inertial measurement unit will produce various cumulative errors such as random walk after long-term operation, which greatly reduces its credibility and leads to low heading estimation accuracy. In the case of limited payload capacity, fusing IMU data with visual odometer into visual inertial odometer (VIO) can effectively combine the advantages of the two sensors to improve positioning and navigation performance and robustness.

[0004] At present, there are two main methods of visual inertial odometry. One is the traditional visual inertial odometry, which relies on accurate sensor calibration and complex modeling to achieve state estimation and has poor robustness. The other is the visual inertial odometry based on deep learning, which is more robust but requires a large number of data sets containing true values, which are often difficult to obtain in actual engineering. Therefore, it is necessary to design a visual inertial fusion navigation and positioning method with high robustness, which can achieve efficient and accurate navigation without complex calibration modeling or the lack of a large number of true value data sets. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a visual-inertial fusion navigation and positioning method based on visual information selection, which can achieve efficient and accurate navigation without complex calibration modeling or lack of a large number of true value data sets.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] The present invention provides a visual inertial fusion navigation and positioning method based on visual information selection, comprising the following steps:

[0008] Obtain IMU data and two adjacent frames of image data, input the visual inertial fusion navigation and positioning model selected based on visual information, and output the navigation and positioning results;

[0009] Among them, the visual-inertial fusion navigation and positioning model based on visual information selection includes an IMU feature extraction network, a visual feature extraction network, a policy network and a fusion network. The IMU feature extraction network is constructed based on an LSTM that introduces dilated convolution, and is used to extract inertial features from IMU data; the visual feature extraction network is constructed based on a CNN that introduces a channel-spatial attention mechanism, and is used to extract image features from two adjacent frames of image data; the policy network is used to determine whether to disable the image feature. If so, all elements in the image feature are replaced with 0 and then input into the fusion network, otherwise, it is directly input into the fusion network; the fusion network is constructed based on an LSTM that introduces an efficient local attention mechanism, and is used to fuse the inertial features with the image features output by the policy network to obtain a navigation and positioning result.

[0010] Furthermore, in the IMU feature extraction network, the features of the IMU data are initially extracted by dilated convolution, and then input into LSTM for further feature extraction to obtain the inertial features. The dilated convolution is to add an interval inside the convolution kernel of the CNN.

[0011] Furthermore, the receptive field RF of the current layer of the dilated convolution i Specifically:

[0012] RF i =RF i-1 +(k-1)×s

[0013] Among them, RF i-1 represents the upper layer receptive field, k represents the size of the convolution kernel, and s represents the step size.

[0014] Furthermore, in the visual feature extraction network, the channel-spatial attention mechanism includes a channel attention module and a spatial attention module, wherein the channel attention module is used to generate a channel attention feature F′ in the channel dimension according to the feature map F, and the spatial attention module is used to generate a spatial attention feature F″ in the spatial dimension according to the channel attention feature F′, wherein the feature map F is generated by a convolution block based on two adjacent frames of image data, F∈R C×H×W , C represents the channel dimension, H represents the height, and W represents the width.

[0015] Furthermore, the specific process of generating the channel attention feature F′ is as follows:

[0016]

[0017] Among them, M c represents the 1-D channel attention feature map, M c ∈R C×1×1 .

[0018] Furthermore, the specific process of generating the spatial attention feature F" is as follows:

[0019]

[0020] Among them, M s represents the 2-D spatial attention feature map, M s ∈R 1×H×W .

[0021] Furthermore, the policy network determines whether to disable the image feature through binary decision making, and the specific process is as follows:

[0022] Calculate the probability p based on the current inertial feature and the previous hidden vector;

[0023] Discretely sample the probability p to obtain the binary parameter d t , if d t =1, it is determined that the image feature is enabled. If d t =0, it is determined that the image feature is disabled.

[0024] Furthermore, the probability p is discretely sampled by reparameterizing Gumbel-Softmax through a discrete distribution.

[0025] Furthermore, the fusion network first fuses the inertial features with the image features output by the strategy network through an efficient local attention mechanism to obtain fused features; then the fused features are input into the LSTM to obtain the navigation positioning results through full-connection layer regression estimation.

[0026] Furthermore, the visual inertial fusion navigation positioning model based on visual information selection is trained by self-supervised learning, and the loss function L used in the training process is pose The root mean square error based on position and attitude is constructed as follows:

[0027]

[0028] Where T represents the length of the training sequence, and denote the estimated translation vectors at time t and time t-1, Δvt i Represents the translation change measured by the inertial measurement unit within a time interval, and denote the estimated rotation vectors at time t and time t-1, Δφ t i It represents the attitude change measured by the inertial measurement unit within a period of time, and α represents the weight of balancing the translation error and the rotation error.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present invention proposes a visual-inertial fusion navigation and positioning method based on visual information selection, designs a visual-inertial fusion navigation and positioning model based on visual information selection, and can obtain navigation and positioning results according to IMU data and two adjacent frames of image data; the visual-inertial fusion navigation and positioning model based on visual information selection specifically includes an IMU feature extraction network, a visual feature extraction network, a strategy network and a fusion network, wherein the IMU feature extraction network is constructed based on an LSTM with an introduction of dilated convolution, and is used to extract inertial features from IMU data; the visual feature extraction network is constructed based on a CNN with an introduction of a channel-spatial attention mechanism, and is used to extract image features from two adjacent frames of image data; the strategy network is used to determine whether to disable the image features, and if so, After replacing all elements in the image features with 0, they are input into the fusion network, otherwise they are directly input into the fusion network; the fusion network is constructed based on the LSTM that introduces an efficient local attention mechanism, which is used to fuse the inertial features with the image features output by the policy network to obtain the navigation and positioning results; the above method can avoid the complex calibration and error modeling process in the traditional method by constructing an end-to-end network architecture. Compared with the navigation method relying on pure IMU or pure vision, the method proposed in the present invention has higher robustness. Even if one of the visual or inertial measurement units is degraded, it can still maintain stable and accurate navigation performance. In addition, the above method selects visual information through the policy network, which can significantly reduce the computational cost of high-dimensional visual features and improve navigation efficiency.

[0031] 2. In the IMU feature extraction network, the present invention preliminarily extracts the features of the IMU data through dilated convolution, and then inputs it into LSTM for further feature extraction to obtain inertial features. Dilated convolution is to add intervals inside the convolution kernel of CNN, which can expand the convolution kernel size and receptive field, so that LSTM can capture long-term dependencies at a higher level of feature representation.

[0032] 3. In the visual feature extraction network, the channel-spatial attention mechanism includes a channel attention module and a spatial attention module. The channel attention module is used to generate a channel attention feature F′ in the channel dimension according to the feature map F, and the spatial attention module is used to generate a spatial attention feature F″ in the spatial dimension according to the channel attention feature F′. In this process, element-wise multiplication involves dimensional transformation and matching of the feature map. This mechanism effectively enhances the network’s sensitivity to important features and can provide richer and more reliable information for subsequent navigation tasks.

[0033] 4. The present invention is trained through self-supervised learning and can still operate effectively even in the absence of a large amount of true value data set, and has significant adaptability and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a schematic diagram of the structure of the visual-inertial fusion navigation and positioning model based on visual information selection.

[0035] Among them, x v represents the image features, x i represents the inertial characteristics, z t represents the fusion feature, h t-1 represents the previous hidden vector;

[0036] Figure 2 This is a schematic diagram of the structure of the visual feature extraction network;

[0037] Figure 3 is a flow chart of the method of the present invention;

[0038] Figure 4 This is a comparison between the navigation results of the method proposed in the present invention and the true value in the dataset No. 10 in the KITTI dataset. DETAILED DESCRIPTION

[0039] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0040] Example:

[0041] This embodiment provides a visual inertial fusion navigation and positioning method based on visual information selection, such as Figure 3As shown, the following steps are included: first, IMU data and two adjacent frames of image data are obtained, a visual inertial fusion navigation and positioning model (network) selected based on visual information is input, and a navigation and positioning result is output. In the visual inertial fusion navigation and positioning model selected based on visual information, the strategy network is used to determine whether the current visual information obtained by the visual feature network based on two adjacent frames of image data needs to be added to the visual inertial fusion network, and the features obtained by the IMU feature extraction network based on the IMU data are fused.

[0042] The visual inertial fusion navigation and positioning model based on visual information selection is as follows Figure 1 As shown, it includes an IMU feature extraction network, a visual feature extraction network, a policy network and a fusion network. Among them, the IMU feature extraction network is built based on the LSTM that introduces dilated convolution, and is used to extract inertial features from IMU data; the visual feature extraction network is built based on the CNN that introduces the channel-space attention mechanism, and is used to extract image features from two adjacent frames of image data; the policy network is used to determine whether to disable the image feature. If so, all elements in the image feature are replaced with 0 and then input into the fusion network, otherwise it is directly input into the fusion network; the fusion network is built based on the LSTM that introduces an efficient local attention mechanism, and is used to fuse the inertial features with the image features output by the policy network to obtain navigation and positioning results. The detailed description of each network is as follows:

[0043] A. IMU feature extraction network

[0044] IMU data includes linear acceleration and angular velocity data collected by inertial sensors. The IMU feature extraction network converts complex acceleration and angular velocity signals into abstract feature representations through layer-by-layer convolution and feature learning in the time dimension to obtain potential information about the current motion state. The traditional CNN structure includes convolution layers and pooling layers. Although it has excellent feature extraction capabilities, the feature information contained in the IMU data is not obvious enough. Using traditional CNN for feature extraction may lead to the neglect of detailed feature information. To solve this problem, this embodiment introduces dilated convolution in the IMU feature extraction network. Dilated convolution expands the receptive field without increasing the number of parameters by adding intervals inside the convolution kernel, thereby capturing contextual information of a larger range and different scales. Dilated convolution adds intervals to the basic convolution kernel to increase the size of the convolution kernel and the receptive field. Receptive field RF of the current layer i for:

[0045] RF i =RF i-1 +(k-1)×s

[0046] Among them, RF i-1represents the upper layer receptive field, k represents the size of the convolution kernel, and s represents the step size. Dilated convolution can effectively expand the scope of the convolution kernel, capture more details and global features in the IMU signal, and provide richer inertial feature information for the subsequent fusion network. The features extracted by the dilated convolution are input into the LSTM for further feature extraction, and finally the inertial feature x can be obtained. i .

[0047] B. Visual Feature Extraction Network

[0048] Image data usually contains rich effective feature information, but if only two adjacent frames of images are simply spliced ​​and then feature extracted, some key information may be lost. Therefore, this embodiment introduces a channel-space attention mechanism in the visual feature extraction network. Convolutional Block Attention Module (CBAM) is a representative method in the channel-space attention mechanism, which has the advantages of lightweight and strong generalization ability. CBAM generates attention feature maps in two dimensions, channel and space, thereby enhancing the network's attention to key information. The two feature maps are finally multiplied with the original input feature map to achieve adaptive feature correction and generate the final feature map. The specific process is as follows:

[0049] like Figure 2 As shown, first, the feature map F is generated by the convolution block, F∈R C×H×W , R C×H×W Denotes the output of the convolutional block, C denotes the channel dimension, H denotes the height, and W denotes the width. Then, the channel attention feature F′ is generated in the channel dimension through the channel attention module, and the spatial attention feature F" is generated in the spatial dimension through the spatial attention module. The specific process is as follows:

[0050]

[0051] Among them, M c represents the 1-D channel attention feature map, M c ∈R C×1×1 , M s represents the 2-D spatial attention feature map, M s ∈R 1×H×W Specifically, M c It can be written as:

[0052] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0053] Among them, AvgPool and MaxPool represent global average pooling and maximum pooling operations respectively, MLP represents multi-layer perceptron, and σ represents Sigmoid activation function.

[0054] M s It can be written as:

[0055] M s (F′)=σ(f n×n ([AvgPool(F′);MaxPool(F′)]))

[0056] where f n×n Represents an n×n convolution operation.

[0057] In this process, element-level multiplication involves dimensional transformation and matching of feature maps. This mechanism effectively enhances the network’s sensitivity to important features, providing richer and more reliable information for subsequent navigation tasks, and finally obtaining the image feature x v .

[0058] C. Policy Network

[0059] Compared with inertial data, image data contains richer feature information. Therefore, the image features extracted from a set of data are far more than inertial features. If all the extracted feature information is used for state estimation, the system computational burden will increase dramatically. Therefore, this embodiment introduces a policy network to disable visual information without significantly affecting the accuracy of posture estimation. This policy network uses an adaptive mechanism to learn how to make binary decisions to determine whether visual information is necessary. This decision process is sampled from a Bernoulli distribution, and its probability p is calculated by the policy network Φ based on the current inertial feature and the hidden vector h of the previous time step t-1 To do the calculation:

[0060]

[0061] In order to enable the system to be trained end-to-end, the probability p is discretely sampled by reparameterizing Gumbel-Softmax through discrete distribution to obtain the binary parameter d t :

[0062] dt=Gumbel(p)

[0063] If d t =1, then the image feature is enabled. If d t = 0, then the image feature is disabled. Replace image features with zero padding.

[0064] D. Converged Network

[0065] The main function of the fusion network is to effectively fuse the previously extracted visual features with the inertial features determined by the policy network. The spatial dimension of the data carries key location information. Existing attention mechanisms often cannot make full use of this spatial information, or they need to reduce the channel dimension when using spatial information, thereby affecting the representation ability of the features. To solve this problem, this embodiment introduces an efficient local attention mechanism (ELA) in the fusion network, which combines one-dimensional convolution and group normalization (GN) feature enhancement technology. First, stripe pooling (SP) is used to extract the spatial features x of the spliced ​​visual features and inertial features. c , capturing long-distance spatial dependencies while maintaining the information integrity of the channel dimension. Then, average pooling is performed in the horizontal and vertical directions respectively, obtaining representations in two directions, namely:

[0066]

[0067] After the above steps, the fusion network can capture long-distance spatial features while maintaining the information integrity of the channel dimension. In order to effectively utilize these features, one-dimensional convolution is applied to the position information in both horizontal and vertical directions to enhance its information, and group normalization GN is used to process the position information that needs to be enhanced. The position attention in the horizontal and vertical directions can be expressed as:

[0068] y h =σ(G n (F h (z h )))

[0069] y w =σ(G n (F w (z w )))

[0070] Among them, σ represents the nonlinear activation function, F h and F w represents one-dimensional convolution, G n Represents group normalization, and finally obtains the output of the network, that is, the final fusion feature:

[0071] z t =x c *y h *y w

[0072] This method can accurately locate the region of interest without dimensionality reduction, and the network has the advantages of being lightweight, fast, and having strong generalization capabilities. The fusion network fuses the inertial features with the image features through the above-mentioned ELA efficient local attention mechanism to obtain fusion features;

[0073]

[0074] in, Indicates an ELA fusion operation.

[0075] Finally, the fused features are input into LSTM, and the data output containing 6-dimensional navigation information is obtained through full connection layer regression estimation:

[0076]

[0077] in, represents the estimated translation vector at time t, Represents the estimated rotation vector at time t.

[0078] The above-mentioned visual-inertial fusion navigation and positioning model based on visual information selection is trained through self-supervised learning. Since the errors of the inertial measurement unit are often accumulated over time, the changes in speed, attitude and position measured in a short time interval are usually more accurate. Therefore, during the network training process, the attitude and position changes measured by the IMU in the current two adjacent image time periods are set to true values. This processing method provides a more reliable reference benchmark for the overall network architecture, so that the model can be optimized based on relatively accurate measurement results during the self-supervised learning training process, thereby improving the overall navigation accuracy of the system, and also enables the system to adjust and compensate more effectively when facing long-term accumulated errors. Therefore, the root mean square error (RMSE) of position and attitude is used to define the loss function L during network training. pose :

[0079]

[0080] Where T represents the length of the training sequence, and denote the estimated translation vectors at time t and time t-1, Δv t i Represents the translation change measured by the inertial measurement unit within a time interval, and denote the estimated rotation vectors at time t and time t-1, Δφ t i It represents the attitude change measured by the inertial measurement unit within a period of time, and α represents the weight of balancing the translation error and the rotation error.

[0081] The above method is suitable for scenarios with limited computing power of devices. The end-to-end network structure effectively avoids the problem of complex calibration and error modeling required in traditional visual inertial fusion navigation and positioning, and the unsupervised learning method avoids the problem of difficulty in obtaining true values ​​in actual engineering applications. In addition, the above method can significantly improve navigation efficiency by reducing the computational cost of high-dimensional visual features.

[0082] To verify the effectiveness of the above method, this embodiment first uses the public dataset KITTI to verify the model. Specifically, the data with sequence numbers 01, 02, 04, 05, 06, 07 and 08 in the dataset are used as training sets, and the data with sequence numbers 09 and 10 are used as test sets to verify the effectiveness of the proposed method. Figure 4 The comparison between the navigation results of the proposed method in the sequence number 10 data set and the true value is shown. It can be seen that the navigation accuracy of the proposed method in the present invention is higher.

[0083] Secondly, the proposed method is compared with two mainstream vision and IMU fusion methods (tightly coupled nonlinear VINS-Mono and deep learning DeepVIO) in terms of accuracy and computational consumption. rel , average rotation error R rel The three parameters of visual feature utilization are compared, and the comparison results are shown in Table 1.

[0084] Table 1 Comparison of the effects of the method of the present invention and the existing method

[0085] method <![CDATA[T rel ]]> <![CDATA[R rel ]]> Visual feature utilization VINS-Mono 16.50 0.02 100％ DeepVIO 0.85 0.01 100％ Method of the present invention 3.59 0.01 25.76％

[0086] By comparing Table 1, it can be found that the method proposed in the present invention reduces the utilization rate of visual features by 74.24% compared with the two mainstream methods, so the method can greatly reduce the amount of calculation required by the system. rel Reduced by 1 / 5, the average rotation error R rel The average translation error T of the method proposed in this invention is reduced by 1 / 2 compared with the DeepVIO method. rel Increased by 4 times, the average rotation error R rel The above comparative experiments prove that the method proposed in the present invention can significantly reduce the amount of calculation while maintaining a certain navigation accuracy, and can meet the actual needs of engineering.

[0087] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0088] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A visual inertial fusion navigation and positioning method based on visual information selection, characterized in that: The following steps are involved: Obtain IMU data and two adjacent frames of image data, input the visual inertial fusion navigation and positioning model selected based on visual information, and output the navigation and positioning results; Among them, the visual-inertial fusion navigation and positioning model based on visual information selection includes an IMU feature extraction network, a visual feature extraction network, a policy network and a fusion network. The IMU feature extraction network is constructed based on an LSTM that introduces dilated convolution, and is used to extract inertial features from IMU data; the visual feature extraction network is constructed based on a CNN that introduces a channel-spatial attention mechanism, and is used to extract image features from two adjacent frames of image data; the policy network is used to determine whether to disable the image feature. If so, all elements in the image feature are replaced with 0 and then input into the fusion network, otherwise, it is directly input into the fusion network; the fusion network is constructed based on an LSTM that introduces an efficient local attention mechanism, and is used to fuse the inertial features with the image features output by the policy network to obtain a navigation and positioning result.

2. The visual inertial fusion navigation and positioning method based on visual information selection according to claim 1 is characterized in that: In the IMU feature extraction network, the features of the IMU data are initially extracted through dilated convolution, and then input into LSTM for further feature extraction to obtain the inertial features. The dilated convolution is to add an interval inside the convolution kernel of the CNN.

3. The visual inertial fusion navigation and positioning method based on visual information selection according to claim 2 is characterized in that: The receptive field RF of the current layer of the dilated convolution i Specifically: RF i =RF i-1 +(k-1)×s Among them, RF i-1 represents the upper layer receptive field, k represents the size of the convolution kernel, and s represents the step size.

4. The visual inertial fusion navigation and positioning method based on visual information selection according to claim 1 is characterized in that: In the visual feature extraction network, the channel-spatial attention mechanism includes a channel attention module and a spatial attention module. The channel attention module is used to generate a channel attention feature F′ in the channel dimension according to the feature map F, and the spatial attention module is used to generate a spatial attention feature F″ in the spatial dimension according to the channel attention feature F′. The feature map F is generated by a convolution block based on two adjacent frames of image data, F∈R C×H×W , C represents the channel dimension, H represents the height, and W represents the width.

5. The visual inertial fusion navigation and positioning method based on visual information selection according to claim 4 is characterized in that: The specific process of generating the channel attention feature F′ is as follows: Among them, M c represents the 1-D channel attention feature map, M c ∈R C×1×1 .

6. The visual inertial fusion navigation and positioning method based on visual information selection according to claim 4 is characterized in that: The specific process of generating the spatial attention feature F" is as follows: Among them, M s represents the 2-D spatial attention feature map, M s ∈R 1×H×W .

7. The visual inertial fusion navigation and positioning method based on visual information selection according to claim 1 is characterized in that: The policy network determines whether to disable the image feature through binary decision making, and the specific process is as follows: Calculate the probability p based on the current inertial feature and the previous hidden vector; Discretely sample the probability p to obtain the binary parameter d t , if d t =1, it is determined that the image feature is enabled. If d t =0, it is determined that the image feature is disabled.

8. The visual inertial fusion navigation and positioning method based on visual information selection according to claim 7 is characterized in that: Reparameterize Gumbel-Softmax via a discrete distribution to discretely sample probability p.

9. The visual inertial fusion navigation and positioning method based on visual information selection according to claim 1, characterized in that: The fusion network first fuses the inertial features with the image features output by the strategy network through an efficient local attention mechanism to obtain fusion features; then the fusion features are input into the LSTM to obtain the navigation positioning results through full connection layer regression estimation.

10. The visual inertial fusion navigation and positioning method based on visual information selection according to claim 1, characterized in that: The visual inertial fusion navigation and positioning model based on visual information selection is trained by self-supervised learning, and the loss function L used in the training process is pose The root mean square error based on position and attitude is constructed as follows: Where T represents the length of the training sequence, and Represent the estimated translation vectors at time t and time t-1 respectively, Represents the translation change measured by the inertial measurement unit within a time interval, and denote the estimated rotation vectors at time t and time t-1, Δφ t i It represents the attitude change measured by the inertial measurement unit within a period of time, and α represents the weight of balancing the translation error and the rotation error.

Images