A visual-inertial fusion navigation positioning method based on visual information selection

Abstract

The application relates to a visual inertial fusion navigation positioning method based on visual information selection, which comprises the following steps: acquiring IMU data and adjacent two frames of image data, inputting a visual inertial fusion navigation positioning model based on visual information selection, and outputting a navigation positioning result; the model comprises an IMU feature extraction network, a visual feature extraction network, a strategy network and a fusion network, the IMU feature extraction network is constructed based on an LSTM with an introduced expansion convolution and is used for extracting inertial features from the IMU data; the visual feature extraction network is constructed based on a CNN with an introduced channel-space attention mechanism and is used for extracting image features from the adjacent two frames of image data; the strategy network is used for judging whether to disable the image features; and the fusion network is constructed based on an LSTM with an introduced efficient local attention mechanism and is used for fusing the inertial features and the image features output by the strategy network. Compared with the prior art, the application can realize efficient and accurate navigation without complex calibration modeling or a large amount of true value data set.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent navigation technology, and in particular relates to a visual-inertial fusion navigation and positioning method based on visual information selection. Background Art

[0002] With the development of artificial intelligence (AI) technology and the engineering application of intelligent chips, autonomous, intelligent, and precise navigation methods have become a major research hotspot. Traditional visual navigation methods typically rely on image data captured by cameras. Cameras are low-cost, compact, and capable of positioning in complex environments, making them widely used in various engineering fields. Leveraging the rich visual information captured by cameras, the position and posture of a device can be calculated by analyzing the spatial information of adjacent image frames, thereby determining its trajectory over a continuous period of time. This process is called visual odometry (VO).

[0003] However, since traditional visual navigation methods have difficulty achieving high-precision positioning tasks under harsh conditions such as blurred images or insufficient light, other sensors are usually required for supplementary measurements. Inertial Measurement Unit (IMU) is widely used in the field of navigation due to its small size, low cost and measurement results that are not easily affected by external interference. However, the data obtained by the inertial measurement unit will produce various cumulative errors such as random walk after long-term operation, which greatly reduces its credibility and leads to low heading estimation accuracy. In the case of limited payload capacity, fusing IMU data with visual odometry into visual inertial odometry (VIO) can effectively combine the advantages of the two sensors to improve positioning and navigation performance and robustness.

[0004] Currently, there are two main approaches to visual-inertial odometry: traditional visual-inertial odometry, which relies on precise sensor calibration and complex modeling to achieve state estimation and suffers from poor robustness. Deep learning-based visual-inertial odometry, on the other hand, offers greater robustness but requires large datasets containing ground truth, which are often difficult to obtain in practical applications. Therefore, a robust visual-inertial fusion navigation and positioning method is needed that can achieve efficient and accurate navigation without complex calibration and modeling or the lack of large datasets containing ground truth. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a visual-inertial fusion navigation and positioning method based on visual information selection, which can achieve efficient and accurate navigation without complex calibration modeling or lack of a large number of true value data sets.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] The present invention provides a visual-inertial fusion navigation and positioning method based on visual information selection, comprising the following steps:

[0008] Obtain IMU data and two adjacent frames of image data, input the visual-inertial fusion navigation and positioning model selected based on visual information, and output the navigation and positioning results;

[0009] Among them, the visual-inertial fusion navigation and positioning model based on visual information selection includes an IMU feature extraction network, a visual feature extraction network, a policy network and a fusion network. The IMU feature extraction network is constructed based on an LSTM that introduces dilated convolution, and is used to extract inertial features from IMU data; the visual feature extraction network is constructed based on a CNN that introduces a channel-spatial attention mechanism, and is used to extract image features from two adjacent frames of image data; the policy network is used to determine whether to disable the image feature. If so, all elements in the image feature are replaced with 0 and then input into the fusion network, otherwise it is directly input into the fusion network; the fusion network is constructed based on an LSTM that introduces an efficient local attention mechanism, and is used to fuse the inertial features with the image features output by the policy network to obtain navigation and positioning results.

[0010] Furthermore, in the IMU feature extraction network, the features of the IMU data are initially extracted by dilated convolution, and then input into LSTM for further feature extraction to obtain the inertial features. The dilated convolution is to add an interval inside the convolution kernel of the CNN.

[0011] Furthermore, the receptive field RF of the current layer of the dilated convolution i Specifically:

[0012] RF i =RF i-1 +(k-1)×s

[0013] Among them, RF i-1 Represents the upper layer receptive field, k represents the size of the convolution kernel, and s represents the step size.

[0014] Furthermore, in the visual feature extraction network, the channel-spatial attention mechanism includes a channel attention module and a spatial attention module. The channel attention module is used to generate a channel attention feature F′ in the channel dimension according to the feature map F, and the spatial attention module is used to generate a spatial attention feature F″ in the spatial dimension according to the channel attention feature F′. The feature map F is generated by a convolution block based on two adjacent frames of image data, F∈R C×H×W , C represents the channel dimension, H represents the height, and W represents the width.

[0015] Furthermore, the specific process of generating the channel attention feature F′ is as follows:

[0016]

[0017] Among them, M c represents the 1-D channel attention feature map, M c ∈R C×1×1 .

[0018] Furthermore, the specific process of generating the spatial attention feature F" is as follows:

[0019]

[0020] Among them, M s represents the 2-D spatial attention feature map, M s ∈R 1×H×W .

[0021] Furthermore, the policy network determines whether to disable the image feature through binary decision making. The specific process is as follows:

[0022] Calculate the probability p based on the current inertial feature and the previous hidden vector;

[0023] Discretely sample the probability p to obtain the binary parameter d t , if d t =1, it is determined that the image feature is enabled, if d t =0, it is determined that the image feature is disabled.

[0024] Furthermore, the probability p is discretely sampled by reparameterizing the Gumbel-Softmax with a discrete distribution.

[0025] Furthermore, the fusion network first fuses the inertial features with the image features output by the strategy network through an efficient local attention mechanism to obtain fusion features; then the fusion features are input into the LSTM, and the navigation positioning results are obtained through full-connection layer regression estimation.

[0026] Furthermore, the visual inertial fusion navigation positioning model based on visual information selection is trained by self-supervised learning, and the loss function L used in the training process is pose The root mean square error based on position and attitude is constructed as follows:

[0027]

[0028] Where T represents the length of the training sequence, and Denote the estimated translation vectors at time t and time t-1, Δvt i Represents the translation change measured by the inertial measurement unit within a time interval, and Represent the estimated rotation vectors at time t and time t-1, Δφ t i It represents the attitude change measured by the inertial measurement unit within a time interval, and α represents the weight of balancing the translation error and the rotation error.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present invention proposes a visual-inertial fusion navigation and positioning method based on visual information selection, and designs a visual-inertial fusion navigation and positioning model based on visual information selection, which can obtain navigation and positioning results based on IMU data and two adjacent frames of image data; the visual-inertial fusion navigation and positioning model based on visual information selection specifically includes an IMU feature extraction network, a visual feature extraction network, a policy network and a fusion network, wherein the IMU feature extraction network is constructed based on the LSTM with the introduction of dilated convolution, and is used to extract inertial features from IMU data; the visual feature extraction network is constructed based on the CNN with the introduction of channel-spatial attention mechanism, and is used to extract image features from two adjacent frames of image data; the policy network is used to determine whether to disable the image feature, and if so, After replacing all elements in the image features with 0, they are input into the fusion network, otherwise they are directly input into the fusion network; the fusion network is constructed based on the LSTM that introduces an efficient local attention mechanism, which is used to fuse the inertial features with the image features output by the policy network to obtain the navigation and positioning results; the above method can avoid the complex calibration and error modeling process in the traditional method by constructing an end-to-end network architecture. Compared with the navigation method relying on pure IMU or pure vision, the method proposed in the present invention has higher robustness. Even if one of the visual or inertial measurement units is degraded, it can still maintain stable and accurate navigation performance. In addition, the above method selects visual information through the policy network, which can significantly reduce the computational cost of high-dimensional visual features and improve navigation efficiency.

[0031] 2. In the IMU feature extraction network, the present invention preliminarily extracts the features of the IMU data through dilated convolution, and then inputs it into LSTM for further feature extraction to obtain inertial features. Dilated convolution is to add intervals inside the convolution kernel of CNN, which can expand the convolution kernel size and receptive field, so that LSTM can capture long-term dependencies at a higher level of feature representation.

[0032] 3. In the visual feature extraction network, the channel-spatial attention mechanism includes a channel attention module and a spatial attention module. The channel attention module is used to generate channel attention features F′ in the channel dimension based on the feature map F, and the spatial attention module is used to generate spatial attention features F″ in the spatial dimension based on the channel attention features F′. In this process, element-wise multiplication involves dimensional transformation and matching of the feature map. This mechanism effectively enhances the network's sensitivity to important features and can provide richer and more reliable information for subsequent navigation tasks.

[0033] 4. The present invention is trained through self-supervised learning and can still operate effectively even in the absence of a large amount of true value data set, with significant adaptability and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a structural diagram of the visual-inertial fusion navigation and positioning model based on visual information selection.

[0035] Among them, x v Represents image features, x i represents the inertial characteristics, z t represents the fusion feature, h t-1 represents the previous hidden vector;

[0036] Figure 2 Schematic diagram of the structure of the visual feature extraction network;

[0037] Figure 3 is a flow chart of the method of the present invention;

[0038] Figure 4 This is a comparison between the navigation results of the proposed method and the true value in the dataset No. 10 in the KITTI dataset. DETAILED DESCRIPTION

[0039] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0040] Example:

[0041] This embodiment provides a visual-inertial fusion navigation and positioning method based on visual information selection, such as Figure 3As shown, the method includes the following steps: First, IMU data and two adjacent frames of image data are obtained, input into a visual-inertial fusion navigation and positioning model (network) based on visual information selection, and the navigation and positioning results are output. In the visual-inertial fusion navigation and positioning model based on visual information selection, the policy network determines whether the current visual information obtained by the visual feature network based on the two adjacent frames of image data needs to be added to the visual-inertial fusion network and fused with the features obtained by the IMU feature extraction network based on the IMU data.

[0042] The visual inertial fusion navigation and positioning model based on visual information selection is as follows: Figure 1 As shown in the figure, it includes an IMU feature extraction network, a visual feature extraction network, a policy network, and a fusion network. Among them, the IMU feature extraction network is built based on an LSTM that introduces dilated convolutions and is used to extract inertial features from IMU data; the visual feature extraction network is built based on a CNN that introduces a channel-spatial attention mechanism and is used to extract image features from two adjacent frames of image data; the policy network is used to determine whether to disable image features. If so, all elements in the image features are replaced with 0 and then input into the fusion network; otherwise, they are directly input into the fusion network; the fusion network is built based on an LSTM that introduces an efficient local attention mechanism and is used to fuse inertial features with image features output by the policy network to obtain navigation and positioning results. The detailed description of each network is as follows:

[0043] A. IMU feature extraction network

[0044] IMU data includes linear acceleration and angular velocity data collected by inertial sensors. The IMU feature extraction network converts complex acceleration and angular velocity signals into abstract feature representations through layer-by-layer convolution and feature learning in the time dimension to obtain potential information about the current motion state. The traditional CNN structure includes convolution layers and pooling layers. Although it has excellent feature extraction capabilities, the feature information contained in the IMU data is not obvious enough. Using traditional CNN for feature extraction may lead to the neglect of detailed feature information. To solve this problem, this embodiment introduces dilated convolution in the IMU feature extraction network. Dilated convolution expands the receptive field without increasing the number of parameters by adding intervals inside the convolution kernel, thereby capturing contextual information of a wider range and different scales. Dilated convolution adds intervals to the basic convolution kernel to increase the size of the convolution kernel and the receptive field. Current layer receptive field RF i for:

[0045] RF i =RF i-1 +(k-1)×s

[0046] Among them, RF i-1denotes the upper layer receptive field, k denotes the size of the convolution kernel, and s denotes the step size. The dilated convolution can effectively expand the range of the convolution kernel, capture more details and global features in the IMU signal, and provide more rich inertial feature information for the subsequent fusion network. The features extracted by the dilated convolution are input into the LSTM for further feature extraction, and finally the inertial features x i .

[0047] B. Visual feature extraction network

[0048] Image data usually contains rich and effective feature information, but if only two adjacent images are simply spliced and then feature extraction is performed, some key information may be lost. Therefore, the channel-spatial attention mechanism is introduced in the visual feature extraction network. Convolutional Block Attention Module (CBAM) is a representative method in the channel-spatial attention mechanism, which has the advantages of lightweight and strong generalization ability. CBAM generates attention feature maps in the channel and spatial dimensions to enhance the network's attention to key information. The two feature maps are finally multiplied with the original input feature map to realize adaptive feature correction and generate the final feature map. The specific process is as follows:

[0049] As shown in Figure 2 , a feature map F is first generated by a convolution block, F ∈ R C×H×W , R C×H×W denotes the output of the convolution block, C denotes the channel dimension, H denotes the height, and W denotes the width. Then, the channel attention module is sequentially passed through the channel dimension to generate the channel attention feature F', and the spatial attention module is sequentially passed through the spatial dimension to generate the spatial attention feature F". The specific process is as follows:

[0050]

[0051] wherein M c denotes a 1-D channel attention feature map, M c ∈ R C×1×1 , M s denotes a 2-D spatial attention feature map, and M s ∈ R 1×H×W . Specifically, M c can be written as:

[0052] M c (F) = σ(MLP(AvgPool(F)) + MLP(MaxPool(F)))

[0053] Among them, AvgPool and MaxPool represent global average pooling and maximum pooling operations respectively, MLP represents multi-layer perceptron, and σ represents Sigmoid activation function.

[0054] M s It can be written as:

[0055] M s (F′)=σ(f n×n ([AvgPool(F′);MaxPool(F′)]))

[0056] where f n×n Represents an n×n convolution operation.

[0057] In this process, element-level multiplication involves dimensional transformation and matching of feature maps. This mechanism effectively enhances the network’s sensitivity to important features, providing richer and more reliable information for subsequent navigation tasks, and ultimately obtaining the image feature x v .

[0058] C. Policy Network

[0059] Compared with inertial data, image data contains richer feature information. Therefore, the image features extracted from a set of data are far more than the inertial features. If all the extracted feature information is used for state estimation, the system computational burden will increase dramatically. Therefore, this embodiment introduces a policy network to disable visual information without significantly affecting the accuracy of posture estimation. This policy network uses an adaptive mechanism to learn how to make binary decisions to determine whether visual information is necessary. This decision process is sampled from a Bernoulli distribution, and its probability p is calculated by the policy network Φ based on the current inertial feature. and the hidden vector h of the previous time step t-1 To perform the calculation:

[0060]

[0061] In order to enable the system to be trained end-to-end, the probability p is discretely sampled by reparameterizing the Gumbel-Softmax through discrete distribution to obtain the binary parameter d t :

[0062] dt=Gumbel(p)

[0063] If d t =1, then the image feature is enabled If d t =0, then the image feature is disabled. Replace image features with zero padding.

[0064] D. Converged Network

[0065] The main function of the fusion network is to effectively fuse the previously extracted visual features with the inertial features determined by the policy network. The spatial dimension of the data carries key position information. The existing attention mechanism often cannot make full use of this spatial information, or it needs to reduce the channel dimension when utilizing spatial information, thereby affecting the representation ability of the features. To solve this problem, this embodiment introduces an efficient local attention mechanism (ELA) in the fusion network, which combines one-dimensional convolution and group normalization (GN) feature enhancement technology. First, stripe pooling (SP) is used to extract the spatial features x of the spliced ​​visual features and inertial features. c , capturing long-range spatial dependencies while maintaining the information integrity of the channel dimension. Then, average pooling is performed in the horizontal and vertical directions respectively, obtaining representations in two directions, namely:

[0066]

[0067] After the above steps, the fusion network is able to capture long-range spatial features while maintaining the information integrity of the channel dimension. In order to effectively utilize these features, one-dimensional convolution is applied to the position information in both horizontal and vertical directions to enhance its information, and group normalization GN is used to process the position information that needs to be enhanced. The position attention in the horizontal and vertical directions can be expressed as:

[0068] y h =σ(G n (F h (z h )))

[0069] y w =σ(G n (F w (z w )))

[0070] Among them, σ represents the nonlinear activation function, F h and F w represents one-dimensional convolution, G n Represents group normalization, and finally the output of the network is obtained, that is, the final fusion feature:

[0071] z t =x c *y h *y w

[0072] This method can accurately locate the region of interest without dimensionality reduction, and the network has the advantages of being lightweight, fast, and having strong generalization capabilities. The fusion network fuses inertial features with image features through the above-mentioned ELA efficient local attention mechanism to obtain fused features;

[0073]

[0074] in, Indicates the ELA fusion operation.

[0075] Finally, the fused features are input into LSTM, and the data output containing 6-dimensional navigation information is obtained through regression estimation of the fully connected layer:

[0076]

[0077] in, represents the estimated translation vector at time t, Represents the estimated rotation vector at time t.

[0078] The above-mentioned visual-inertial fusion navigation and positioning model based on visual information selection is trained through self-supervised learning. Since the errors of the inertial measurement unit are often accumulated over time, the changes in speed, attitude and position measured in a short time interval are usually more accurate. Therefore, during the network training process, the attitude and position changes measured by the IMU in the current two adjacent image time periods are set to true values. This processing method provides a more reliable reference benchmark for the overall network architecture, so that the model can be optimized based on relatively accurate measurement results during the self-supervised learning training process, thereby improving the overall navigation accuracy of the system, and also enabling the system to adjust and compensate more effectively when facing long-term accumulated errors. Therefore, the root mean square error (RMSE) of position and attitude is used to define the loss function L during network training. pose :

[0079]

[0080] Where T represents the length of the training sequence, and Denote the estimated translation vectors at time t and time t-1, Δv t i Represents the translation change measured by the inertial measurement unit within a time interval, and Denote the estimated rotation vectors at time t and time t-1, Δφ t i It represents the attitude change measured by the inertial measurement unit within a time interval, and α represents the weight of balancing the translation error and the rotation error.

[0081] This method is suitable for scenarios with limited computing power. Its end-to-end network structure effectively avoids the complex calibration and error modeling required in traditional visual-inertial fusion navigation and positioning. Its unsupervised learning approach avoids the difficulty of obtaining true values ​​in practical engineering applications. Furthermore, by reducing the computational cost of high-dimensional visual features, this method can significantly improve navigation efficiency.

[0082] To verify the effectiveness of the above method, this example first uses the public KITTI dataset to validate the model. Specifically, the data with sequence numbers 01, 02, 04, 05, 06, 07, and 08 in the dataset are used as the training set, and the data with sequence numbers 09 and 10 are used as the test set to verify the effectiveness of the proposed method. Figure 4 The comparison between the navigation results of the proposed method in the sequence 10 data set and the true value is shown. It can be seen that the navigation accuracy of the proposed method is higher.

[0083] Secondly, the proposed method is compared with two mainstream vision and IMU fusion methods (tightly coupled nonlinear VINS-Mono and deep learning DeepVIO) in terms of accuracy and computational consumption, respectively for the average translation error T rel , average rotation error R rel The three parameters of visual feature utilization are compared, and the comparison results are shown in Table 1.

[0084] Table 1 Comparison of the effects of the method of the present invention and the existing method

[0085] Method <![CDATA[T rel ]]> <![CDATA[R rel ]]> Visual feature utilization VINS-Mono 16.50 0.02 100％ DeepVIO 0.85 0.01 100％ The method of the invention 3.59 0.01 25.76％

[0086] By comparing Table 1, we can find that the method proposed in this invention reduces the utilization rate of visual features by 74.24% compared with the two mainstream methods. Therefore, this method can significantly reduce the amount of calculation required by the system. Compared with VINS-Mono, the average translation error T rel Reduced by 1 / 5, the average rotation error R rel Reduced by 1 / 2; Compared with the DeepVIO method, the method proposed in this invention has an average translation error T rel Increased by 4 times, the average rotation error R rel The above comparative experiments prove that the method proposed in the present invention can significantly reduce the amount of calculation while maintaining a certain navigation accuracy, and can meet the actual needs of engineering.

[0087] If the above method is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the part of the prior art that essentially contributes or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0088] The above description of the embodiments is to facilitate the understanding and use of the application by those skilled in the art. Those skilled in the art can easily make various modifications to these embodiments, and apply the general principles described herein to other embodiments without having to go through creative labor. Therefore, the present application is not limited to the above embodiments, and any improvements and modifications made by those skilled in the art based on the disclosure of the present application without departing from the scope of the present application should be within the scope of protection of the present application.

Claims

1. A visual-inertial fusion navigation and positioning method based on visual information selection, characterized in that: The following steps are involved: Obtain IMU data and two adjacent frames of image data, input the visual-inertial fusion navigation and positioning model selected based on visual information, and output the navigation and positioning results; Among them, the visual-inertial fusion navigation and positioning model based on visual information selection includes an IMU feature extraction network, a visual feature extraction network, a policy network and a fusion network. The IMU feature extraction network is constructed based on an LSTM that introduces dilated convolution, and is used to extract inertial features from IMU data; the visual feature extraction network is constructed based on a CNN that introduces a channel-spatial attention mechanism, and is used to extract image features from two adjacent frames of image data; the policy network is used to determine whether to disable the image feature. If so, all elements in the image feature are replaced with 0 and then input into the fusion network; otherwise, it is directly input into the fusion network; the fusion network is constructed based on an LSTM that introduces an efficient local attention mechanism, and is used to fuse the inertial features with the image features output by the policy network to obtain a navigation and positioning result; The policy network determines whether to disable the image feature through binary decision making. The specific process is as follows: Calculate the probability p based on the current inertial feature and the previous hidden vector; Discretely sample the probability p to obtain the binary parameter d t , if d t =1, it is determined that the image feature is enabled, if d t =0, it is determined that the image feature is disabled; Reparameterize Gumbel-Softmax via a discrete distribution to discretely sample probability p.

2. The visual-inertial fusion navigation and positioning method based on visual information selection according to claim 1 is characterized in that: In the IMU feature extraction network, the features of the IMU data are initially extracted by dilated convolution, and then input into LSTM for further feature extraction to obtain the inertial features. The dilated convolution is to add an interval inside the convolution kernel of the CNN.

3. The visual-inertial fusion navigation and positioning method based on visual information selection according to claim 2 is characterized in that: The receptive field RF of the current layer of the dilated convolution i Specifically: RF i =RF i-1 +(k-1)×s Among them, RF i-1 Represents the upper layer receptive field, k represents the size of the convolution kernel, and s represents the step size.

4. The visual-inertial fusion navigation and positioning method based on visual information selection according to claim 1, characterized in that: In the visual feature extraction network, the channel-spatial attention mechanism includes a channel attention module and a spatial attention module. The channel attention module is used to generate a channel attention feature F′ in the channel dimension according to the feature map F, and the spatial attention module is used to generate a spatial attention feature F″ in the spatial dimension according to the channel attention feature F′. The feature map F is generated by a convolution block based on two adjacent frames of image data, F∈R C×H×W , C represents the channel dimension, H represents the height, and W represents the width.

5. The visual-inertial fusion navigation and positioning method based on visual information selection according to claim 4 is characterized in that: The specific process of generating the channel attention feature F′ is as follows: Among them, M c represents the 1-D channel attention feature map, M c ∈R C×1×1 .

6. The visual-inertial fusion navigation and positioning method based on visual information selection according to claim 4 is characterized in that: The specific process of generating the spatial attention feature F" is as follows: Among them, M s represents the 2-D spatial attention feature map, M s ∈R 1×H×W .

7. The visual-inertial fusion navigation and positioning method based on visual information selection according to claim 1, characterized in that: The fusion network first fuses the inertial features with the image features output by the strategy network through an efficient local attention mechanism to obtain fusion features; then the fusion features are input into the LSTM and the navigation positioning results are obtained through full connection layer regression estimation.

8. The visual-inertial fusion navigation and positioning method based on visual information selection according to claim 1 is characterized in that: The visual inertial fusion navigation and positioning model based on visual information selection is trained by self-supervised learning. The loss function L used in the training process is pose The root mean square error based on position and attitude is constructed as follows: Where T represents the length of the training sequence, and Denote the estimated translation vectors at time t and time t-1, Δv t i Represents the translation change measured by the inertial measurement unit within a time interval, and Represent the estimated rotation vectors at time t and time t-1, Δφ t i It represents the attitude change measured by the inertial measurement unit within a time interval, and α represents the weight of balancing the translation error and the rotation error.

Images