An underwater visual-inertial odometry method based on dynamic sparse attention

Abstract

This invention discloses an underwater visual-inertial odometry method based on dynamic sparse attention, belonging to the fields of computer vision and robot navigation. An initial visual feature sequence is obtained from two adjacent underwater images. The dark channel map of the underwater image is calculated and encoded as a transmittance-aware embedding. The transmittance-aware embedding is fused with the position encoding to form a 3D position embedding. The 3D position embedding is added to the initial visual feature sequence and input into a visual encoder composed of N stacked dynamic sparse attention modules, outputting a visual feature sequence. An inertial feature sequence is introduced, and a bidirectional cross-attention mechanism is used to achieve deep interaction and alignment between visual and inertial features. An adaptive gating mechanism is used for weighted fusion, and finally, a regression network is used to predict the six-DOF pose. This invention effectively improves the robustness and accuracy of pose estimation in complex underwater environments by introducing physically prior-guided 3D position embedding, dynamic sparse attention, and deep adaptive fusion.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision and robot navigation, and specifically relates to an underwater visual inertial odometry method based on dynamic sparse attention. Background Technology

[0002] With the increasing demand for marine resource development and scientific exploration, autonomous underwater vehicles (AUVs) play an irreplaceable role in tasks such as seabed mapping, pipeline inspection, and deep-sea search and rescue. Accurate attitude estimation is crucial for the navigation of AUVs and remotely operated vehicles (ROVs). Due to the rapid attenuation of electromagnetic waves in water, global navigation satellite systems (GNSS) become completely ineffective underwater. While traditional acoustic positioning systems are effective, they often suffer from limitations such as large size, high cost, and low update frequency. In contrast, visual inertial odometry (VIO) has become a research hotspot in the field of underwater robot navigation due to its low cost and convenience. VIOMA is an autonomous, real-time, six-degree-of-freedom pose estimation technology that integrates multi-source heterogeneous sensing data from visual sensors and inertial measurement units.

[0003] While visual inertial odometry has achieved remarkable results in terrestrial and aerial scenarios (such as VINS-Mono and ORB-SLAM3), directly transferring it to underwater environments still faces significant challenges. First, the unique optical characteristics of underwater environments (such as light attenuation and scattering) result in low image contrast, color distortion, and severe fogging effects, which greatly damage the gradient information of the image, making it difficult for geometric methods based on optical flow or feature points to extract stable features. Second, underwater environments often exhibit weak textures or dynamic disturbances (such as bubbles and suspended particles), leading to frequent feature tracking failures or mismatches at the visual front end, resulting in severe drift or even system divergence. To address the limitations of manually designed features, deep learning-based methods have emerged. However, while data-driven methods based on deep learning alleviate the limitations of manual feature design to some extent, existing optical flow-based convolutional neural network architectures often focus on local textures, making it difficult to capture global contextual information in weakly textured underwater regions. Furthermore, most of them ignore the physical priors of underwater imaging (such as transmittance distribution), resulting in insufficient generalization ability of the models in complex waters.

[0004] Even with the introduction of the Transformer architecture, which demonstrates strong global modeling capabilities in many visual tasks, its standard form faces new challenges underwater. For example, while some studies have applied Dark Channel Prior (DCP), which reflects the relative depth and transmittance of a scene, to underwater visual tasks, existing works mostly treat it as an image preprocessing step or an auxiliary loss function, failing to encode its depth and organically integrate it into the network's main learning process, thus limiting the guiding effectiveness of the physical prior on model perception. Furthermore, the Transformer's core global self-attention mechanism, in underwater scenes with sparse features and redundant backgrounds, indiscriminately calculates relationships between all regions, introducing a significant amount of noise interference.

[0005] The key to designing a visual-inertial odometry method that can effectively overcome underwater optical degradation, feature sparsity, and noise interference lies in how to integrate underwater physical prior depth into feature encoding and how to make the attention mechanism adaptively focus on key areas, thereby achieving robust and accurate fusion and pose estimation of visual and inertial information in complex underwater environments. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides an underwater visual inertial odometry method based on dynamic sparse attention.

[0007] The solution adopted in this invention is as follows:

[0008] In a first aspect, the present invention proposes an underwater visual-inertial odometry method based on dynamic sparse attention, comprising the following steps:

[0009] S1. Obtain two adjacent underwater images and embed them into slices to obtain the initial visual feature sequence;

[0010] Dark channel maps are calculated from underwater images and encoded into transmittance-aware embeddings. Transmittance-aware embeddings are then fused with position encodings to form 3D position embeddings.

[0011] The 3D position embedding is added to the initial visual feature sequence and flattened into a one-dimensional vector. The input is a visual encoder consisting of N stacked dynamic sparse attention modules, and the output is the encoded visual feature sequence.

[0012] S2. Obtain the IMU data sequence corresponding to the two adjacent underwater images in time, input it into the causal dilated convolutional network based on the WaveNet architecture, and extract the inertial feature sequence.

[0013] S3. The encoded visual feature sequence and inertial feature sequence are interactively aligned through a bidirectional cross-attention mechanism to obtain aligned visual features and inertial features. Then, the fusion weights are dynamically generated through an adaptive gating mechanism. The aligned visual features and inertial features are weighted and summed according to the fusion weights to obtain the final fused features.

[0014] S4. Input the final fused features into the pose regression network to obtain the six-degree-of-freedom pose change of the current frame relative to the previous frame.

[0015] Furthermore, the calculation of the dark channel map based on the underwater image includes:

[0016] Obtain the input image composed of two adjacent underwater images stitched together;

[0017] For each pixel position in the input image, within its corresponding local window, the minimum value of each pixel in all its color channels is first taken to obtain a channel minimum value; then, the channel minimum value is taken again among all pixels in the local window to be used as the dark channel value at the pixel position.

[0018] By iterating through all pixel positions, a complete dark channel image is obtained.

[0019] Furthermore, the transmittance sensing embedding is obtained by downsampling and normalizing the dark channel map.

[0020] Furthermore, the fusion of transmittance sensing embedding and position encoding specifically involves:

[0021] The single channel of the transmittance sensing embedding is expanded, and the expanded transmittance sensing embedding is concatenated with the learnable row code and the learnable column code along the channel dimension. The channel dimension of the concatenated channel is the same as the channel dimension of the initial visual feature sequence.

[0022] Furthermore, the dynamic sparse attention module includes a DSA sublayer and a feedforward MLP sublayer, with layer normalization and residual connections applied before and after the sublayer.

[0023] Furthermore, the calculation process of the DSA sublayer includes:

[0024] The global context of the input features of the DSA sublayer is perceived by the gated network, and the sparsity of the current scene is predicted. At the same time, the input features of the DSA sublayer are spatially mapped in parallel to obtain the query vector and key value vector, and the initial dense attention matrix is ​​calculated.

[0025] The number of key vector connections to be retained for each query vector is determined based on the sparsity rate. In the initial dense attention matrix, only the score of the key vector with the highest similarity and the specified number of connections is retained, and the scores of the remaining connections are set to negative infinity to generate a sparse attention matrix.

[0026] The attention result is calculated based on the sparse attention matrix and used as the output feature of the DSA sublayer.

[0027] Furthermore, the gating network employs a two-layer perceptron.

[0028] Secondly, the present invention proposes a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the above-mentioned underwater visual inertial odometry method based on dynamic sparse attention.

[0029] Thirdly, the present invention proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described underwater visual inertial odometry method based on dynamic sparse attention.

[0030] Fourthly, the present invention provides a computer electronic device, including a memory and a processor;

[0031] The memory is used to store computer programs;

[0032] The processor is configured to implement the above-described underwater visual-inertial odometry method based on dynamic sparse attention when executing the computer program.

[0033] The beneficial effects of this invention are as follows:

[0034] (1) The present invention designs a transmission-aware 3D position embedding module. This module encodes the dark channel prior (DCP) information from underwater images into a transmittance-aware embedding and integrates it with learnable row and column position codes to form a novel 3D position embedding. It not only marks the spatial coordinates of features, but also explicitly carries the light intensity attenuation degree of different regions, thereby enhancing the network's physical perception ability of complex underwater optical degradation and scene geometry from the original data input level.

[0035] (2) To address the problems of sparse effective features and high background noise interference in underwater images, this invention designs a Dynamic Sparse Attention (DSA) mechanism. This mechanism replaces the standard global attention in the Transformer encoder, dynamically predicting the sparsity of the scene through a lightweight gating network, and adaptively retaining only the top-k connections most relevant to the key vector for each query vector. This strategy enables the network to actively suppress irrelevant background noise and focus limited computational resources on key regions, thereby significantly improving the robustness of visual features.

[0036] (3) This invention utilizes bidirectional cross-attention to achieve deep alignment of visual features and inertial features, and adaptively adjusts modal weights to fuse features through a gating mechanism, thereby achieving cross-modal contextual information complementarity. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the framework of an underwater visual-inertial odometry method based on dynamic sparse attention;

[0038] Figure 2 This is a flowchart illustrating the underwater visual-inertial odometry method based on dynamic sparse attention.

[0039] Figure 3 This is a schematic diagram of extracting visual feature sequences;

[0040] Figure 4 This is a schematic diagram of a dynamic sparse attention module;

[0041] Figure 5 This is a diagram showing the verification results of the present invention;

[0042] Figure 6 This is a framework diagram of an underwater visual inertial odometry system based on dynamic sparse attention.

[0043] Figure 7 This is a schematic diagram of the components of a computer electronic device. Detailed Implementation

[0044] The present invention will be further described and illustrated below with reference to specific embodiments. The embodiments described are merely examples of the content of this disclosure and do not limit the scope of the invention. The technical features of each embodiment in the present invention can be combined accordingly, provided that there is no mutual conflict.

[0045] The accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0046] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0047] like Figure 1 As shown, this invention proposes an underwater visual inertial odometry method based on dynamic sparse attention. First, a visual encoder processes two adjacent underwater images, introducing a transmittance-aware embedding based on dark channel priors. This embedding is then fused with row and column position encoding to form a 3D position embedding, enhancing the model's perception of underwater attenuation and geometric structures. Subsequently, a stacked dynamic sparse attention (DSA) module adaptively focuses on key regions, outputting a robust visual feature sequence. Simultaneously, a causal dilated convolutional network based on the WaveNet architecture processes synchronized IMU temporal data to extract inertial feature sequences. Next, a fusion module performs deep interaction between the features of the two modalities. This module employs a bidirectional cross-attention mechanism for feature alignment and utilizes an adaptive gating network for dynamic weighted fusion. Finally, the fused features are input into a regression network based on a multilayer perceptron (MLP), directly outputting the six-DOF pose change of the current frame relative to the previous frame.

[0048] like Figure 2 As shown, the main steps include:

[0049] S1. Obtain two adjacent underwater images and embed them into slices to obtain the initial visual feature sequence;

[0050] Dark channel maps are calculated from underwater images and encoded into transmittance-aware embeddings. Transmittance-aware embeddings are then fused with position encodings to form 3D position embeddings.

[0051] After adding the 3D location embedding to the initial visual feature sequence, the input is a visual encoder composed of N dynamic sparse attention modules stacked together, and the output is the encoded visual feature sequence.

[0052] S2. Obtain the IMU data sequence corresponding to the two adjacent underwater images in time, input it into the causal dilated convolutional network based on the WaveNet architecture, and extract the inertial feature sequence.

[0053] S3. The encoded visual feature sequence and inertial feature sequence are interactively aligned through a bidirectional cross-attention mechanism to obtain aligned visual features and inertial features. Then, the fusion weights are dynamically generated through an adaptive gating mechanism. The aligned visual features and inertial features are weighted and summed according to the fusion weights to obtain the final fused features.

[0054] S4. Input the final fused features into the pose regression network to obtain the six-degree-of-freedom pose change of the current frame relative to the previous frame.

[0055] The steps S1 to S4 above represent the steps of extracting visual feature sequences, extracting inertial feature sequences, fusing multimodal features, and pose regression, respectively. The specific implementation process of each step is described below.

[0056] In one specific embodiment of the present invention, in S1, the process of extracting the visual feature sequence is as follows: Figure 3 As shown, the details are as follows:

[0057] First, two adjacent underwater images are acquired, and the initial visual feature sequence is obtained by slicing and embedding.

[0058] The tensor formed by stitching together two adjacent frames is denoted as... H, W, and C represent the height, width, and number of channels of the underwater image, respectively. This tensor is first processed through non-overlapping slices and a linear embedding layer, and then projected onto an initial visual feature sequence. ,in , These represent the height and width of the spatial grid after the image is divided into patches, where P is the patch size and D is the feature embedding dimension.

[0059] Subsequently, a dark channel map is calculated based on the underwater image and encoded as a transmittance-aware embedding. The transmittance-aware embedding is then fused with the position encoding to form a 3D position embedding.

[0060] Calculate the dark channel map based on the color channels of the input image. Its intensity is highly correlated with transmittance and can reflect the spatial attenuation distribution of underwater scenes. The formula is as follows:

[0061]

[0062] in, For color channel set, It is a pixel A local window centered on y, where y is the local window The pixels in yes The minimum value of channel c corresponding to pixel y. As a representative of spatial attenuation distribution, it can effectively capture underwater optical degradation features. The dark channel image is downsampled to the same resolution as the slice feature and normalized to obtain a priori-guided transmittance-aware embedding. .

[0063] Learnable line encoding Column coding Embedded with the dark channel after projection By stitching along the channel dimension, a fused 3D position embedding is obtained. It contains both the spatial coordinates of pixels and explicitly carries the degree of light intensity attenuation in different areas, assisting the network in perceiving the 3D structure of the scene. The specific operation is as follows:

[0064]

[0065] in, For channel replication operations, extend a single channel to dimension.

[0066] Finally, the 3D location embedding is added to the initial visual feature sequence and then flattened. The input consists of a visual encoder composed of N dynamically sparse attention modules stacked together, and the output is the encoded visual feature sequence.

[0067] Flattened The process can be represented as:

[0068]

[0069] Where N is It is a dimension. ; It is a flattening operation.

[0070] To address the issues of sparse effective features and large background proportions in underwater images, this invention designs a Dynamic Sparse Attention (DSA) module as the core computational unit of the visual encoder. This module inherits the residual architecture of the standard Transformer module and replaces traditional multi-head self-attention with Dynamic Sparse Attention to achieve adaptive focusing on key geometric regions.

[0071] like Figure 4 As shown, each DSA block contains two cascaded sub-layers: a DSA sub-layer and a feedforward multilayer perceptron (MLP) sub-layer. Layer normalization (LN) and residual connections are used before and after each sub-layer to stabilize feature distribution, mitigate gradient vanishing, and preserve low-level information. For the ... The forward propagation of a DSA block is calculated using the following formula:

[0072]

[0073]

[0074] in, It is the first The output characteristics of each DSA block, when hour, ; It is layer normalization. It is a DSA sublayer. It is the first Output features of DSA sublayers in a DSA block It is a feedforward MLP sublayer.

[0075] In the DSA sublayer, input features are first sensed through a lightweight gating network. Given the global context, predict the sparsity of the current scene. The formula is as follows:

[0076]

[0077] in, For the Sigmoid function, It is the ReLU activation function. For learnable weights of fully connected layers, It is an average pooling layer used to extract global context information.

[0078] Based on sparsity Calculate the number of dynamically retained key connections. ( For the total number of features, (For flooring). Meanwhile, the DSA sublayer performs spatial mapping on the input features in parallel to obtain the underlying variables for attention computation. Specifically, the features normalized by the layer are... The query matrix is ​​generated through three independent linear mapping layers and a dimension reshaping operation. Key matrix Sum matrix When calculating the attention map, the query matrix Q is first multiplied by the transpose of the key matrix K, and then divided by a scaling factor. , Given the feature dimension of the key vector, we obtain a size of Initial dense attention score matrix Next, regarding this dense scoring matrix... ,definition For the sparse masking operator: for each query vector, only the attention scores of the k key vectors with the highest similarity to it are retained, which correspond to the sparse matrix in the graph. Set the attention scores for the remaining positions to negative infinity. This is to achieve an effective mask. The final output of the dynamic sparse attention is... The calculation formula is:

[0079]

[0080] This strategy directs computational resources to key areas with strong geometric consistency, such as rock edges and artificial structures, while automatically suppressing irrelevant background. By stacking n DSA blocks, the visual encoder can progressively optimize the input sequence into a robust visual feature sequence. .

[0081] In one specific embodiment of the present invention, the process of extracting the inertial feature sequence in S2 is as follows:

[0082] In the inertial measurement unit (IMU) feature extraction stage, to efficiently process high-frequency and noisy inertial measurement data, this invention employs a causal dilated convolutional network based on the WaveNet architecture. This structure exponentially expands the temporal receptive field through dilated convolution without increasing computational cost, thereby effectively capturing long-distance motion dependencies. Specifically, the IMU data sequence I corresponding temporally to two adjacent underwater image frames is obtained as input. The causal dilated convolutional network consists of four layers, and each layer's features are nonlinearly filtered using stacked gated activation units, ultimately generating an inertial feature sequence. .

[0083]

[0084] in, It is a causal dilated convolutional network based on the WaveNet architecture.

[0085] In one specific embodiment of the present invention, in S3, the process of fusing multimodal features is as follows:

[0086] This invention constructs a unified fusion module that integrates bidirectional cross-attention and adaptive gating mechanism. The bidirectional cross-attention mechanism is used to interactively align visual feature sequences and inertial feature sequences to obtain aligned visual features and inertial features. Subsequently, the adaptive gating mechanism dynamically generates fusion weights, and the aligned visual features and inertial features are weighted and summed according to the fusion weights to obtain the final fused features.

[0087] Specifically, symmetrical cross-attention is first deployed for feature interaction. Visual features are used in the image-guided IMU attention branch. As a query Q, inertial features As key K and value V; however, in the IMU-guided image attention branch, the roles are reversed, using inertial features. As query Q, visual features As the key K and value V, this bidirectional interaction process achieves implicit alignment between the visual frame and the inertial sequence.

[0088] Subsequently, a confidence-aware gated fusion module is deployed. The aligned global visual features are then... and aligned inertial characteristics The data is concatenated and input into a gating network to generate a weight vector. The final fusion features The following dynamic aggregation formula is used for calculation:

[0089]

[0090] in, This indicates dot product.

[0091] In one specific embodiment of the present invention, S4, the attitude regression process is as follows:

[0092] During the pose regression stage, features will be fused. The input is fed into a pose regression network, which directly regresses and generates a six-DOF pose change vector of the current frame relative to the previous frame, including three-dimensional displacement. and 3D rotation The calculation formula is as follows:

[0093]

[0094] in, It is a pose regression network. In this embodiment, a two-layer multilayer perceptron network is used, and the ReLU activation function is applied between the layers.

[0095] This invention uses the mean square error between the predicted pose and the true pose as the supervision index of the network, and the loss function is defined as follows:

[0096]

[0097] in, Indicates the batch size for training; and These represent actual translation and rotation, respectively. and This indicates the prediction result corresponding to the model, and the subscript i indicates the corresponding dimension.

[0098] Figure 5The performance of the method of this invention on a typical test sequence is demonstrated. In the figure, the blue trajectory represents the actual pose path, and the red trajectory represents the pose path estimated by the method of this invention. In this complex reciprocating motion scenario, the red trajectory predicted by the method of this invention is highly consistent with the actual blue trajectory in terms of overall path shape, and the two closely match. Especially in the turning sections where the direction of motion changes, the red trajectory can still accurately follow the changes of the blue trajectory without obvious drift or divergence. This result intuitively confirms that the method proposed in this invention can achieve continuous, stable, and high-precision pose estimation in complex motion modes, effectively tracking the actual motion trajectory of the system.

[0099] It should also be noted that the underwater visual-inertial odometry method based on dynamic sparse attention in the above embodiments can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides an underwater visual-inertial odometry system based on dynamic sparse attention, corresponding to the underwater visual-inertial odometry method based on dynamic sparse attention provided in the above embodiments, such as... Figure 6 As shown, it includes:

[0100] The visual feature extraction module is used to acquire two adjacent underwater images and obtain an initial visual feature sequence through slice embedding. The dark channel map is calculated based on the underwater image and encoded into a transmittance-aware embedding. The transmittance-aware embedding is fused with the position encoding to form a 3D position embedding. The 3D position embedding is added to the initial visual feature sequence and flattened into a one-dimensional vector. This vector is then input to a visual encoder composed of N dynamic sparse attention modules stacked together, and the encoded visual feature sequence is output.

[0101] The inertial feature extraction module is used to acquire the IMU data sequence corresponding to the two adjacent underwater images in time, input it into a causal dilated convolutional network based on the WaveNet architecture, and extract the inertial feature sequence.

[0102] The multimodal fusion module is used to interactively align the encoded visual feature sequence and the inertial feature sequence through a bidirectional cross-attention mechanism to obtain aligned visual and inertial features. Then, an adaptive gating mechanism is used to dynamically generate fusion weights. Based on the fusion weights, the aligned visual and inertial features are weighted and summed to obtain the final fused features.

[0103] The pose estimation module is used to input the final fused features into the pose regression network to obtain the six-degree-of-freedom pose change of the current frame relative to the previous frame.

[0104] It is understood that the underwater visual-inertial odometry method based on dynamic sparse attention in the above embodiments can essentially be implemented by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer program product corresponding to the underwater visual-inertial odometry method based on dynamic sparse attention provided in the above embodiments, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, they can implement the underwater visual-inertial odometry method based on dynamic sparse attention as described in the above embodiments.

[0105] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the underwater visual-inertial odometry method based on dynamic sparse attention provided in the above embodiments, such as... Figure 7 As shown, it includes a memory and a processor;

[0106] The memory is used to store computer programs;

[0107] The processor is configured to implement the underwater visual-inertial odometry method based on dynamic sparse attention in the above embodiments when executing the computer program.

[0108] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0109] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the underwater visual inertial odometry method based on dynamic sparse attention provided in the above embodiments. The storage medium stores a computer program, which, when executed by a processor, can implement the underwater visual inertial odometry method based on dynamic sparse attention in the above embodiments.

[0110] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0111] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0112] It should also be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the embodiments provided in this application, the division of steps or modules in the system and method is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or steps may be combined or integrated together, and a module or step may also be split.

[0113] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. An underwater visual-inertial odometry method based on dynamic sparse attention, characterized in that, Includes the following steps: S1. Acquire two adjacent underwater images, and denote the tensor formed by stitching the two adjacent images as follows: H, W, and C represent the height, width, and number of channels of the underwater image, respectively. The tensor is first divided into non-overlapping slices and a linear embedding layer, and then projected onto an initial visual feature sequence. ,in , These represent the height and width of the spatial grid after the image is sliced, where P is the slice size and D is the feature embedding dimension. Calculate the dark channel map based on the color channels of the input image. The formula is as follows: ； in, For color channel set, It is a pixel A local window centered on y, where y is the local window The pixels in yes The minimum value of channel c corresponding to pixel y; the dark channel image is downsampled to match the slice feature resolution and normalized to obtain the prior-guided transmittance-aware embedding. ; Learnable line encoding Column coding Embedded with the dark channel after projection By stitching along the channel dimension, a fused 3D position embedding is obtained. The formula is as follows: ； in, For channel copying operations, a single channel is expanded to D / 3 dimensions; After adding the 3D location embedding to the initial visual feature sequence, it is flattened as follows: The flattening formula is as follows: ； Where N is Dimensions ; It is a flattening operation; Will The input consists of a visual encoder composed of N dynamically sparse attention modules stacked together, and the output is the encoded visual feature sequence. S2. Obtain the IMU data sequence corresponding to the two adjacent underwater images in time, input it into the causal dilated convolutional network based on the WaveNet architecture, and extract the inertial feature sequence. S3. The encoded visual feature sequence and inertial feature sequence are interactively aligned through a bidirectional cross-attention mechanism to obtain aligned visual features and inertial features. Then, the fusion weights are dynamically generated through an adaptive gating mechanism. The aligned visual features and inertial features are weighted and summed according to the fusion weights to obtain the final fused features. S4. Input the final fused features into the pose regression network to obtain the six-degree-of-freedom pose change of the current frame relative to the previous frame.

2. The underwater visual-inertial odometry method based on dynamic sparse attention according to claim 1, characterized in that, The dynamic sparse attention module includes a DSA sublayer and a feedforward MLP sublayer, with layer normalization and residual connections applied before and after the sublayer.

3. The underwater visual-inertial odometry method based on dynamic sparse attention according to claim 2, characterized in that, The calculation process of the DSA sublayer includes: The global context of the input features of the DSA sublayer is perceived by the gated network, and the sparsity of the current scene is predicted. At the same time, the input features of the DSA sublayer are spatially mapped in parallel to obtain the query vector and key value vector, and the initial dense attention matrix is ​​calculated. The number of key vector connections to be retained for each query vector is determined based on the sparsity rate. In the initial dense attention matrix, only the score of the key vector with the highest similarity and the specified number of connections is retained, and the scores of the remaining connections are set to negative infinity to generate a sparse attention matrix. The attention result is calculated based on the sparse attention matrix and used as the output feature of the DSA sublayer.

4. The underwater visual-inertial odometry method based on dynamic sparse attention according to claim 3, characterized in that, The gating network employs a two-layer perceptron.

5. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they can implement the underwater visual inertial odometry method based on dynamic sparse attention as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the underwater visual inertial odometry method based on dynamic sparse attention as described in any one of claims 1 to 4.

7. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to implement, when executing the computer program, the underwater visual-inertial odometry method based on dynamic sparse attention as described in any one of claims 1 to 4.

Images