A double-flow end-to-end visual autonomous positioning method and system based on bio-inspired spatio-temporal reasoning

Abstract

This invention relates to a two-stream end-to-end visual autonomous localization method and system based on bio-inspired spatiotemporal reasoning, belonging to the field of computer vision technology. The method includes: acquiring an input image sequence containing adjacent frames; inputting the input image sequence into a dorsal visual pathway to extract and aggregate inter-frame motion features to obtain a high-dimensional motion feature vector; inputting the input image sequence into a ventral visual pathway to extract inter-frame spatial correlation features through adaptive temporal convolution and a visual state space module to obtain spatial semantic features; performing feature fusion and interaction through a gated cross-attention fusion module, using spatial semantic features as queries and high-dimensional motion feature vectors as keys and values; and estimating the relative pose between adjacent frames based on the fused features. This scheme achieves accurate and robust pose estimation without requiring camera intrinsics, striking a balance between inference speed, model complexity, and training cost, making it suitable for deployment in practical robot applications.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology and relates to a dual-stream end-to-end visual autonomous localization method and system based on bio-inspired spatiotemporal reasoning. Background Technology

[0002] Visual odometry (VO) enables real-time pose estimation without relying on external positioning infrastructure, forming a core support for visual SLAM systems and vision-inertial navigation systems. It is a key component of autonomous systems such as autonomous vehicles, drones, and augmented reality platforms. Monocular visual odometry has attracted widespread attention from researchers due to its low cost and ease of installation. Currently, existing visual odometry methods mainly include: Traditional geometric methods, such as ORB-SLAM and VINS-Mono, estimate camera motion by imposing geometric constraints between features matched in multiple views. While these methods offer high accuracy and interpretability, they rely on manual feature extraction and matching, resulting in insufficient stability in low-texture, varying lighting, or dynamic scenes. Blurred or inconsistent correspondences further degrade performance. To overcome these limitations, learning-based VO / SLAM has emerged as a promising research direction. Hybrid methods such as DPVO, DROID-SLAM, and MBRVO combine learning-based optical flow tracking with a differentiable beam adjustment (BA) backend, optimizing camera pose by minimizing reprojection errors and improving the robustness of feature extraction. However, these methods still rely on geometric assumptions regarding camera calibration, photometric consistency, and feature-level data association, making them susceptible to scale drift and calibration errors in unstructured or uncalibrated scenes.

[0003] End-to-end motion capture (VO) takes a more thorough approach: inferring six-DOF motion directly from the raw image sequence. Early works such as LVO, DeepVO, and TartanVO employed convolutional neural network and recurrent neural network architectures to capture temporal dependencies. In recent years, Transformer-based methods such as BotVIO, BrightVO, TSformerVO, SwinFormerVO, and STDN-VO have further improved estimation accuracy by modeling long-range spatiotemporal dependencies. However, these methods typically lack explicit motion structure inference and rely on general visual backbone networks, limiting their accuracy and stability compared to geometry-based or hybrid methods. While this paradigm eliminates the need for handcrafted components and prior calibration, fully data-driven VO still faces challenges: accurate pose estimation requires simultaneous modeling of spatial structure and temporal coherence. Therefore, most end-to-end models still lag behind geometric and hybrid methods in terms of accuracy and scale consistency.

[0004] However, existing visual odometry methods still require a trade-off between accuracy, efficiency, and generalization ability. Geometry-based methods have a solid theoretical foundation but are limited by fragile feature dependencies; hybrid methods improve robustness but still suffer from calibration sensitivity issues; and end-to-end methods are limited by weak spatiotemporal modeling capabilities. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a two-stream end-to-end visual autonomous localization method and system based on bio-inspired spatiotemporal reasoning, which achieves spatial and temporal reasoning through the fusion of two-stream interaction. This method can achieve accurate and robust pose estimation without camera intrinsic parameters, and strikes a balance between inference speed, model complexity, and training cost, making it suitable for deployment in practical robot application scenarios.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A two-stream end-to-end visual autonomous localization method based on bio-inspired spatiotemporal reasoning, the method specifically includes the following steps: S1: Obtain the input image sequence containing adjacent frames; S2: Input the input image sequence into the back-side visual pathway, extract and aggregate the motion features between frames to obtain a high-dimensional motion feature vector; S3: Input the input image sequence into the ventral visual pathway, and extract the spatial correlation features between frames through adaptive temporal convolution and the visual state space module to obtain spatial semantic features; S4: Through the gated cross-attention fusion module, feature fusion and interaction are performed using the spatial semantic features as queries and the high-dimensional motion feature vectors as keys and values; S5: Estimate the relative pose between the adjacent frames based on the fused features.

[0007] Furthermore, step S2 specifically includes: S21: The input image sequence is feature-mapped using a feature extractor to obtain visual feature maps and contextual features; S22: Construct a fully correlated volume based on the visual feature map to quantify the spatial dependencies between frames; S23: The fully correlated entity is fused with the context features and input into the minimum gated recurrent unit network for parallel temporal aggregation; S24: Perform a nonlinear transformation on the aggregated features to output the high-dimensional motion feature vector.

[0008] Furthermore, step S3 specifically includes: S31: Perform adaptive temporal convolution processing on the input image sequence, where the convolution weights are dynamically generated based on the global temporal context; S32: Input the adaptively convolutional processed features into a hierarchical architecture containing at least one visual state space module, capture long-range spatial dependencies with linear complexity through the state space mechanism, and output the spatial semantic features.

[0009] Furthermore, the weights of the adaptive temporal convolution are generated in the following way: 1) Calculate the time descriptor and global descriptor based on the input sequence; 2) The temporal descriptor and the global descriptor are mapped and fused using a network containing one-dimensional convolutions to generate dynamic calibration weights; 3) Combine the dynamically calibrated weights with the basic convolutional kernel to obtain the adaptive convolutional weights at the current time.

[0010] Furthermore, in step S4, the gated cross-attention fusion module performs the following operations: S41: Project the spatial semantic features into a query vector, and project the high-dimensional motion feature vector into a key vector and a value vector; S42: Calculate the cross-attention weights between the query vector and the key vector, and apply them to the value vector to obtain attention features; S43: Generate gate weights through a channel-level gating network, weight the attention features with the gate weights, and perform residual fusion with the original spatial semantic features.

[0011] Furthermore, the feature extractor employs a residual network architecture, which includes cascaded residual blocks, and uses 1×1 convolutions in the downsampling branch to adjust the dimension of the residual connections.

[0012] Furthermore, in step S23, the update process of the minimum gated recurrent unit network is simplified as follows: Where Z is the update gate. Here, is the sigmoid activation function, Linear represents a fully connected layer with output dimension d, x is the input at the current time step, and h is the candidate hidden state.

[0013] Furthermore, prior to step S1, a step of data augmentation of the input image sequence is included, wherein the data augmentation includes color jitter and random erasure.

[0014] The present invention also provides a dual-stream end-to-end visual autonomous localization system based on bio-inspired spatiotemporal reasoning, which employs the method described above.

[0015] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0016] The beneficial effects of this invention are as follows: Compared to existing learning-based visual odometry methods, the proposed method achieves accurate and drift-resistant pose estimation while maintaining excellent model size and inference speed. Specifically: 1. An adaptive ventral pathway based on Mamba is designed, capturing coarse-grained structure and temporal dependencies through linear complexity computation and dynamic temporal convolution, achieving efficient long-range motion modeling. 2. A guided back-side pathway based on MinGRU is designed, extracting high-frequency motion cues through parallel spatial scanning and linear spatial memory, and employing a gated cross-attention fusion mechanism to guide the ventral pathway in spatial feature aggregation.

[0017] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0018] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a system architecture diagram of the present invention; Figure 2 This is the rear-side visual flow path of the present invention; Figure 3 This is the ventral visual access path of the present invention; Figure 4 A schematic diagram to compare the differences in inference time, absolute trajectory error (ATE), and parameters of the VO method on the KITTI dataset; Figure 5 This is a schematic diagram illustrating the ablation study of the contributions of DVS and VVS to VO performance on the KITTI dataset. Detailed Implementation

[0019] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0020] This invention provides a two-stream end-to-end visual autonomous localization method based on bio-inspired spatiotemporal reasoning, comprising the following steps: Step S1: Using RGB images of two adjacent frames as input data, and simultaneously employing basic data augmentation strategies. Step S2: Obtaining high-dimensional motion features from the data via the dorsal visual pathway. Step S3: Obtaining data association features such as shape and color consistency between adjacent frames via the ventral visual pathway. Step S4: Motion features guide the ventral stream to learn spatial features, and finally, the two streams fuse the features for pose estimation.

[0021] Furthermore, after acquiring the enhanced image sequence, the present invention first extracts its features, then calculates visual similarity, and further aggregates motion features through a minGRU network. By quantifying the inter-frame spatial dependence through the correlation between feature points, it provides a basis for motion association.

[0022] Furthermore, after acquiring the enhanced image sequence, this invention employs adaptive temporal convolution to generate dynamic calibration weights based on the spatiotemporal descriptors of adjacent frames. The calibrated features are then fed into a four-level VSS module. Through a state-space mechanism, long-range spatial dependencies are captured with linear complexity, and spatial semantic features such as texture and shape are encoded hierarchically.

[0023] Finally, through the gated cross-attention (GCA) module, using ventral spatial features as queries and dorsal motion features as keys, the fusion intensity is dynamically adjusted through channel-level gating to achieve collaborative optimization of motion and spatial features.

[0024] Figure 1 The system architecture diagram of the present invention is shown. The framework takes an image sequence as input and first sends it to the basic data augmentation module: color jitter and random erasing are used to simulate photometric distortion (such as changes in illumination) and occlusion (such as moving pedestrians and vehicles), respectively, thereby improving data diversity and model robustness.

[0025] Figure 2 The dorsal visual pathway of this invention focuses on high-frequency motion cues, taking image sequences as input and outputting high-dimensional motion features. Specifically, it first obtains temporal feature maps and contextual information through a feature extractor, then calculates the visual similarity between feature maps, while further expanding the receptive field of the contextual features; the fusion of these two types of information enables data association between image sequences, and then the aggregated features are processed to generate feature vectors representing high-frequency motion, which implicitly encode high-dimensional motion cues.

[0026] Figure 3The ventral visual stream of this invention aims to aggregate spatial dimensional information. We introduce an adaptive temporal convolution mechanism to dynamically capture inter-frame correspondences. Unlike traditional static convolution kernels, this mechanism recalibrates kernel weights based on inter-frame correlations, enabling joint modeling of local motion patterns and long-range temporal dependencies. Subsequently, a four-scale visual state space (VSS) module is used to learn visual representations of the spatial dimensions, extracting data association features such as shape and color consistency between adjacent frames. This design enhances the context-aware spatial association capability, thereby improving pose estimation accuracy.

[0027] Subsequently, the dual-stream outputs are fused by combining statistical patterns representing inter-frame spatiotemporal patterns: all information is integrated and camera motion is output through a multi-layer perceptron (MLP), ultimately achieving accurate visual odometry estimation. This dual-stream design fully utilizes the complementary capabilities of the two visual pathways, combining data augmentation and statistical modeling to achieve robust and accurate motion estimation.

[0028] The following is a detailed explanation of this plan.

[0029] 1. Dorsal visual pathway 1.1 Feature Extraction: To achieve image sequence... To address the end-to-end feature mapping, this invention designs a feature extraction network architecture with residual connections at its core. This architecture effectively captures multi-level feature representations while meeting the requirements of lightweight model and computational efficiency. The same feature extractor is used for extracting the visual feature map F and the contextual information ctx, and its mathematical expression is as follows: in The weight parameters of the extractor are represented. The initial feature extraction layer uses 3×3 convolutional kernels as the main feature extraction units; the residual feature enhancement layer consists of two cascaded residual blocks, enabling hierarchical feature extraction and contextual information fusion. Specifically, layer 1 focuses on refined feature extraction, while layer 2 performs secondary downsampling to further expand the receptive field, thereby capturing a wider range of contextual information. Each residual block uses a 1×1 convolution in its downsampling branch to adjust the dimension of the residual connections, effectively mitigating the gradient vanishing problem while preserving feature integrity during deep network training.

[0030] 1.2 Visual Similarity and Feature Aggregation To establish inter-frame spatial dependencies, this invention employs the core components of RAFT to construct a full correlation volume to calculate the similarity between feature maps of each pair. Taking image pair I1 and I2 as an example, their corresponding feature maps are represented as FI1 and FI2, respectively, and their corresponding correlation volume C. This correlation volume, together with contextual features, promotes the aggregation of motion cues and guides the ventral pathway to learn spatial structure.

[0031] To efficiently capture temporal dependencies while supporting parallel computing, this invention employs a minimal gated recurrent unit (minGRU). By removing hidden state dependencies from the input, forget, and update gates, minGRU eliminates the need for backpropagation through time (BPTT) and supports parallel scan computation. The input sequence and The output sequence can be obtained through parallel computation. .

[0032] This simplification process removes both the reset gate and the tanh activation function from the candidate hidden states, ultimately resulting in: Where Z is the update gate. Here, is the sigmoid activation function, Linear represents a fully connected layer with output dimension d, x is the input at the current time step, and h is the candidate hidden state.

[0033] Following minGRU, two convolutional layers with 3×3 kernels and ReLU activation functions are used to introduce nonlinearity, and finally, a fully connected layer is used to output high-dimensional motion vector features. This design achieves efficient, parallel-time modeling while preserving the rich motion information required by the dorsal pathway, which is crucial for the accuracy of motion estimation.

[0034] 2. Ventral visual pathway The ventral visual pathway focuses on capturing spatial and semantic cues from sequential frames. Unlike previous methods that use shared convolutional weights across all spatial locations and temporal frames, this invention introduces adaptive spatiotemporal modeling, which dynamically adjusts the convolutional kernels based on the scene context. This design enables the model's feature extraction to better adapt to dynamic appearance changes and motion parallax, thereby improving data association and overall pose estimation accuracy.

[0035] like Figure 3As shown, this invention incorporates two core innovations: 1) adaptive temporal convolution, which dynamically calibrates convolution weights based on temporal and global contexts; and 2) a visual state space (VSS) module, which models spatial dependencies through multi-scale hierarchical modeling. Finally, a gated cross-attention fusion module enables interaction between ventral and dorsal pathways, allowing high-level spatial semantics to be optimized through motion-aware cues.

[0036] 2.1 Adaptive Temporal Convolution Given an input sequence, the convolution weights at time t are adaptively adjusted in the following way: Where Wb is the basic convolution kernel. Learnable calibration weights are generated based on contextual cues. This dynamic adjustment enables the convolution to adapt to scene dynamics such as changes in lighting, occlusion, and appearance. For each frame, a time descriptor is computed using global average pooling. (T is the time step, C is the number of channels); then calibration weights are generated through a two-layer 1D convolutional network. The process is as follows: Where Act uses the ReLU activation function, BN represents batch normalization, and C / r is the number of intermediate channels in the 1D convolution (r is the channel compression ratio). To incorporate the global temporal context, a global descriptor g is obtained through spatiotemporal average pooling, and fusion is achieved through fully connected mapping. The calibration weight expression is then: To ensure training stability, adaptive convolution is initialized with an identity operation equivalent to standard convolution, achieved through residual calibration. In the initial state At this point, the adaptive convolution degenerates into a standard convolution, avoiding the non-convergence problem caused by sudden changes in weights in the early stages of training.

[0037] 2.2 Visual State Space Modeling The features F, after adaptive convolution, are fed into a hierarchical architecture consisting of four visual state space (VSS) modules to model spatial dependencies at multiple scales. Where X(s) is the output feature map at scale s, h(s) is the latent state at scale s, and X(0)=F is the initial input feature.

[0038] Each VSS module captures selective space dependencies through a state-space mechanism: Where A is the state transition matrix (controlling the evolution of the latent state), B is the input influence matrix (regulating the effect of inputs on the latent state), C is the output mapping matrix (realizing the projection from the latent state to the output), and D is the residual mapping matrix. This state-space design achieves efficient long-range spatial modeling with linear complexity, which aligns with the biological initiation mechanism of the ventral pathway, which "captures target-centered spatial semantics."

[0039] 2.3 Gated Cross-Attention Fusion To achieve effective interaction between the ventral and dorsal visual pathways, this invention proposes a gated cross-attention (GCA) module. This module utilizes ventral feature maps... (B is the batch size, H and W are the feature map dimensions) As a query, the backside feature map It serves as both a key and a value. This design enables the ventral representation to adaptively integrate motion and contextual cues from the dorsal pathway.

[0040] First, the features are projected into query, key, and value embeddings using a 1×1 convolution: in , and These are the projection weight matrices for the query, key, and value, respectively. Then, the cross-attention weights are calculated: Where d is used to scale the attention score to avoid gradient vanishing, and the attention features are... .

[0041] To stabilize training and prevent the model from overfitting to instantaneous features, a channel-level gating mechanism is introduced to adaptively adjust the fusion intensity: Where G is the gated network (the gate weights are calculated by global pooling followed by convolutional layers). is the learnable residual scaling factor (initialized to 0.1, controlling the contribution magnitude of attention features).

[0042] This gating design enables the ventral pathway to selectively fuse spatial alignment motion information from the dorsal pathway, enhancing the modeling accuracy of spatiotemporal correspondences while maintaining semantic consistency.

[0043] The technical effects of the present invention are verified through the following embodiments.

[0044] This invention presents a bio-inspired spatiotemporal reasoning-based two-stream visual autonomous localization method. It bypasses explicit geometric modeling while considering inter-frame temporal and spatial correlations and the guidance between the two streams, achieving accurate pose estimation while balancing parameter quantity, inference time, and even training cost. Here, the proposed method is evaluated using the publicly available autonomous driving datasets KITTI and 4Seasons. The evaluation includes the following comparisons: All experiments were conducted on a workstation equipped with an Intel Core i7-13700K CPU and an NVIDIA RTX 4090D GPU (24 GB VRAM). The proposed network was implemented using the PyTorch framework and trained for 200 epochs with the Adam optimizer (default parameters) at a learning rate of 1×10⁻⁵ and a batch size of 8. The Visual State Space (VSS) module was initialized using the weights of a VMamba model pre-trained on the ImageNet-1K dataset.

[0045] All input images were resized to 224×224 pixels and normalized using the pre-computed mean and variance from the training set to stabilize the optimization process. The network was trained using supervised learning, and the loss function was the mean squared error (MSE) between the predicted relative pose and the true relative pose (ground-truth), as shown in the following formula: Where B represents the batch size. and Let i and ii represent the true 6-DOF (6-DOF) relative pose and the i-th dimension of the predicted 6-DOF relative pose of the b-th sample, respectively.

[0046] On the KITTI dataset, this invention compares the proposed method with three main classes of visual odometry (VO) methods: first, with state-of-the-art end-to-end supervised VO methods, including DeepVO, TSformer-VO, SWformer-VO, and STDN-VO; second, with the geometry-based method ORB-SLAM3; and third, with hybrid VO methods (such as DROID-SLAM, TartanVO, DPVO, XVO, and MambaVO); and finally, with representative unsupervised methods. These methods achieve VO estimation by jointly learning deep learning and self-motion, including SfMLearner, Deep-VO-Feat, GeoNet, Bian et al., MaskedGANs, Wang et al., DFR, and NeuralBundler.

[0047] The method of this invention is compared with geometry-based VO methods and supervised learning VO methods on KITTI test sequences. Table 1 summarizes the absolute trajectory error (ATE) of all methods. Table 2 shows the absolute trajectory error (ATE) of geometry-based, hybrid, and end-to-end VO methods on the KITTI dataset. The method of this invention outperforms all benchmark methods, achieving the lowest mean ATE (14.55m) and standard deviation (7.1m), indicating that it has better accuracy and robustness in diverse driving scenarios.

[0048] Table 1. Comparison of the present invention with different types of methods, ATE, on the KITTI dataset. Table 2 compares the results of this invention with geometry-based, hybrid, and end-to-end methods on the KITTI dataset for relative translation error Terr and relative rotation error Rerr. Among the geometry- and hybrid methods, ORB-SLAM3 achieved the lowest error on sequences 03 and 10, but the error increased significantly on sequences 05 and 06, resulting in a high overall variance (18.69m). TartanVO performed the most balanced in this group, ranking second in average ATE (15.50m). DPVO, DROID-VO, MambaVO, and XVO had larger errors and weaker stability, with XVO performing the worst despite using camera intrinsics. For other end-to-end VO benchmark methods, DeepVO performed poorly with a significantly high ATE value; TSformer-VO, SWformer-VO, and STDN-VO achieved moderate levels of accuracy and stability. Overall, the method of this invention not only surpasses all benchmark methods in accuracy but also exhibits the most stable performance across sequences, highlighting its effectiveness in real-world visual odometry scenarios. Monocular VO inherently suffers from scale ambiguity, making translation estimation particularly sensitive to often unreliable far points. This also explains the significant translational errors observed in sequences 05 and 06 by the geometry-based and hybrid VO methods (see Table 2). These errors manifest as noticeable scale drift on the trajectory. In contrast, rotation estimation is largely unaffected by scene depth: methods such as TartanVO, which rely on dense optical flow, achieve lower rotational errors despite scale ambiguity.

[0049] The method of this invention effectively alleviates the translation drift problem, reducing the average translation error Terr by 28% compared to DPVO, while maintaining competitive rotational accuracy. Specifically, compared to the current state-of-the-art STDN-VO, the method of this invention reduces the translation error Terr and rotational error Rerr by 24.6% and 28.4%, respectively. This demonstrates that even without introducing explicit geometric priors, the method of this invention can robustly constrain the trajectory scale while achieving balanced performance in translation and rotation estimation.

[0050] Figure 4 This paper demonstrates the trade-offs between absolute trajectory error (ATE), inference time, and model size in different VO methods. The present invention achieves an optimal balance between accuracy, speed, and parameter efficiency, outperforming both geometry-based and learning-based methods. While geometry-based methods (such as ORB-SLAM3, DPVO, and DROID-VO) offer advantages in speed and lightweight design, they rely on camera intrinsics and exhibit lower accuracy in complex scenes. Transformer-based end-to-end methods, while achieving competitive accuracy, typically have longer inference times and larger model sizes. Overall, the method presented in this invention achieves an optimal trade-off, making it highly suitable for real-time VO applications.

[0051] On KITTI sequences 09 and 10, this invention is compared with representative unsupervised VO methods; to ensure fairness, the results of all comparison methods are directly cited from their original papers (see Table 3). The method of this invention achieves the lowest average translation error (6.88%), a 34.4% improvement over the previously best-performing NeuralBundler, despite the latter's additional pose graph optimization module. Regarding rotation error, the average error of this invention is 3.015° / 100m, a slight improvement over NeuralBundler. Notably, the method of this invention consistently outperforms other methods on both sequences, especially in complex scenarios where traditional methods and previous learning-based methods struggle, significantly reducing estimation errors. These results demonstrate that the bio-inspired dual-flow design of this invention can effectively capture motion cues without auxiliary supervision, achieving superior overall accuracy in both translation and rotation estimation.

[0052] Table 3 compares the results of this invention and the self-supervised method on the KITTI dataset regarding relative translation error Terr and relative rotation error Rerr. This invention was evaluated on the challenging 4Seasons dataset, which contains large-scale, cross-seasonal, and lighting-variable scenes. As shown in Table 4, BEVO consistently achieved the lowest or second-lowest translational error, rotational error, and absolute trajectory error (ATE) across all sequences. Compared to binocular ORB-SLAM3, the average ATE, Terr, and Rerr of this invention are reduced by 55.3%, 36.5%, and 48.3%, respectively; and compared to state-of-the-art learning-based methods, improvements of 29%, 41.3%, and 34% are achieved on the same metrics. These results demonstrate that the dual-stream design of this invention effectively captures spatial and motion cues, achieving a balance between strong generalization ability and high efficiency with only a 224×224 input image.

[0053] Table 4 compares the results of this invention with representative VO methods on the 4Seasons dataset in terms of Terr, Rerr, and ATE. Ablation experiments verifying the dual-stream architecture Figure 5 The contributions of the ventral visual stream (VVS) and dorsal visual stream (DVS) were evaluated. Removing the VVS resulted in a sharp performance drop, with the absolute trajectory error (ATE), translation error, and rotation error increasing to 65.47 m, 42.97%, and 23.09 / 100 m, respectively. Removing the DVS also increased the error, but to a lesser extent. This highlights the crucial role of the DVS in efficiently capturing motion cues within a lightweight network architecture. These results confirm the complementarity of the two pathways: the VVS focuses on refined spatial feature analysis, while the DVS efficiently encodes motion information; together, they achieve accurate and robust pose estimation.

[0054] Ablation experiments to validate the core components of the ventral pathway This invention explores the contributions of Adaptive Convolution (AC) and Gated Cross-Attention Fusion (GCA) modules in the ventral visual pathway (VVS), and the results are summarized in Table 5. Introducing the AC module enhances temporal modeling capabilities, reducing ATE, Terr, and Rerr by 12.1%, 13.3%, and 14.4%, respectively. The GCA module utilizes visual saliency information from the dorsal visual pathway (DVS) to guide the VVS to focus on salient geometric structures, reducing Terr by 9.9% and Rerr by 8.4%. To balance dimensionality consistency and computational efficiency, this invention applies GCA only to the first layer feature map of the VVS. These results demonstrate that the two types of components have complementary advantages and can significantly improve visual odometry performance.

[0055] Table 5 Ablation experiments on the impact of adaptive convolution (AC) and gated cross-attention fusion (GCA) on the KITTI dataset.

[0056] Parameter sensitivity analysis of GRU hidden dimensions The hidden dimension of GRU units in the dorsal visual pathway (DVS) directly affects the model's ability to capture motion features. As shown in Table 6, increasing the hidden dimension can improve model performance, achieving optimal results in the range of 384-512 units; beyond this range, the performance improvement tends to plateau, but the computational cost continues to increase. Based on this trade-off, this invention selects 384 as the GRU hidden dimension, achieving the lowest translational error and absolute trajectory error while maintaining computational efficiency.

[0057] Table 6. Impact of GRU hidden size on performance on the KITTI dataset. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications should be covered within the scope of the claims of the present invention.

Claims

1. A two-stream end-to-end visual autonomous localization method based on bio-inspired spatiotemporal reasoning, characterized in that: The method specifically includes the following steps: S1: Obtain the input image sequence containing adjacent frames; S2: Input the input image sequence into the back-side visual pathway, extract and aggregate the motion features between frames to obtain a high-dimensional motion feature vector; S3: Input the input image sequence into the ventral visual pathway, and extract the spatial correlation features between frames through adaptive temporal convolution and the visual state space module to obtain spatial semantic features; S4: Through the gated cross-attention fusion module, feature fusion and interaction are performed using the spatial semantic features as queries and the high-dimensional motion feature vectors as keys and values; S5: Estimate the relative pose between the adjacent frames based on the fused features.

2. The dual-stream end-to-end visual autonomous localization method based on bio-inspired spatiotemporal reasoning according to claim 1, characterized in that: Step S2 specifically includes: S21: The input image sequence is feature-mapped using a feature extractor to obtain visual feature maps and contextual features; S22: Construct a fully correlated volume based on the visual feature map to quantify the spatial dependencies between frames; S23: The fully correlated entity is fused with the context features and input into the minimum gated recurrent unit network for parallel temporal aggregation; S24: Perform a nonlinear transformation on the aggregated features to output the high-dimensional motion feature vector.

3. The dual-stream end-to-end visual autonomous localization method based on bio-inspired spatiotemporal reasoning according to claim 2, characterized in that: Step S3 specifically includes: S31: Perform adaptive temporal convolution processing on the input image sequence, where the convolution weights are dynamically generated based on the global temporal context; S32: Input the adaptively convolutional processed features into a hierarchical architecture containing at least one visual state space module, capture long-range spatial dependencies with linear complexity through the state space mechanism, and output the spatial semantic features.

4. The dual-stream end-to-end visual autonomous localization method based on bio-inspired spatiotemporal reasoning according to claim 3, characterized in that: The weights of adaptive temporal convolution are generated in the following way: 1) Calculate the time descriptor and global descriptor based on the input sequence; 2) The temporal descriptor and the global descriptor are mapped and fused using a network containing one-dimensional convolutions to generate dynamic calibration weights; 3) Combine the dynamically calibrated weights with the basic convolutional kernel to obtain the adaptive convolutional weights at the current time.

5. The dual-stream end-to-end visual autonomous localization method based on bio-inspired spatiotemporal reasoning according to claim 4, characterized in that: In step S4, the gated cross-attention fusion module performs the following operations: S41: Project the spatial semantic features into a query vector, and project the high-dimensional motion feature vector into a key vector and a value vector; S42: Calculate the cross-attention weights between the query vector and the key vector, and apply them to the value vector to obtain attention features; S43: Generate gate weights through a channel-level gating network, weight the attention features with the gate weights, and perform residual fusion with the original spatial semantic features.

6. The dual-stream end-to-end visual autonomous localization method based on bio-inspired spatiotemporal reasoning according to claim 5, characterized in that: The feature extractor employs a residual network architecture, which includes cascaded residual blocks and uses 1×1 convolutions in the downsampling branch to adjust the dimensions of the residual connections.

7. The dual-stream end-to-end visual autonomous localization method based on bio-inspired spatiotemporal reasoning according to claim 6, characterized in that: In step S23, the update process of the minimum gated recurrent unit network is simplified as follows: Where Z is the update gate. Here, is the sigmoid activation function, Linear represents a fully connected layer with output dimension d, x is the input at the current time step, and h is the candidate hidden state.

8. The dual-stream end-to-end visual autonomous localization method based on bio-inspired spatiotemporal reasoning according to claim 1, characterized in that: Prior to step S1, a data augmentation step is included for the input image sequence, which includes color jitter and random erasure.

9. A dual-stream end-to-end visual autonomous localization system based on bio-inspired spatiotemporal reasoning, characterized in that: The system employs the method described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images