Dynamic scene navigation method based on two-stage optimization structure and space-time perception mechanism

Abstract

A dynamic scene navigation method based on a two-stage optimization structure and a space-time perception mechanism comprises the steps that four frames of time sequence depth images are obtained through depth estimation and a future frame prediction network, current and future space changes are represented, time sequence depth features and a target state are subjected to joint modeling through a ViT structure with the minimum change, and the time sequence depth images are obtained; global attention association between an image area and a target is realized, and target-oriented spatial-temporal features are extracted; in the strategy learning stage, basic obstacle avoidance and target driving capabilities are learned in a low-dynamic environment by imitation learning, and then two-stage optimization is performed in a high-dynamic environment by combining reinforcement learning with an evaluation result of an imitation strategy, so that the strategy convergence speed and the environmental adaptability are improved. According to the method, the robust navigation performance in a high-dynamic narrow environment can still be realized under the condition that a global map is not needed.

Description

Technical Field

[0001] This invention relates to a technology in the field of unmanned equipment navigation, specifically a dynamic scene navigation method based on a two-level optimization structure and a spatiotemporal perception mechanism. Background Technology

[0002] Current robot navigation largely relies on static maps or local planning, which can easily lead to local optima or collisions in dynamic or confined environments. While end-to-end methods are emerging, LiDAR often lacks sufficient semantic information. Image-based methods, which model without target information, result in feature extraction that lacks target guidance. Summary of the Invention

[0003] This invention addresses the shortcomings of existing indoor obstacle avoidance and navigation technologies, which fail to consider safe obstacle avoidance and arrival behaviors in real-world complex scenarios such as dynamic obstacles and narrow environments. These shortcomings include reliance on global maps, difficulty in handling obstacle changes in dynamic environments, lack of modeling capabilities for future scenarios, slow convergence speed of reinforcement learning, poor policy generalization ability, and inability to handle real-time planning requirements caused by non-rigid disturbances such as dynamic pedestrians. The invention proposes a dynamic scene navigation method based on a two-level optimization structure and a spatiotemporal perception mechanism. This method integrates spatiotemporal information extracted by a deep estimation network and a future frame prediction network, and jointly models the navigation strategy with target instructions through a minimally modified ViT structure. This enhances the navigation strategy's understanding of the interaction between passage space, dynamic obstacles, and target guidance. Furthermore, through a two-level optimization mechanism of imitation learning and reinforcement learning, the strategy performance is rapidly improved in the early training stage with expert demonstrations, and in the later stage, the strategy achieves self-iteration and generalization through reinforcement learning. This improves the robot's navigation robustness and task completion efficiency in dynamic, narrow indoor environments without relying on a high-precision global map, demonstrating good generalization ability and practical deployment value.

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a dynamic scene navigation method based on a two-level optimization structure and a spatiotemporal awareness mechanism, comprising:

[0006] The first step, spatiotemporal feature extraction: The current frame image is acquired and combined with historical frame sequences. Spatial structure and temporal change information of the environment are extracted using a depth estimation network and a future frame prediction network, respectively. This constructs a four-frame temporal depth image sequence with spatiotemporal feature representations possessing both current and future semantics. Specifically, this includes:

[0007] 1.1) Extract spatial geometric information of the environment from RGB images using a depth estimation network to construct spatial structure information of the current environment.

[0008] The depth estimation network described herein is implemented using, but is not limited to, the DepthAnything-V2 model.

[0009] 1.2) Extract time change information from historical frame sequences through a future frame prediction network to form a temporal depth image sequence that reflects the current state and future evolution trend.

[0010] The future frame prediction network is implemented using a recursive structure (PredRNN) based on spatiotemporal sequence modeling, which consists of multiple spatiotemporal long short-term memory networks (ST-LSTM) stacked together. It propagates and updates simultaneously from the historical frame sequence, i.e. the current three frames of depth images, in both the horizontal (spatial) and vertical (temporal) dimensions. This enables joint modeling of the spatiotemporal sequence and prediction of the depth information of the next frame image, thus forming a four-frame temporal depth image sequence that reflects the current state and future evolution trend.

[0011] The second step is target-oriented information fusion based on ViT: The improved ViT structure is used to fuse the features of the four-frame temporal depth image sequence with the target feature representation in the navigation task, achieving global modeling and attention association between image regions and target states. This extracts target-oriented spatiotemporal features, specifically including:

[0012] 2.1) Divide the current temporal depth image sequence obtained in the first step into image patches and encode them into image feature vector sequences to characterize the spatial structure and temporal change features of the environment.

[0013] The division into image patches refers to a sequence of sub-blocks representing local regions that are regularly segmented into the entire image according to a preset scale and stride, for use in subsequent encoding and modeling.

[0014] 2.2) By using a fully connected neural network, the preset target position information and the robot's own state information collected by the sensor are jointly encoded to generate a unified target feature representation.

[0015] 2.3) The target features and image features are further encoded and mapped to a unified implicit feature space. The input is then fed into the improved ViT structure to extract image features that are more relevant to the target and free of obstacles, which are then used to train the reinforcement learning network.

[0016] The improved ViT structure replaces the mapping position in the implicit feature space with the target feature representation. It models the contextual relationship between the image feature representation and the target feature through a self-attention mechanism, thereby enhancing the model's ability to model and understand target-oriented information, obtaining target-oriented spatiotemporal features, and fusing the target-guided attention mechanism with the predicted spatiotemporal scene information.

[0017] The third step, two-level policy optimization: After extracting the goal-oriented spatiotemporal features, a two-level optimization method combining imitation learning and reinforcement learning is used to obtain control instructions, thereby improving the learning efficiency and environmental adaptability of the reinforcement learning network. Specifically, this includes:

[0018] 3.1) In the early stages of training, imitation learning is used to supervise training on expert demonstration data. Specifically, the environment in the deep learning framework is set as a narrow corridor scene with a length of 60-70m and a width of no more than 3m, with 4-5 pedestrians randomly present to create local dynamic interference; the image features extracted in step 2.3 are used as input; the behavior space is defined as two-dimensional continuous control actions. This refers to linear velocity and angular velocity; the decision mapping is completed by a fully connected neural network, realizing an end-to-end mapping from perceived features to control actions. Demonstration data consisted of approximately 2000 state-action pairs collected manually in the aforementioned environment, and this data was used to train an imitation learning strategy.

[0019] 3.2) The reinforcement learning network outputs actions based on the image features extracted in step 2.3, and uses the Soft Behavior Policy Optimization (SAC) algorithm as the training backbone to output two-dimensional continuous control actions.

[0020] 3.3) In the two-stage optimization process: Boltzmann distribution is used to fuse and model the actions output by the imitation policy and the reinforcement policy. The behavior is selected probabilistically based on the state value distribution of the action. Specifically, the environment is set as a narrow corridor scene with a length of 60-70m and a width of no more than 3m, with 12 pedestrians. The pedestrians are more densely packed and the passage distance is narrower. The evaluation network (Critic) of the reinforcement learning network selects the action with higher value and more in line with the current environment from the behaviors of the two policy networks and interacts with the environment. The action is stored in the experience replay buffer to assist the continuous optimization of reinforcement learning and the stable training of the policy.

[0021] The fourth step is to perform online dynamic scene navigation through the trained reinforcement learning network.

[0022] Technical effect

[0023] This invention, within a modified ViT-based policy framework, integrates a target-guided attention mechanism with current and future spatiotemporal information, thereby enhancing the ViT network's ability to model dynamic environments and its forward-looking judgment of traversable areas. In the end-to-end obstacle avoidance and navigation algorithm, a two-layer optimization strategy is proposed. In the first layer, imitation learning is used to learn basic navigation and obstacle avoidance capabilities in sparse dynamic scenes. Building upon this, reinforcement learning explicitly considers the imitation learning strategy during updates to further learn adaptive navigation and obstacle avoidance strategies in dense dynamic environments. The two networks operate independently. Compared to existing technologies, this invention maintains a high arrival rate and demonstrates verifiable safety and efficiency in highly dynamic and narrow-channel testing environments. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the structure of the present invention;

[0025] Figure 2 This is a schematic diagram of the spatiotemporal feature extraction module;

[0026] Figure 3 This is a schematic diagram of a goal-oriented information fusion module based on ViT.

[0027] Figure 4 This is a schematic diagram of a two-level strategy optimization module;

[0028] Figure 5 Diagram of the test environment;

[0029] Figure 6 The image shown is a rendering of an example. Detailed Implementation

[0030] like Figure 1As shown in the figure, this embodiment relates to a dynamic scene navigation system based on a two-level optimization structure and a spatiotemporal awareness mechanism, including: a spatiotemporal feature extraction module 100, a ViT-based target guidance information fusion module 200, and a two-level strategy optimization module 300. Specifically: the spatiotemporal feature extraction module 100 uses a depth estimation network to extract the spatial structure information of the current frame image and predicts the depth information of future frame images based on the spatiotemporal long short-term memory network in PredRNN, constructing spatiotemporal features reflecting the current state and future evolution trends; the ViT-based target guidance information fusion module 200 first crops the image into image patches. The target point and the robot's own state are encoded through a fully connected network. The contextual relationship between the target and the environment is modeled through the self-attention mechanism of ViT, generating a target-oriented fused perception representation. The two-level policy optimization module 300 first uses radar perception data for imitation learning during the training phase to acquire basic policy capabilities. Then, it further optimizes the policy performance through image perception data and reinforcement learning methods. Finally, it fuses the output actions of the imitation and reinforcement policies through Boltzmann distribution to improve the robustness and generalization ability of the policy, enabling the robot to efficiently avoid obstacles and accurately navigate to the target location in a dynamic and narrow indoor environment.

[0031] like Figure 2 As shown, the spatiotemporal feature extraction module 100 includes: a current frame depth estimation unit 101 and a future frame prediction modeling unit 102, wherein: the current frame depth estimation unit 101 uses the acquired RGB image as input, and uses a depth estimation network to extract the spatial geometric information of continuous image frames to obtain the depth representation of the current environment; the future frame prediction modeling unit 102 models the current three frame depth images and predicts the depth information of the next frame image, thereby forming a four-frame temporal depth map sequence containing the current state and future change trends.

[0032] The four-frame temporal depth map sequence effectively reflects the dynamic evolution of the environmental structure, providing environmental perception input with temporal context for the subsequent target-oriented information fusion module.

[0033] like Figure 3As shown, the ViT-based target guidance information fusion module 200 includes: an image patch encoding unit 201, a target state encoding unit 202, and an embedded feature fusion unit 203. Specifically: the image patch encoding unit 201 divides the continuous depth image sequence output by the spatiotemporal feature extraction module into fixed-size image patches and encodes them into a structured image feature sequence to represent the spatial structure and temporal change information of the environment; the target state encoding unit 202 jointly encodes the spatial location information of the navigation target point and the robot's current state using a fully connected neural network to generate a unified target guidance feature vector; the embedded feature fusion unit 203 inputs the target features and image features into the improved ViT structure, transforms the first learnable class label vector (cls) into a target representation, models the contextual relationship between the image region and the target state through a self-attention mechanism, and outputs the fused navigation perception representation. This representation has a clear target guidance capability, guiding subsequent policy modules to focus on traversable areas related to target arrival in the dynamic environment, thereby improving overall navigation efficiency and robustness.

[0034] like Figure 4 As shown, the two-level policy optimization module 300 includes: an imitation learning policy network 301, a reinforcement learning policy network 302, and an evaluation and selection unit 303. Specifically: the imitation learning policy network 301 is trained in a simple dynamic scene using expert demonstration trajectories to supervise the state-action pairs, constructing an initial policy network with basic obstacle avoidance capabilities, suitable for static or low-dynamic scenes; the evaluation and selection unit 303 trains the reinforcement learning policy network 302 using a two-level policy; the reinforcement learning policy network 302 uses a Boltzmann distribution to fuse the action distributions of the imitation learning policy and the reinforcement learning policy, guiding the final behavior selection based on state value. The resulting actions are used to interact with the environment and stored in an experience replay pool to improve the sample efficiency and policy stability of reinforcement learning.

[0035] Through specific practical experiments, in the constructed simulated corridor environment, such as Figure 5 As shown: Total length 110–120m, width 2.5m, robot width 0.45m, minimum passage gap 0.7m, 12 sub-target points and 30 dynamic pedestrians are set, with blue arrows indicating pedestrian walking directions. Using the Gazebo simulation platform as the software environment and with uniform hardware configuration, the method described in this invention was repeatedly run five times. The experimental data obtained are: 49 successful arrivals in a total of 60 sub-target points, 7 timeouts, 4 collisions, an average travel distance of 104.76m, and an average time of 122.18s. The best path diagram from the five experiments is shown below. Figure 6As shown in Table 1, blue represents the robot's path trajectory, and yellow represents the trajectory where the robot collides with a pedestrian. Compared with the traditional rule-based DWA method and the latest LiDAR-based method PF-SAC, the arrival rate is significantly improved, and the average distance and time are minimized.

[0036] Table 1. Test performance of the comparison algorithm in complex scenarios.

[0037] Compared to existing technologies, this invention constructs spatiotemporal features with dynamic understanding capabilities by integrating depth estimation and future frame prediction techniques, and achieves deep fusion of target and environmental information by modifying the visual ViT structure. Furthermore, it employs a two-level optimization strategy combining imitation learning and reinforcement learning, which together comprehensively improve the learning efficiency and environmental adaptability of the navigation strategy. Compared to existing navigation methods that rely on static maps or lack robust spatiotemporal feature modeling capabilities, this invention not only possesses the ability to model future scene evolution trends but also achieves stronger target orientation, thereby guiding the strategy to focus on traversable areas in dynamic environments and improving navigation success rates while shortening travel time and distance. The overall system exhibits higher navigation success rates and obstacle avoidance robustness in dynamic, confined indoor environments, providing a more efficient and generalized map-free navigation solution for service robots, intelligent logistics, automated inspection, and other fields.

[0038] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A dynamic scene navigation method based on a two-stage optimization structure and a space-time perception mechanism, characterized in that, The application relates to a target-oriented information fusion method for dynamic scene navigation based on a vision transformer (ViT) and a reinforcement learning network. The method comprises the following steps: Firstly, spatio-temporal feature extraction: obtaining a current frame image and combining a historical frame sequence, respectively using a depth estimation network and a future frame prediction network to extract spatial structure information and time change information of the environment, and constructing a four-frame time sequence depth image sequence with current and future semantic spatio-temporal feature representation; Secondly, target-oriented information fusion based on a ViT: using an improved ViT structure to perform feature fusion on the features of the four-frame time sequence depth image sequence and target feature representation in a navigation task, realizing global modeling and attention association between image regions and target states, and thus extracting spatio-temporal features with target orientation; Thirdly, double-stage strategy optimization: after extracting the target-oriented spatio-temporal features, a double-stage optimization method combining imitation learning and reinforcement learning is used to obtain control instructions, so as to improve the learning efficiency and environmental adaptability of the reinforcement learning network; 2. The dynamic scene navigation method based on two-stage optimization structure and space-time perception mechanism according to claim 1, characterized in that, Fourthly, online dynamic scene navigation is performed through the trained reinforcement learning network. The first step specifically comprises the following steps: 1.1) extracting spatial geometric information of the environment from an RGB image through a depth estimation network, and constructing spatial structure information of the current environment; 3. The dynamic scene navigation method based on two-stage optimization structure and space-time perception mechanism according to claim 1 or 2, characterized in that, 1.2) extracting time change information from a historical frame sequence through a future frame prediction network, and constructing a time sequence depth image sequence reflecting the current state and future evolution trend.

4. The dynamic scene navigation method based on two-stage optimization structure and space-time perception mechanism according to claim 1, characterized in that, The future frame prediction network is realized by using a recursive structure (PredRNN) based on spatio-temporal sequence modeling which is stacked by multiple spatio-temporal long short-term memory networks (ST-LSTM), the historical frame sequence, i.e. the current three-frame depth image, is simultaneously propagated and updated in the horizontal (spatial) and vertical (temporal) dimensions, so as to realize joint modeling of the spatio-temporal sequence, and then the depth information of a future frame image is predicted, thereby constructing a four-frame time sequence depth image sequence reflecting the current state and future evolution trend. The second step specifically comprises the following steps: 2.1) dividing the current time sequence depth image sequence obtained in the first step into image blocks (patches) and encoding the image blocks into image feature vector sequences, so as to represent the spatial structure and time sequence change characteristics of the environment; 2.2) jointly encoding preset target position information and robot state information collected by a sensor through a fully connected neural network, and generating a unified target feature representation; 5. The dynamic scene navigation method based on two-stage optimization structure and space-time perception mechanism according to claim 4, characterized in that, 2.3) further encoding and mapping the target feature and the image feature to a unified implicit feature space, and inputting the image feature and the target feature to an improved ViT structure to extract the image feature and the target feature which are more relevant and have no obstacles, and then the image feature and the target feature are used for training the reinforcement learning network.

6. The dynamic scene navigation method based on two-stage optimization structure and space-time perception mechanism according to claim 1 or 5, characterized in that, The division into image blocks (patches) refers to a sub-block sequence of local regions obtained by regularly cutting the whole image according to a preset scale and step, and the sub-block sequence is used for subsequent encoding modeling.

7. The dynamic scene navigation method based on two-stage optimization structure and space-time perception mechanism according to claim 1, characterized in that, The improved ViT structure changes the mapping position on the implicit feature space to the target feature representation, models the context relationship between the image feature representation and the target feature through a self-attention mechanism, thereby enhancing the modeling ability and understanding ability of the model to target-oriented information, obtaining spatio-temporal features with target orientation, and fusing the target-oriented attention mechanism and the predicted generated spatio-temporal scene information. The third step specifically comprises the following steps: 3.1) In the early stage of training, imitation learning is used to supervise the training of expert demonstration data, specifically: set the environment in the deep learning framework as a narrow corridor scene with a length of 60-70m and a width of not more than 3m, and randomly exist 4-5 pedestrians to form local dynamic interference; the image features extracted in step 2.3 are used as input; the action space is defined as a two-dimensional continuous control action , i.e. linear velocity and angular velocity; the decision mapping is completed by a fully connected neural network, realizing end-to-end mapping from perception features to control actions, and about 2000 state-action pairs are collected in the above environment by manual teleoperation, and the data is used to train the imitation learning strategy; 3.2) The reinforcement learning network outputs actions according to the image features extracted in step 2.3, and uses the soft action policy optimization algorithm (SAC) as the training backbone to output two-dimensional continuous control actions; 3.3) In the two-stage optimization process: the actions output by the imitation policy and the reinforcement policy are modeled by simultaneously using the Boltzmann distribution to fuse the actions, and the actions are selected in a probabilistic manner according to the state value distribution of the actions, specifically: the environment is also a narrow corridor scene with a length of 60-70m and a width of no more than 3m, the number of pedestrians is 12, the pedestrians are more crowded, and the passable distance is more narrow; the evaluation network (Critic) of the reinforcement learning network selects the action that is more valuable and more consistent with the current environment from the actions of the two policy networks, and interacts with the environment, and stores it in the experience replay buffer (ReplayBuffer) for the purpose of assisting the continuous optimization and policy stable training of the reinforcement learning.

Images