Audio-visual navigation method based on hierarchical strategy-planner and dynamic navigation

Abstract

The invention discloses a hierarchical strategy-planner hybrid architecture for audio-visual navigation (AV-Nav) of a body-equipped agent, and aims to solve the challenge of tight coupling of multi-modal reasoning and precise action control in a traditional end-to-end model. According to the high-level strategy, a Transform-based network is adopted, deep fusion is carried out on real-time visual and acoustic features through a token and self-attention mechanism, and a navigation target point on a 3 * 3 local grid is output. The local target is then passed to a low-level classical planner that calculates the shortest path on the internally maintained dynamic navigation map using the Dijkstra algorithm and determines the atomic action to be performed. The method is characterized in that a planner is integrated with an interactive trial and error obstacle avoidance mechanism: once a collision signal Ct is received from the environment, the planner can immediately remove a corresponding blocked path edge from a navigation map, so that a system is forced to automatically re-plan a collision-free path in the next time step, and the navigation efficiency and robustness are remarkably improved. In addition, the architecture may perform end-to-end training in conjunction with novel audio enhancement strategies, including interfering sound sources, mixing or spectrogram masks.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence, embodied intelligence, and robot navigation technology. Specifically, this invention relates to an audio-visual navigation (AV-Nav) technology and system applied to complex 3D virtual or real environments. This technology combines multimodal fusion perception, hierarchical policy-planner, and dynamic path planning, aiming to improve the decision-making efficiency, navigation performance, and environmental robustness of intelligent agents in long-term tasks. Background Technology

[0002] Embodied agents still face significant challenges when performing long-sequential tasks such as audiovisual navigation in complex environments. Current technologies primarily employ end-to-end reinforcement learning (RL) models, directly mapping raw multimodal inputs, including visual and auditory inputs, to atomic actions such as moving forward, turning, or stopping. This design places a double burden on a single neural network: it must simultaneously perform complex multimodal reasoning, including understanding the spatial environment, real-time localization of sound sources, fusion of historical information, and precise low-level action control. This coupling requirement leads to extremely high sample complexity and training difficulty, making it challenging to learn efficient navigation strategies with environmental robustness.

[0003] While hierarchical reinforcement learning methods have been explored to simplify long-term decision-making tasks, existing approaches often require low-level planners to execute a complete sequence of actions to reach a pre-defined intermediate waypoint. This approach lacks flexibility, failing to adjust local targets in real-time based on the latest map and audio feedback at each environmental time step, thus limiting the agility of decision-making. Furthermore, traditional audiovisual information fusion models typically rely on convolutional neural network-recurrent neural network architectures, which struggle to achieve depth- and context-aware cross-modal information interaction when processing heterogeneous sensor data.

[0004] In particular, existing methods lack an efficient and real-time collision handling mechanism during actual navigation. When an agent collides with another object, the model often relies on external reward signals for slow learning and policy adjustment, failing to immediately identify and avoid infeasible paths, resulting in a significant reduction in navigation efficiency. Therefore, there is an urgent need for a novel hybrid architecture that can effectively decouple high-level policy planning from low-level action execution, and integrate a robust planner capable of dynamically correcting paths and achieving automatic obstacle avoidance, fundamentally improving the navigation performance and robustness of embodied agents in complex environments. Summary of the Invention

[0005] This invention aims to address the shortcomings of existing end-to-end embodied intelligent agent models in long-term tasks such as audiovisual navigation (AV-Nav), which suffer from low training efficiency, high sample complexity, and lack of real-time, active obstacle avoidance mechanisms due to the tight coupling between multimodal reasoning and low-level action control.

[0006] To achieve the above objectives, this invention provides a hierarchical strategy-planner hybrid architecture to effectively decouple high-level decision-making from low-level execution and achieve robust real-time obstacle avoidance.

[0007] Reference Figure 1 and Figure 2 The method and system framework proposed in this invention include the following main modules and steps:

[0008] Cross-modal feature extraction and tokenization: The visual observation data (RGB image, depth map) and auditory observation data (binaural spectrogram) acquired by the agent are input into independent visual encoders and audio encoders, both based on CNN architecture, to extract their high-dimensional feature maps F. v and F a A block embedding strategy similar to Vision Transformer (ViT) is employed, using 2D convolutional layers to extract the multimodal feature maps F. v and F a Transformed into a unified embedding dimension D embed visual token sequence Harmony token sequence Introduce a learnable [CLS] token T cls and combine it with all available visual token sequences. Harmony token sequence By concatenating the sequences, the input sequence of the Transformer can be constructed.

[0009] Transformer-based cross-modal deep fusion: The constructed input sequence T... t The input is fed into a standard Transformer encoder. Internally, the encoder utilizes a multi-head self-attention mechanism to enable deep information interaction between tokens from different modalities in the sequence, achieving context-aware cross-modal information fusion. The vector c corresponding to the [CLS] token is extracted from the output of the Transformer encoder. t This vector It serves as a global context vector for the current state, which incorporates information from all modalities.

[0010] High-level strategy temporal decision-making and target output: the global context vector c output from the Transformer tThe data is fed into a gated recurrent unit (GRU). The GRU unit is used to capture the temporal dependencies during navigation and output the updated hidden state h. t The output h of the GRU t The Actor head, fed into the policy network, ultimately calculates a probability distribution π(g) on ​​a 3×3 local grid. t |h t ), and sample or select local navigation targets g from them. t This high-level strategy focuses solely on deciding "where to go," rather than directly outputting atomic actions.

[0011] Low-level dynamic planner and interactive obstacle avoidance: The low-level planner runs independently and receives the local target g. t Convert to target node v on the map g Subsequently, a dynamic navigation graph maintained internally using Dijkstra's algorithm is employed. Calculate the current node v c to v g The shortest path P is found. The planner extracts the next node v from the shortest path P. n And according to v c and v n Relative to the agent's current orientation θ t The location is calculated, and the first atomic action a is executed. t The planner's key innovation lies in its real-time obstacle avoidance mechanism. When the agent executes action a... t Subsequently, a collision indication signal C was received from the environment. t When =True, the planner will immediately identify the node v that was attempted to be reached in the previous step. n ′, and from the edge set E of the navigation graph t Remove connection v c and v n The blocked edge e(v) of ′ c ,v n This dynamic update mechanism forces the planner to automatically replan in subsequent steps, effectively preventing the agent from repeatedly trying infeasible paths. Attached Figure Description

[0012] Figure 1This is a schematic diagram of the overall framework of the hierarchical policy-planner audiovisual navigation proposed in this invention. The diagram illustrates the decoupling of layer decision-making from low-level action execution. The high-level policy network focuses on outputting a local navigation target on a 3×3 grid through unified tokenization and fusion of audiovisual features. This target is then received by an independent planner, which uses the navigation graph to calculate the optimal atomic action. Its key collision handling mechanism enables the planner to dynamically remove obstructed paths and automatically replan new paths around obstacles (as shown in blue) when encountering obstacles (as shown in red circles), thereby ensuring robust navigation of the model in complex environments.

[0013] Figure 2 This is a flowchart illustrating the technical process of the method proposed in this invention. The diagram details the data flow: the high-level policy network first extracts visual and acoustic features using a CNN encoder, then tokenizes them, and finally performs deep fusion in a Transformer. The fused state is processed by a GRU to ultimately determine the local navigation target on a 3×3 action map. This target is then passed to a low-level path planner, which converts it into global coordinates and uses Dijkstra's algorithm to calculate the next atomic action 'a' on the dynamic navigation map. t The planner employs interactive trial and error. When the agent encounters a collision, the planner immediately removes the corresponding obstructed path from the navigation map, ensuring that the system can automatically replan a new path around the obstacle in the next moment, thus achieving robust dynamic obstacle avoidance navigation.

[0014] Figure 3 This is an example diagram of the multimodal input of the audiovisual navigation system involved in this invention in the Replica and MP3D dataset environment. The diagram includes: the original RGB image acquired by the agent, the corresponding depth map input, and the final top-down map. This top-down map clearly shows the planning space, the agent's starting point, target point, shortest path (green), and the agent's current path (blue). This diagram aims to illustrate the heterogeneous multimodal data upon which the model of this invention relies for perception and decision-making.

[0015] Figure 4 This diagram illustrates a comparison of navigation paths and efficiency between the proposed hierarchical strategy-planner model and the Soundspaces baseline model on the Replica and MP3D datasets. As shown, the proposed model achieves perfect path efficiency (SPL) in all four scenario examples. In contrast, the Soundspaces baseline model exhibits a significantly lower SPL value in the same scenarios, strongly demonstrating the superior performance of the proposed method in path finding and navigation efficiency.

[0016] Beneficial effects

[0017] This invention proposes a hierarchical policy-planner hybrid architecture for audiovisual navigation, aiming to address the inefficiency caused by the coupling of multimodal reasoning and low-level action control in existing end-to-end models. This architecture is implemented as follows: First, the system extracts visual features F through independent visual and auditory encoders. v Acoustic characteristics F a And use block embedding technology to transform these features into token sequences. and These tokens are then concatenated with the [CLS] token and fed into the Transformer encoder. A multi-head self-attention mechanism is used to achieve deep cross-modal fusion, and the context vector c representing the global state is extracted. t Next, the vector c t After time-series modeling by GRU cells, the policy network outputs a local navigation target g on a 3×3 local mesh. t This completes the output of high-level decision-making; finally, the low-level classical planner receives g. t Transform it into the target node v g Dijkstra's algorithm is used in its maintained dynamic navigation graph Calculate the shortest path and determine the atomic action to be performed. t Its core innovation lies in the planner's integration of an "interactive trial-and-error" obstacle avoidance mechanism. Once a collision signal C is received from the environment... t Immediately from the picture The corresponding obstructed path edge 'e' is removed from the algorithm to force the system to automatically replan in the next time step, achieving real-time and efficient active obstacle avoidance. Furthermore, this invention can be combined with audio enhancement strategies such as interference sources, mixing, or spectrogram masks to improve the model's robustness in complex acoustic environments.

Claims

1. An audiovisual navigation method based on hierarchical strategy-planner and dynamic navigation, characterized in that, The process includes the following steps: receiving visual and auditory features from the environment; tokenizing and deeply fusing the features using a Transformer-based policy network; and outputting a local navigation target (g) on ​​a preset local grid by a recurrent neural network (GRU unit). t A separate low-level planner is used to receive the local navigation target g. t Convert it into a target node v on the navigation graph. g Dynamic navigation map maintained internally The shortest path P is calculated using Dijkstra's algorithm, and the first atomic action a on the shortest path is extracted. t ; the atomic action a t Submitted to the environment for execution; in executing the atomic action a t Subsequently, if a collision indication signal C is received from the environment... t If the value is True, the lower-level planner immediately removes the dynamic navigation graph. Connect the current node v in the middle c The next node v expected in the previous step n The blocked path edge e(v) between ′ c ,v n This forces the path to be automatically replanned in the next time step.

2. The audiovisual navigation method according to claim 1, characterized in that... The high-level strategy decision-making step achieves cross-modal deep fusion by converting the visual and auditory features into a multimodal token sequence; concatenating the token sequence with a [CLS] token to construct the input sequence of the Transformer encoder. The token sequence is fused using the multi-head self-attention mechanism of the Transformer encoder to extract the global context vector c corresponding to the [CLS] token. t The global context vector c t The data is then input into the GRU unit to complete the modeling of navigation timing dependencies and output the local navigation target g. t .

3. The audiovisual navigation method according to claim 1 or 2, characterized in that... The low-level planning execution step implements path calculation in the following manner, using the local navigation target g... t Transform the agent's egocentric coordinate system into global coordinates on the map; in the dynamic navigation map Above, the low-level planner uses Dijkstra's algorithm to calculate the algorithm from the current node v. c To the target node v g The planner extracts the next node v from the shortest path P; n And according to the current node v c and the next node v n Relative to the agent's current orientation θ t From the location, the atomic action a is derived. t .

4. The audiovisual navigation method according to claim 1, characterized in that... The dynamic graph update and obstacle avoidance steps are an interactive trial-and-error obstacle avoidance mechanism, which aims to enable the planner to learn and dynamically correct its internal path planning model in physical collisions. The method also includes an audio data augmentation step during the training process. The audio data augmentation includes at least one of the following strategies: adding interference sound sources at non-target points. A second sound source may be mixed in at the target location; or a time-frequency mask may be applied to the auditory feature spectrogram.

Images