Embodied visual language navigation method based on modal mass dynamic regulation

Abstract

The present application relates to the technical field of embodied navigation, and particularly relates to an embodied visual language navigation method based on modal quality dynamic regulation, comprising: constructing cross-modal input containing visual, language and topological features; inputting the cross-modal input into a multi-modal expert reasoning network composed of visual experts, language experts and topological experts which are functionally decoupled, respectively generating action prediction scores of each modality; calculating modal quality scores and basic confidence of each expert through a modal quality driven dynamic regulation module, fusing to generate quality enhanced confidence, and then determining the final fusion weight, weighting and fusing the action prediction scores of each expert, obtaining the final navigation action and controlling the agent movement. The present application quantifies the quality of multi-modal input and dynamically adjusts the decision weight of experts, autonomously weakens the influence of low-quality modalities in complex scenes such as visual occlusion and language ambiguity, enhances the contribution of high-quality modalities, and significantly improves the robustness, navigation accuracy and task success rate of the system.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and embodied navigation technology, specifically relating to an embodied visual language navigation method based on dynamic control of modal quality. Background Technology

[0002] Visual-language navigation is an important task in the field of embodied intelligence. Its goal is to enable intelligent agents to autonomously navigate from a starting point to a destination in realistic and complex indoor environments, relying solely on first-person visual perception, based on natural language instructions. This task requires intelligent agents to possess deep cross-modal understanding capabilities, enabling them to align and reason about the semantics of language instructions with real-time changing visual information in the environment.

[0003] Since the task was proposed, researchers have developed various technical approaches. Early methods, such as the sequence-to-sequence "Speaker-Follower" model, achieved basic instruction following through cross-modal matching, but were prone to error accumulation in long paths and environments with visual interference. To improve performance in complex environments, subsequent research introduced techniques such as cross-modal attention, topology map memory mechanisms, and Transformer-based architectures. For example, in the DUET architecture navigation scheme, a topology map of the environment is dynamically constructed, and two branches are set up: coarse-grained path planning and fine-grained local action prediction. During decision-making, the prediction results of the two branches are fused with fixed weights.

[0004] Furthermore, existing technologies also include solutions that incorporate human-computer interaction to improve navigation performance. For example, Chinese patent application 202211729544.2 discloses a visual language navigation system and method based on VR devices. This system obtains user eye movement information through an eye movement feature extraction module and fuses it with visual, text, and other features through an eye movement information fusion module. Its core lies in triggering a human-computer collaboration mechanism when the model's decision confidence is insufficient, introducing the user's eye movement information as an additional input modality, and then re-making the decision through a GRU network module. This solution is essentially an interactive enhancement architecture of "decision-feedback-re-decision," and its core navigation still relies on an attention weight module to weight multiple features and a GRU network for action prediction.

[0005] However, through in-depth analysis and practice, the applicant discovered that the existing technologies, including the aforementioned DUET scheme and the interactive scheme represented by patent application No. 202211729544.2, all have the following defects:

[0006] Multimodal information fusion strategies are rigid and lack autonomous quality perception and adaptive capabilities. Non-interactive methods such as DUET employ fixed fusion strategies to integrate prediction results from different sources (e.g., coarse / fine granularity), failing to proactively adjust decision-making priorities based on instantaneous quality fluctuations in visual and linguistic modalities caused by environmental interference (e.g., occlusion, sudden changes in illumination, semantic ambiguity). While scheme 202211729544.2 introduces confidence level judgments, these confidence levels are only used to trigger external human-computer interaction (requesting eye-tracking input) and are not used internally by the model to evaluate the reliability of existing modalities or inference paths or to reallocate weights. The attention weight module within the model remains a static or forward-computed process, lacking the ability to dynamically adjust information flow based on input.

[0007] The model structure is too simplistic: Existing solutions (including DUET's dual-branch and the GRU network of 202211729544.2) are essentially still fixed model structures, where all parts or main branches of the model are activated to participate in computation for all inputs. This structure cannot achieve on-demand allocation of computing resources and targeted information processing, thus limiting the efficiency and performance of the model.

[0008] The lack of a systematic multi-scale planning and decision-making dynamic balancing mechanism is a significant issue. While the DUET scheme proposes coarse / fine-grained dual branches, the integration between the two is fixed. The scheme disclosed in Chinese patent application 202211729544.2 completely fails to distinguish between the different scales of global planning and local decision-making. Existing technologies generally lack an explicit, uncertainty-based mechanism to dynamically balance the contributions of long-term global planning and short-term local decisions, leading to agents easily getting trapped in local optima or accumulating errors due to early misjudgments in complex paths. Summary of the Invention

[0009] To address the shortcomings of existing embodied visual language navigation methods, such as rigid multimodal information fusion strategies, lack of autonomous quality perception and adaptive capabilities, and lack of systematic multi-scale planning and dynamic balancing mechanisms for decision-making, an embodied visual language navigation method based on dynamic modal quality control is proposed.

[0010] The objective of this invention can be achieved through the following technical solutions:

[0011] An embodied visual language navigation method based on dynamic modal quality control includes the following steps:

[0012] S1. Construct a cross-modal input that includes visual features, linguistic features, and topological graph features;

[0013] S2. Input the cross-modal input into a multimodal expert inference network, which includes functionally decoupled visual experts, language experts, and topology experts, respectively used to generate action prediction scores based on visual modality, language modality, and topology modality;

[0014] S3. Based on the modal quality-driven dynamic control module, the output of the multimodal expert inference network is fused: S31. Calculate the modal quality score of each expert; S32. Calculate the basic confidence score based on the action prediction score of each expert; S33. Fuse the basic confidence score with the corresponding modal quality score to generate a quality-enhanced confidence score; S34. Calculate the final fusion weight of each expert based on the quality-enhanced confidence score; S35. Use the final fusion weight to perform weighted fusion of the action prediction scores of each expert to obtain the final navigation action;

[0015] S4. Control the movement of the intelligent agent according to the final navigation action, and repeat steps one to four until the navigation task is completed.

[0016] Further, in step S34, the process of calculating the final fusion weight includes:

[0017] The quality enhancement confidence score is logarithmically normalized to obtain the relative confidence score;

[0018] The quality enhancement confidence score and the relative confidence score are weighted and superimposed to generate a collaborative confidence score;

[0019] The collaborative confidence is standardized to obtain the final fusion weight.

[0020] Furthermore, in step S31:

[0021] The modal quality score of the visual expert is calculated based on the temporal variation of their visual features and their attention concentration.

[0022] The language expert's modal quality score is calculated based on the semantic alignment between the language description and the visual scene, as well as the semantic consistency with the topological structure.

[0023] The modal quality score of the topology expert is calculated based on the temporal consistency of the topology and its similarity to visual global features.

[0024] The beneficial effects of this invention are:

[0025] This invention quantifies the input quality of visual, linguistic, and topological modalities, and dynamically adjusts the weights of each expert in decision-making based on the quality assessment results. In complex scenarios such as visual occlusion, ambiguous linguistic instructions, or uncertain environmental structures, the system autonomously weakens the influence of low-quality or unreliable modalities while enhancing the decision-making contribution of high-quality modalities. This enables the agent to possess stronger self-correction and adaptability in dynamic environments, significantly improving the robustness and stability of the overall system.

[0026] Furthermore, a multimodal expert inference network enables differentiated and specialized processing of information from different modalities. The gating network initially assigns weights to experts based on the task status, facilitating rapid selection of the decision-making leader; the dynamic adjustment module further refines the fusion weights based on real-time quality indicators. This two-tiered mechanism works together to achieve intelligent and precise fusion of multimodal information, effectively suppressing the accumulation of erroneous decisions and significantly improving the success rate of navigation tasks and the accuracy of destination positioning. Attached Figure Description

[0027] The invention will now be further described with reference to the accompanying drawings.

[0028] Figure 1 This is an overall flowchart of the embodied visual language navigation method based on dynamic control of modal quality in this invention;

[0029] Figure 2 This is a comparison diagram of the navigation trajectories of the present invention and the DUET method;

[0030] Figure 3 This is a comparison diagram of the method of the present invention and the actual path (demonstration path). Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] refer to Figure 1 This invention proposes an embodied visual language navigation method based on dynamic modal quality control, comprising the following steps:

[0033] Step 1: Model and represent the observation information of the environment, language instructions, and intelligent agents, and construct a cross-modal input that includes visual features, language features, and topological graph features.

[0034] The indoor navigation environment is modeled as an undirected graph G={V,E}, where V={v iLet} be the set of navigable nodes, and E be the connecting edges between nodes. Language instructions consist of word sequences of length L, and semantic information is encoded through word vector embedding. At each time step t, the agent acquires a panoramic view of the current node and its position information. The panoramic view is divided into image sets in six directions, and each image is accompanied by corresponding visual features and orientation encoding.

[0035] During navigation, this invention, based on the DUET architecture, dynamically updates the visual representation of nodes using continuously arriving new node information, gradually constructing a topological subgraph G. t ={V t E t The topology graph encapsulates the visual observation information obtained from the previous t navigation steps. It contains three types of nodes: visited nodes (representing positions previously visited by the agent), current position nodes (representing the agent's current position), and navigable nodes (representing candidate positions reachable by the agent in its current state).

[0036] Whenever the agent reaches a new node, its current language instruction, topological graph representation, and panoramic visual information of its current location are jointly fed into the multimodal expert inference network as input. Through cross-modal interaction and expert decision-making mechanisms, the visual information, language information, and graph structure information are jointly inferred to output the next action instruction.

[0037] Step 2: Construct a multimodal expert reasoning network

[0038] After receiving cross-modal input, the agent processes the input information by constructing a functionally differentiated expert network. Experts are categorized into three types: vision-dominant experts, language-dominant experts, and topology experts. Vision-dominant experts are responsible for extracting obstacle information and scene details from local visual cues; language-dominant experts are responsible for parsing long-term navigation goals and directional constraints from command semantics; and topology experts assess the current map structure and provide global path planning references.

[0039] To avoid the homogeneity limitations of traditional expert structures, this invention assigns different functional emphases to three types of experts in terms of expert input, focus features, and network processing priorities. At each time step t, the three types of experts work in parallel, each outputting its own action prediction score based on its own modality. Simultaneously, based on the current navigation status, an initial participation weight is generated for each expert, achieving preliminary adaptive allocation of the decision-making leader.

[0040] Visual experts

[0041] First, features are extracted from the multi-view images at time step t using a visual encoder. Then, using the linguistic semantic vectors as queries, a relevance weight for the multi-view features is calculated via an attention mechanism. And form a language-guided visual aggregation representation. To characterize the temporal stability of visual modalities, the temporal changes in visual features are calculated: Used to identify visual drift conditions such as occlusion and changes in lighting.

[0042] Constructing visual modal quality indicators ,

[0043] Where c t Indicates attention concentration u t Indicates the consistency of visual convergence features at adjacent time points γ1, γ2, γ3 are learnable parameters, and σ is the Sigmoid function.

[0044] The visual expert input vector is constructed using the feature fusion mapping function ψ. , where h is the encoding function. Ultimately, visual experts use the prediction network f vis Output the action score vector based on the visual modality: .

[0045] The visual modal quality index Q constructed in this embodiment t vis This method effectively addresses the robustness deficiencies of traditional visual language navigation methods caused by visual drift or distraction. By introducing attention concentration and temporal consistency as positive enhancement factors, and feature variation amplitude as a negative inhibition factor, it achieves proactive filtering and stability assessment of visual information. This mechanism ensures that visual experts can reason based on focused and stable visual features, significantly reducing the risk of misjudgment due to visual interference or distraction, thereby comprehensively improving the model's adaptability and navigation reliability in dynamically changing environments.

[0046] Language experts

[0047] Language experts use it to extract navigation targets, directional constraints, and key semantic fragments from natural language instructions, and combine them with semantic consistency information from vision and topology to provide semantically driven decision-making for action prediction.

[0048] First, based on the current visual features and topological features, the semantic consistency score between the language description and the scene is calculated, including the semantic consistency score m between the language description and the current visual features. t vis And the semantic consistency score m between the language description and the current topology. t topo .

[0049] Constructing language modal quality indicators:

[0050] ,

[0051] Where β1 and β2 are learnable parameters, and σ is the Sigmoid function. Subsequently, language experts combine the above semantic and matching-related features into an input vector using a feature fusion mapping function ψ:

[0052] ,

[0053] Where h is used to map scalar quality values ​​to a unified latent space. l represents the semantic global vector, l loc It is a local key semantic vector, v t att G represents multi-view visual weighted aggregation features. t This represents the pooling feature of the topological graph.

[0054] Ultimately, language experts used predictive networks f lang Output action score: .

[0055] In this embodiment, the language modality quality index Q t lang It is designed as a metric that dynamically reflects the semantic consistency between language instructions and the current scene. This metric enables language experts to proactively assess the executability of current instructions in a specific environment, thereby effectively avoiding navigation errors caused by mismatches between instructions and the local visual or topological environment, and enhancing the system's ability to identify and adapt to semantic-scene misalignment.

[0056] Topology experts

[0057] Topology experts are used to model the structural information of the current navigation graph, including the connectivity relationships between nodes. First, the topological subgraph G, consisting of the agent's current node and its neighboring nodes, is constructed. t Inputting the graph coding network yields a structure vector representation:

[0058] ,

[0059] in This represents the encoding function for the topology, used to extract structural features such as node features and node connectivity. To identify structural abrupt changes or local map instability, the temporal changes in the topology are calculated.

[0060] ,

[0061] Then, construct the topological modal quality index:

[0062] ,

[0063] b tSimilarity between topological features and visual global features; , Learnable parameters; This is the Sigmoid function, used to normalize the topology quality score.

[0064] Subsequently, the topological features and quality indicators are combined into topology expert input through a feature fusion mapping function:

[0065] ,

[0066] Where ψ is the fusion network and h is the encoding function. Finally, the topology expert uses the prediction network f. topo Output a structure-based action score vector:

[0067] .

[0068] This embodiment dynamically suppresses reliance on unstable or unreliable map information by comprehensively evaluating the stability of the topology and its consistency with visual global features. It effectively solves the problem of errors in local path planning that often occur in traditional methods when environmental map information is missing or the structure changes abruptly, significantly improving the planning reliability of the navigation system in unstructured or dynamically changing scenarios.

[0069] Initial expert weights and fusion

[0070] To achieve adaptive allocation of decision-making power, this invention proposes an initial weight allocation method for quality and action prediction intensity perception.

[0071] First, construct a low-dimensional decision feature vector X. t dec It integrates the two most critical quantitative indicators for three types of experts: the modal quality index Q and the predictive strength of the action prediction score S.

[0072] ,

[0073] By X t dec Convert to initial weights w for three types of experts t initial , , where Wg is a learnable weight matrix, and its three rows of parameters correspond to visual experts, language experts and topology experts, respectively; Let be the initial weight vectors of the three types of experts at time t.

[0074] This implementation considers both modal quality and action prediction strength. The agent can perceive the reliability and decision confidence of each expert in the current state at the inference stage, providing a reasonable starting point for subsequent fine-tuning. The generation process of the initial weights is transparent and traceable. The weight matrix can learn the applicable patterns of each expert in different scenarios through training data, making the system's decision logic more interpretable and scenario-adaptable, and avoiding decision fluctuations caused by random or fixed assignments.

[0075] Step 3: Construct a dynamic control module based on modal quality driving.

[0076] To achieve reliable action prediction, a modality quality-driven multi-expert dynamic control module is proposed. Unlike the existing fusion methods that use fixed hyperparameters or attention mechanisms, this module adaptively adjusts the contribution of different prediction results by evaluating modality reliability and action prediction confidence, thereby maintaining stable decision-making capabilities in dynamic scenarios.

[0077] To assess the uncertainty of action scores predicted by different networks, the ConfBase confidence network is first used to differentiate the individual confidence scores of different expert navigation networks. The individual confidence score reflects the initial confidence level of the prediction, and the specific calculation formula is as follows:

[0078] ,

[0079] Step two has already obtained the quality score for the corresponding modality, and then a shared two-layer perceptron f is used. qc By fusing the base confidence score and modal quality at the feature level, a quality-enhanced single confidence score is obtained:

[0080] ,

[0081] To characterize the relative confidence levels among the three types of experts, the enhanced individual prediction strengths are log-normalized to obtain the relative confidence levels:

[0082] ,

[0083] Where ε is a smoothing constant. Then, the individual confidence scores and relative confidence scores are weighted and summed to form the collaborative confidence score for each expert:

[0084] ,

[0085] u t (i) The larger the value, the higher the expert's overall credibility at the current moment. Then, the final fusion weight is obtained:

[0086] ,

[0087] ,

[0088] The motion prediction scores of the three types of experts are then weighted using these scores to obtain the final fused motion prediction vector:

[0089] ,

[0090] It is a learnable gating sensitivity parameter used to adjust the modulation intensity of each expert modal quality index on the prediction score of each expert action. The Hadamard product (element-wise multiplication) applies a gating factor to each element of the action score vector.

[0091] This invention, through its proposed modal quality-driven dynamic control steps, can simultaneously characterize the basic prediction reliability of each expert and the current modal input quality when fusing prediction results from three types of experts. Specifically, it first utilizes the formula... Obtain the basic confidence level of the prediction network, and then pass it through a multilayer perceptron f qc The baseline confidence level and modal quality q t (i) Fusion yields enhanced confidence in quality This explicitly reflects the impact of observation quality on prediction reliability; further, a log-normalized form is used to obtain the relative confidence level. and with Weighted summation to form collaborative confidence The dynamic fusion weights w of the three types of experts are obtained through the Softmax function. t In the final action fusion step, a quality-gated modulation mechanism is introduced to obtain the final fused action prediction vector score. The next navigation action is decided based on the predicted vector score.

[0092] Compared with existing fusion strategies that use fixed hyperparameters or simple attention scoring, this step can automatically reduce the weight of a certain modality in decision-making when the quality of a certain modality deteriorates due to environmental interference, while strengthening the contribution of high-quality modalities. This significantly improves the robustness of multimodal action prediction and the stability of overall decision-making in complex dynamic scenarios.

[0093] Step 4: Control the movement of the intelligent agent according to the final navigation action, and repeat steps 1 to 3 until the navigation task is completed.

[0094] This embodiment employs a phased training and inference execution approach to gradually build the agent's comprehensive capabilities in visual understanding, language parsing, and navigation decision-making, thereby improving the model's task completion rate in complex environments. This process is divided into a pre-training phase, a fine-tuning phase, and an inference execution phase.

[0095] During the pre-training phase, the model's initial capabilities are built through multiple auxiliary tasks, including masked language modeling, masked region classification, and single-step action prediction. Through these tasks, the model learns the semantic structure of language instructions, visual elements of panoramic images, and causal relationships between basic actions. Furthermore, an additional target localization task is introduced, enabling the model to establish a correlation between language descriptions and visual targets during pre-training, thereby enhancing the model's ability to identify and locate target objects during subsequent navigation.

[0096] In the fine-tuning phase, a combination of imitation learning and reinforcement learning is employed to enable the agent to further optimize its navigation strategy in an interactive environment. Specifically, the agent continuously adjusts its action choices based on the demonstration path or reward signals in a simulated environment, allowing the model to adapt to dynamic environmental changes in different scenarios. Unlike the pre-training phase, the fine-tuning phase emphasizes the model's adaptive capabilities in sequential decision-making, enabling it to make effective decisions regarding the shortest path and key nodes based on the current view, topological information, and language objectives.

[0097] During the inference execution phase, the agent performs actual inference on the navigation task based on the trained model strategy. During inference, the agent gradually predicts the optimal next action based on the currently acquired panoramic image features, language commands, and environmental topology, and continuously approaches the target location by executing consecutive actions. When the model determines that the matching degree between the current location and the target location reaches a set threshold, or when the number of steps reaches a preset maximum limit, the agent will stop navigation and output the final target node.

[0098] To clearly demonstrate the performance differences between this invention and existing technologies, this embodiment compares the navigation trajectories of the method of this invention and the DUET model under the same instructions and environmental settings. Figure 2 As shown, each row represents a panoramic view of the agent at different time steps, and its predicted action is labeled.

[0099] The test command was: "Enter the room with the large TV. Continue forward, past the sofa, then exit through the back door to the backyard terrace. Stop outside the door facing an outdoor dining facility." Experimental results show that the method of this invention can accurately parse the semantic structure of complex commands and, through a multimodal expert reasoning network and dynamic scheduling mechanism, effectively integrate and reason about environmental information. During navigation, the system can dynamically select the leading expert based on scene changes and flexibly adjust decision-making strategies, thus maintaining efficient path reasoning capabilities in complex indoor-outdoor transition environments. The agent successfully identified and utilized key landmarks such as "TV," "sofa," "door," and "outdoor dining facility," ultimately accurately reaching the target location specified in the command.

[0100] In contrast, the DUET model exhibits significant limitations in action reasoning. For example... Figure 2 As shown, in step 2, it failed to correctly understand the path constraint implied by the instruction "passing the sofa," causing the agent to turn prematurely and deviate from the correct path. In subsequent navigation, although the DUET model also observed some target objects such as "door" and "table," its fixed fusion strategy could not correct the early inference bias, making it difficult for the system to recover from the erroneous decision. Ultimately, it stopped at a position inconsistent with the target description and failed to complete the navigation task.

[0101] To verify the robustness of the present invention under visual interference environments, Figure 3 This paper presents a comparison between the method of this invention and a real-world path (demonstration path) in a complex scenario. The real-world path (demonstration path) here refers to the standard answer path in the dataset that corresponds one-to-one with the natural language instruction; it is the optimal or expert path taken by a human annotator in a simulated environment to complete the instruction. The test instruction was a multi-stage task: "Leave the bathtub—cross the corridor—enter from the right entrance—stop against the wall." This environment featured significant changes in lighting, narrow passages, and visual occlusion, posing a severe challenge to the stability of the visual modality.

[0102] The actual trajectory (a) shows that the agent can continuously complete a series of key actions such as turning, moving forward, and turning right, and successfully reach the target position. However, the method of the present invention (b) encounters visual modality misjudgment at step ②. The reason is that there is a strong sudden change in illumination in the corridor area at this position, which leads to local overexposure and uneven boundary between light and dark, which seriously interferes with the extraction and recognition of visual features.

[0103] It is important to emphasize that, despite visual interference, the modal quality assessment and dynamic balancing mechanism proposed in this invention plays a crucial role: the system identifies a significant decline in visual modal quality in real time and immediately reduces the weight of visual experts through a dynamic adjustment module, while simultaneously increasing the decision-making participation of language and topology experts, thereby effectively avoiding action deviations caused by visual misjudgments. As the agent continues to move and enters a normally lit area, the visual modal quality gradually recovers, the predictions of the experts converge again, and ultimately the agent successfully locates a stopping position that perfectly matches the actual path, accurately completing the task.

[0104] This invention, by constructing a functionally decoupled multimodal expert network and innovatively introducing a dynamic evaluation and control mechanism for modal quality, achieves a fundamental leap from fixed fusion to intelligent adaptation. This enables the navigation system to maintain accurate decision-making even in complex environments such as visual occlusion and sudden changes in lighting, significantly improving the robustness, navigation accuracy, and task success rate of the intelligent agent in real-world scenarios. The above describes one embodiment of the invention in detail, but this content is only a preferred embodiment and should not be considered as limiting the scope of the invention. All equivalent variations and improvements made within the scope of this invention should still fall within the patent coverage of this invention.

Claims

1. An embodied visual language navigation method based on dynamic modal quality control, characterized in that, Includes the following steps: S1. Construct a cross-modal input that includes visual features, linguistic features, and topological graph features; S2. Input the cross-modal input into a multimodal expert inference network, which includes functionally decoupled visual experts, language experts, and topology experts, respectively used to generate action prediction scores based on visual modality, language modality, and topology modality; S3. Based on the modal quality-driven dynamic control module, the output of the multimodal expert inference network is fused: S31. Calculate the modal quality score of each expert; S32. Calculate the baseline confidence level based on the action prediction scores of each expert; S33. The basic confidence score is fused with the corresponding modal quality score to generate a quality-enhanced confidence score; S34. The final fusion weight of each expert is calculated based on the quality-enhanced confidence score; S35. The quality gating factor of each expert is generated based on the modal quality score and the learnable gating sensitivity parameter. The action prediction score vector output by the corresponding expert is modulated using the quality gating factor to obtain the modulated action prediction score of each expert. The modulated action prediction scores of each expert are weighted and fused according to the final fusion weight to generate the final fused action prediction score, thereby determining the final navigation action; S4. Control the movement of the intelligent agent according to the final navigation action, and repeat steps one to four until the navigation task is completed.

2. The embodied visual language navigation method based on dynamic modal quality control according to claim 1, characterized in that, In step S31: The modal quality score of the visual expert is calculated based on the temporal variation of their visual features and their attention concentration. The language expert's modal quality score is calculated based on the semantic alignment between the language description and the visual scene, as well as the semantic consistency with the topological structure. The modal quality score of the topology expert is calculated based on the temporal consistency of the topology and its similarity to visual global features.

Images