Target guiding navigation method

Abstract

The invention discloses a target guiding navigation method which comprises the following steps: S1, coding an environment image by adopting a multi-teacher distillation pre-trained RADIO framework, extracting an initial visual feature, and splicing the initial visual feature with a target embedded vector to obtain a cross-modal fusion initial input feature; s2, generating causal alignment fusion features through global hybrid factor estimation, residual elimination and anti-fact co-occurrence matrix intervention; and S3, inputting the features into the LSTM, and generating a navigation action strategy through double-commentator conservative value estimation, Gaussian perturbation and semantic post-view playback training in combination with a cross attention fusion semantic graph relationship and time sequence features. According to the method, multi-source prior knowledge is fused to improve visual feature generalization, hybrid interference is separated through a causal de-confusion mechanism, semantic enhancement is combined to reinforce learning training, the problems of category dependence, false correlation, insufficient exploration efficiency and the like of an existing method are solved, and the stability, causal rationality and training effect of a navigation strategy are improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of robot control and artificial intelligence, and particularly relates to a target-oriented navigation method. BACKGROUND

[0002] With the rapid development of artificial intelligence and embodied intelligence technology, it has become an important research direction in the field of intelligent robots to enable robots to perform navigation in unknown environments according to natural instructions. Target-oriented navigation requires an intelligent agent to identify and reach a specified target object in an unfamiliar indoor 3D environment, relying only on visual observation centered on itself. To achieve efficient target-oriented navigation, the intelligent agent needs to integrate object recognition, contextual prior knowledge (such as associating a remote control with a nearby television) and scene memory capabilities to guide the environmental exploration process.

[0003] However, current target-oriented navigation technology still faces many key challenges. At the level of visual perception, the intelligent agent has difficulty capturing spatiotemporal dependencies from continuous RGB observation data, leading to errors when dealing with object occlusion and motion artifacts. At the same time, distribution shifts in environmental factors such as lighting, texture and layout can significantly reduce the robustness of the system, while false associations introduced by environmental changes can further amplify positioning errors. These problems directly lead to inefficient path planning and high task failure rates for the intelligent agent in partially observable scenarios, and poor generalization ability for new objects and disturbed environments. The above challenges highlight the importance of learning robust visual representations and developing navigation strategies that can adapt to complex and unpredictable environments.

[0004] In the aspect of visual representation construction, existing research has made relevant explorations. Some works rely on pre-trained visual-language models such as CLIP to extract robust visual features through the alignment of images and text descriptions; other research constructs object-centered graph structures to explicitly model the spatial and semantic relationships between objects, thereby enhancing visual representation capabilities. However, these visual representation methods still have obvious limitations: they tend to capture static associations and lack adaptability to dynamic visual information that evolves during the navigation process, which can easily lead to the generation of false associations and thus compromise reliable perception performance in long-term tasks.

[0005] At the level of navigation strategy design, existing technologies mostly use reinforcement learning (RL) for end-to-end training, and some improvement schemes introduce trajectory diffusion models, hierarchical deep reinforcement learning with prediction mechanisms, and other reinforcement learning variants to address long-term planning needs. However, in scenarios with sparse rewards and complex environments, these methods generally face problems such as overestimation of value, ambiguous definition of reward signals, and are prone to cause the intelligent agent to fall into a deadlock or plan an inefficient path. SUMMARY

[0006] In order to solve the above technical problems, the application provides a target-oriented navigation method, which is cooperated by three core modules of multi-teacher distillation visual feature alignment (MDVFA), causal visual graph disentangled factor (CVGD) and semantic hindsight actor-critic (SHDAC), overcomes the problems that the traditional target navigation agent often faces, such as insufficient ability to adapt to dynamic environment, weak scene generalization ability, and difficulty in recovering suboptimal path caused by false association and limited reward signal, and improves the perception accuracy and strategy adaptability of the agent in a complex environment.

[0007] The application adopts the following technical solutions:

[0008] A target-oriented navigation method comprises the following steps:

[0009] S1: a RADIO framework pre-trained by multi-teacher distillation is used as an encoder to encode the image of the observed environment to extract initial visual features; after the initial visual features are reduced in dimension by a convolution layer, the initial visual features are spliced with an embedding vector of a target object to obtain initial input features of cross-modal fusion;

[0010] S2: global disentangled factor estimation and residual elimination operations are performed on the initial input features, and an intervention mechanism based on counterfactual co-occurrence matrix is introduced to adjust the target attention weight, to generate causal alignment fusion features that remove the interference of disentangled factors such as background texture and illumination;

[0011] S3: the causal alignment fusion features are input into a long short-term memory network (LSTM), the semantic graph relationship and the time sequence features are fused by combining a cross-attention mechanism, a double critic architecture is used to calculate a conservative value estimation, and a Gaussian perturbation and a semantic hindsight playback technology are introduced for end-to-end training, to generate a navigation action strategy, so that the agent can complete the identification and arrival of the target object in an unknown indoor 3D environment.

[0012] Further, the RADIO framework pre-trained by multi-teacher distillation in S1 is specifically: the RADIO framework is pre-trained on a large-scale unlabeled image dataset, the prior knowledge of CLIP, DINOv2 and SAM visual basic models is fused through a multi-teacher distillation mechanism, and dense and resolution-adaptive representations are generated.

[0013] Further, the dimension reduction of the initial visual features in S1 is specifically: a 1x1 convolution layer is used to compress the channels of the initial visual features, and the spatial resolution of 7x7 is maintained, so as to retain the hierarchical cues from the edge to the object.

[0014] Further, the global disentangled factor estimation and residual elimination operation in S2 is specifically: the initial input features are spatially mean-pooled to extract global features, and the calculation formula is:

[0015] (2)

[0016] where, , which aggregates global statistics of approximately potential confounders, is the batch dimension, h is the height index of the input feature map in the spatial dimension, and ω is the width index of the input feature map in the spatial dimension; then a projected confounder is generated through linear projection, and the formula is:

[0017] (3)

[0018] In the above formula, and ;

[0019] Step 2.3, the confounder is extended to , and the confounder influence is eliminated through a residual operation, and the deconfused visual feature is defined as the residual of the original feature and the projected confounder:

[0020] (4)

[0021] This subtraction operation approximately realizes the de-biased estimation of the conditional expectation on the target, thereby suppressing the false correlation caused by , eliminating the global statistical bias, while preserving the local features of the encoding space relationship and object boundary.

[0022] Further, the intervention mechanism based on the counterfactual co-occurrence matrix in S2 is specifically: introducing an object class co-occurrence probability matrix , constructing a counterfactual co-occurrence matrix as a weighted mixture of observation statistics and intervention statistics:

[0023] (5)

[0024] In the above formula, is a co-occurrence matrix obtained through class masking or resampling, balances the observation statistics and intervention statistics.

[0025] Further, the generation of the causal alignment fusion feature that removes the interference of confounding factors such as background texture and lighting in S2 is specifically: after splicing the deconfusion feature, the modulated target embedding, the existence indicator, and the action embedding, cross-modal interaction is realized through point-by-point convolution to generate the causal alignment fusion feature F.

[0026] Further, the input of the causal alignment fusion feature into the long short-term memory network in S3 is specifically: taking the causal alignment fusion feature F as the input of the LSTM, combining the hidden state Update the current hidden state, formula is:

[0027] (8)

[0028] In the above formula represents the feature representation after visual distillation alignment and causal deconfusion, is the hidden state of the previous time step.

[0029] Further, the S3 combines the cross-attention mechanism to fuse the semantic graph relationship and the time sequence feature, which is: the query Q of cross-attention is generated by the current hidden state h t Linear projection generation, key K and value V are generated by linear projection of semantic graph representation, the calculation formula is:

[0030] (10)

[0031] In the above formula , and is a learnable matrix; next, calculate the attention distribution and attention representation , the formula is as follows:

[0032] (11)

[0033] By combining with , the fusion representation is obtained:

[0034] (12)

[0035] In the above formula is a learnable projection, represents layer normalization; the fusion vector is input to the two critics, and is optionally input to the actor network.

[0036] Further, the S3 adopts a double critic architecture to calculate the conservative value estimate, which is: the double critic network estimates the state value function , and (s) and (s) through parameters ,

[0037] respectively, and the conservative value is the minimum value of the two, and the calculation formula is:

[0038] In the above formula represents the state of the agent, and are respectively generated by and Parameterization.

[0039] Further, the comprehensive loss function of the S3 end-to-end training is:

[0040] (20)

[0041] wherein is the critic loss, is the actor loss, and is a hyperparameter.

[0042] Compared with the prior art, the beneficial effects of the present application are:

[0043] 1) The present application uses the multi-teacher distillation visual feature alignment module to fuse multiple source prior knowledge such as CLIP, DINOv2 and SAM using the RADIO framework to obtain high-fidelity and generalizable visual feature representation, which can maintain stable recognition ability in complex environments and open vocabulary targets, overcoming the problem that existing methods rely on a limited class set and are difficult to adapt to new object classes;

[0044] 2) The present application introduces a causal visual graph deconfounder mechanism, uses backdoor adjustment and counterfactual co-occurrence modeling to effectively separate global mixed factors such as background texture and lighting from the interference of strategy learning, avoiding false correlations between objects and actions in traditional methods, thereby improving the causal rationality and explainability of strategy learning;

[0045] 3) The semantic hindsight dual actor-critic module proposed by the present application combines double critic conservative value estimation and semantic enhancement mechanism in reinforcement learning, and realizes effective training on sparse reward scenarios through Gaussian perturbation and trajectory playback, significantly alleviating the deficiencies of existing methods in terms of insufficient exploration efficiency and overestimation of value. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 is the overall framework diagram of the target-oriented navigation method of the present application.

[0047] Figure 2 is the causal visual graph deconfounder model proposed by the present application.

[0048] Figure 3 is the trajectory comparison chart of the present application and other advanced target-oriented navigation methods. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0050] The following steps are specifically implemented:

[0051] Step 1, by fusing the complementary prior knowledge of visual-linguistic alignment, the structural self-supervised mechanism and the edge perception boundary clue, a semantic enhanced embedding vector is constructed, and the environmental perception robustness is significantly improved.

[0052] The specific steps are:

[0053] Step 1.1, using the RADIO framework as the basic visual representation model to pre-train on a large-scale unlabeled image dataset. The framework fuses the prior knowledge of CLIP, DINOv2, SAM and other visual basic models through a multi-teacher distillation mechanism, which is used to generate dense and resolution adaptive representations. After the image of the observed environment is processed by the pre-trained RADIO encoder, an initial feature map of 1280x7x7 size is generated, so as to encode the hierarchical clues from edges to objects.

[0054] Step 1.2, in order to adapt the above features to the navigation task, the 1280 channels of the initial feature map are compressed to 64 channels through a 1x1 convolutional layer, while maintaining the spatial resolution of 7x7, to generate a low-dimensional feature map , which retains the hierarchical clues from edges to objects. The local representation is then sent to the causal deconfusion module to estimate and remove the global confounding effects.

[0055] Step 1.3, the reduced visual features are spliced with the target embedding vector (obtained by step 2.5), and the visual semantic and target instruction information are integrated through a cross-modal fusion module to provide high-quality input for subsequent causal deblurring.

[0056] Step 2, by decoupling causal features and confounding factors, eliminating false correlations in visual data, and retaining causal information related to the target object navigation task, the explainability and generalization ability of the strategy are improved.

[0057] The specific steps are:

[0058] Step 2.1, the causal confusion problem is modeled as Figure 2 In visual navigation, the scene features extracted from the original observation are often object-centered components fusion to construct relational representations. However, global confounding variables may pass through paths with , thus introducing spurious dependency relationships when learning policies. To suppress this bias, the backdoor adjustment idea is adopted and global statistics are explicitly modeled to approximately remove the influence of visual features. This bias can cause the agent to misjudge co-occurring objects as having causal relationships, which in turn can be misled in policy learning based on language-specified goals. To suppress this bias, the backdoor adjustment idea is adopted and global statistics are explicitly modeled to approximately remove the influence of visual features. The formal representation (structural equation) is:

[0059] (1)

[0060] In the above formula represents the task-related causal properties, represent interference factors such as background texture, lighting, etc. The goal is to estimate a disentangled representation that is not disturbed by for policy learning.

[0061] Step 2.2, global confounding factor estimation. Spatial mean pooling is performed on the concatenation of the visual features and the target embedding vector in step 1.3 to extract global features, and the calculation formula is:

[0062] (2)

[0063] produces , which aggregates global statistics of the approximate latent confounding factors, is the batch dimension, which is taken as in this embodiment; the projected confounding factor is generated to capture global interference information such as lighting, background texture, etc., and the calculation formula is:

[0064] (3)

[0065] In the above formula and .

[0066] Step 2.3, extend the confounding factor to , eliminate the influence of the confounding factor through a residual operation, and define the disentangled visual feature after disentanglement as the residual of the original feature and the projected confounding factor:

[0067] (4)

[0068] This subtraction operation approximates the conditional expectation on the target and suppresses spurious correlations caused by global statistical biases while preserving local features of encoding spatial relationships and object boundaries.

[0069] Step 2.4, to further suppress spurious associations caused by object co-occurrence statistics and improve relationship reasoning, an intervention-based co-occurrence adjustment is introduced. The object class co-occurrence probability matrix (22 is the number of object classes) is introduced. The counterfactual (intervention) co-occurrence matrix is constructed as a weighted mixture of observed statistics and intervention statistics:

[0070] (5)

[0071] In the above formula is a co-occurrence matrix obtained by class masking or resampling, balances the observed statistics and intervention statistics. This intervention mechanism simulates counterfactual reasoning, thereby improving robustness to dataset-specific biases.

[0072] Step 2.5, target attention weight and target modulation. Given a target one-hot vector , the attention weight is calculated as follows:

[0073] (6)

[0074] It emphasizes functional dependencies (e.g., "cup" near "sink") while suppressing spurious associations. Use as a modulation factor for target embedding which integrates visual, location information and language features.

[0075] Step 2.6, multi-modal feature fusion. Deconfounded features , target embedding , presence indicator and action embedding are fused to obtain fused representations, cross-modal interaction is realized through point-wise convolution, generating causally aligned fusion features to realize cross-modal interaction under observed distribution and intervention distribution:

[0076] (7)

[0077] This will serve as input to the semantic hindsight module. The above CVGD module is end-to-end trainable while constraints can reduce spurious associations caused by landscape statistics or sampling biases, improving causal robustness.

[0078] Step 3, combine the relationship features and temporal features by the dual critic architecture, which combines the graph-based semantic object representation with the cross-attention mechanism, and use semantic hindsight replay and prioritized sampling techniques to achieve optimization.

[0079] The specific steps are as follows:

[0080] Step 3.1, use the fused multi-modal feature extractor to encode the observation state of the current time step , and input it into the recurrent neural network (LSTM) to update the hidden state . The update formula is:

[0081] (8)

[0082] In the above formula, represents the feature representation after visual distillation alignment and causal deconfusion, is the hidden state of the previous time step.

[0083] Step 3.2, after obtaining the LSTM hidden state of each time step, is concatenated with the action of the corresponding time step to form a sequence of state-action pairs , which is used to describe the trajectory of the agent over a period of time. The specific formula for generating the sequence is:

[0084] (9)

[0085] Step 3.3, use the cross-attention mechanism to fuse the semantic graph representation and the hidden state to obtain a semantic enhanced representation . The cross-attention query, key and value are obtained through a learnable linear projection, and the calculation formula is:

[0086] (10)

[0087] In the above formula, , and are learnable matrices. Next, calculate the attention distribution and the attention representation , the formula is as follows:

[0088] (11)

[0089] By combining and , the fused representation is obtained:

[0090] (12)

[0091] In the above formula is a learnable projection, denotes layer normalization. The fusion vector is inputted as two critics and optionally as the input of the actor network.

[0092] Step 3.4, estimate the value function by two independent critics through the dual critic network , to alleviate the overestimation problem in the sparse reward environment. Each critic estimates the state value function , and the conservative value calculation formula is:

[0093] (13)

[0094] In the above formula denotes the state of the agent, and are respectively parameterized by and .

[0095] Step 3.5, in the training process, Gaussian perturbation is injected to the multi-modal embedding e to improve robustness:

[0096] (14)

[0097] In the above formula, e represents the multi-modal embedding, I is the unit matrix, and σ is the perturbation standard deviation (σ=0.2 is set in the embodiment).

[0098] Step 3.6, define the critic loss function as:

[0099] (15)

[0100] (16)

[0101] In the above formula, the time difference target is:

[0102] (17)

[0103] with the reward , the discount factor , the next state , and the episode termination indicator . Here denotes that the target is not backpropagated.

[0104] Step 3.7, define the actor loss function as:

[0105] (18)

[0106] The actor loss function combines the value function and entropy regularization, where is the policy function, represents the entropy of the policy, which encourages exploration, is the entropy coefficient. The advantage function is calculated based on the conservative value:

[0107] (19)

[0108] Step 3.8, the comprehensive loss function is:

[0109] (20)

[0110] and is a hyperparameter that controls the weight of each loss. By optimizing the total loss function , the agent can quickly adapt to multiple tasks and environments, optimize the policy, and effectively transfer experience.

[0111] The following experiments show that the target-oriented navigation method of the present application is effective and feasible, and has certain advantages:

[0112] The test set is input into the established causal deconfusion generation model based on the present application, and compared with the traditional target-oriented navigation method and the ablation experiment is carried out, to analyze the prediction performance of the model;

[0113] The specific steps are as follows:

[0114] (1) The present application uses an open-source indoor three-dimensional simulation scene Ai2Thor as the training and testing environment, which covers multiple typical indoor scenes such as kitchens, living rooms, bedrooms, bathrooms, etc. Each type of scene contains multiple target objects (such as refrigerators, beds, toilets, etc.) and their position information, and is equipped with corresponding scene structure diagrams to express the spatial or functional association between objects. The above data has a high degree of reality restoration, providing good semantic support for subsequent model training. 20 scenes are divided for training, 5 for verification, and 5 for testing. The key is that the test scene is completely excluded from the training set, ensuring that the agent is evaluated in a completely new environment.

[0115] At the beginning of each navigation task, the agent is randomly placed in a certain position in the selected scene and assigned a target object to be located. The agent must find the target autonomously within a predetermined maximum number of steps. The environment is discretized into a uniform grid, with each cell representing a unique reachable position point. The agent's action space is composed of a set of discrete atomic actions, including moving forward 0.25 meters, rotating 45° left and right along the vertical axis, adjusting the camera pitch angle up and down by 30°, and issuing a termination action. This discretized action space design both constrains the search range and improves the efficiency of policy learning. The success of navigation must meet the following conditions: (1) the agent issues a "complete" command within the allowed step limit; (2) there is indeed a target object in the current field of view; (3) the distance between the agent and the target object is within 1.5 meters. If any of the above conditions is not met, the navigation attempt is considered a failure.

[0116] During training, the Adam optimizer with an initial learning rate of 0.0001 is used, and 600,000 training rounds are performed to ensure that the model has sufficient exploration and policy learning ability in different environments. In the evaluation phase, 1000 test rounds are evenly distributed across all scene categories. Gradient updates are not performed during testing, only to evaluate the model's generalization ability. The final checkpoint with the highest success rate on the validation set is selected for testing set evaluation, and its performance indicators are reported to ensure that the results are representative and reproducible.

[0117] (2) Table 1 is a comparison of the proposed target-oriented navigation method with existing advanced navigation methods. The proposed target-oriented navigation method exhibits superior comprehensive performance in multiple evaluation indicators compared to existing advanced navigation techniques. Under the ALL condition, the proposed method achieves a success rate (SR) of 86.76%, a shortest path length (SPL) of 52.47%, a scene alignment efficiency (SAE) of 41.13%, and a distance to success (DTS) of 0.30 meters, all of which are at the optimal level among all comparison methods. Compared with the representative existing methods L-sTDE and AKGVP, the proposed method improves the SR and SPL indicators by 12.58% / 9.24% and 11.45% / 11.21%, respectively, significantly improving the navigation success rate and path efficiency, and reflecting its robustness and strategic effectiveness in complex scene navigation.

[0118] Further, in the long-distance navigation task of defining the optimal path length L≥5, the method of the application still maintains high performance, achieving an SR of 82.36%, an SPL of 50.23%, an SAE of 40.79%, and a DTS of 0.36 meters, respectively. Compared with the above-mentioned comparison methods, the SR / SPL of the method of the application is increased by 18.44% / 8.47% compared with L-sTDE and 12.13% / 5.97% compared with AKGVP, respectively, in this setting. The results show that the navigation method proposed in the application has good generalization ability in short-range and long-range navigation scenarios, and consistently outperforms in success rate, path efficiency, semantic alignment, and end-point accuracy, thus significantly improving the overall performance and application practicality of the navigation system.

[0119] As shown in Figure 3 The trajectory comparison between the method of the application and the advanced navigation method shows that the method of the application has significant advantages over the existing baseline method in complex indoor environments. Especially in dealing with complex scenes such as light changes and mirror reflections, it can still maintain accurate perception, effectively separate target causal features from environmental interference, significantly reduce the influence of false associations on decision-making, and improve the strategy explainability.

[0120] Specifically, in the bathroom scene, a towel is selected as the task target. Due to the mirror highlight reflection and the similarity in color and texture between the towel and the glass shower door, false visual correlations are generated, misleading the baseline model to misidentify the shower door as the target. The causal model of the application can identify and ignore these false correlations and focus on causally related clues. The controlled agent completes the navigation task in only 16 steps, outperforming all comparison methods, demonstrating the significant advantage of the application in causal reasoning accuracy.

[0121] In summary, the experimental results verify the superior navigation performance of the method of the application in complex visual scenes, especially in handling fine-grained target recognition and visual interference disambiguation.

[0122] To verify the effectiveness of each module in the method of the application, an ablation experiment was performed on the model. Under the premise of keeping the network architecture, training plan and auxiliary modules (CVGD and SHDAC) unchanged, only the visual encoder parameters were changed. The experiment compared three configurations: ResNet-50 based on ImageNet pre-training, multi-modal CLIP model and RADIO pre-training model. All variants were trained under the same conditions and evaluated in the full validation set (ALL). As shown in Table 2, the RADIO configuration performed best in the SR, SPL and SAE indicators, and had the lowest DTS indicator, all of which exceeded other pre-training sources. These results show that the visual embedding generated by the RADIO pre-training has richer semantic connotations and more coherent spatial context, thereby achieving more accurate and efficient instruction-guided navigation.

[0123] To evaluate the independent and joint contribution of the CVGD and SHDAC modules, a controlled ablation experiment was conducted. Each variant used the RADIO pre-training feature extractor as the visual backbone network, and systematically disabled the CVGD and SHDAC modules, ensuring that all four combinations were trained and evaluated under the same network architecture, training plan and auxiliary components. Table 3 summarizes the test results on the full validation set (ALL) and long trajectories (L≥5). The results show that removing any key module will cause the navigation performance to decline to varying degrees in all evaluation indicators, further verifying the key role of each module in the overall system performance. Specifically, when the CVGD module is removed, the system's spatial understanding ability and visual-linguistic basic representation ability are significantly weakened, causing the SR and SPL indicators to decline significantly. This result fully demonstrates the core function of the CVGD module in achieving multi-modal feature alignment and improving perception accuracy in complex scenes. Similarly, removing the MART module also significantly reduces the task success rate and navigation efficiency, indicating that the module plays an irreplaceable role in implementing high-level semantic reasoning and adaptive strategy generation in dynamic scenes.

[0124] The model of the application, which integrates the CVGD and SHDAC modules, achieves the best performance in all test environments, especially in complex trajectory scenarios with optimal path length L≥5. Such tasks usually require higher requirements for the long-term planning ability and generalization ability of the agent, and the method of the application still maintains a leading level under such conditions, further verifying the robustness and stability of the model in long-term navigation tasks. The above experimental results not only reveal the technical advantages of the CVGD and SHDAC modules in their respective functional dimensions, but also highlight the synergistic enhancement effect of the two modules in the navigation task, providing a solid foundation for the overall performance improvement of the method of the application.

[0125] Table 1: Comparison results of the embodiment with existing advanced navigation methods

[0126]

[0127] Table 2: Ablation study results for different pre-training schemes

[0128]

[0129] Table 3: Quantitative results of the ablation experiments in the examples

[0130]

[0131] The above description is merely that of the embodiments of this application and is not intended to limit the application. Modification and variation of the application can be effected by those skilled in the art to which this application pertains without departing from the spirit and principle of the application. Any modification, equivalent substitution, improvement, etc. within the spirit and principle of the application are intended to be included in the scope of the claims of the application.

Claims

1. A target-oriented navigation method, characterized in that, Includes the following steps: S1: The RADIO framework, which is pre-trained by multi-teacher distillation, is used as the encoder to encode the image of the observed environment to extract initial visual features; after the initial visual features are reduced in dimension by convolutional layers, they are concatenated with the embedding vector of the target object to obtain the initial input features for cross-modal fusion. S2: Perform global confounding factor estimation and residual elimination operations on the initial input features, and at the same time introduce an intervention mechanism based on the counterfactual co-occurrence matrix to adjust the target attention weights, and generate causal alignment fusion features that remove the interference of background texture, lighting and other confounding factors. S3: Input the causal alignment fusion features into the long short-term memory network, combine the cross-attention mechanism to fuse semantic graph relationships and temporal features, use a dual critic architecture to calculate conservative value estimates, and introduce Gaussian perturbation and semantic hindsight replay techniques for end-to-end training to generate navigation action strategies.

2. The target-oriented navigation method according to claim 1, characterized in that, The RADIO framework pre-trained by multi-teacher distillation in S1 is as follows: The RADIO framework is pre-trained on a large-scale unlabeled image dataset, and the prior knowledge of the CLIP, DINOv2, and SAM visual basic models is fused through a multi-teacher distillation mechanism to generate dense and resolution-adaptive representations.

3. The target-oriented navigation method according to claim 1, characterized in that, The dimensionality reduction of the initial visual features in S1 specifically involves using a 1×1 convolutional layer to compress the channels of the initial visual features while maintaining a 7×7 spatial resolution, thus preserving hierarchical cues from the edge to the object.

4. The target-oriented navigation method according to claim 1, characterized in that, The global confounding factor estimation and residual elimination operation in S2 specifically involves: extracting global features by spatial mean pooling of the initial input features, calculated using the following formula: (2) in, It aggregates global statistics that approximate potential confounding factors. Let h be the height index of the input feature map in the spatial dimension, and ω be the width index of the input feature map in the spatial dimension. Then, a projection confounding factor is generated through linear projection, with the formula: (3) In the above formula and ; Step 2.3, expand the confounding factors to The influence of confounding factors is eliminated through residual operations. The deconfounded visual feature is defined as the residual between the original feature and the projected confounding factor: (4) This subtraction method approximates the desired condition on the operational objective. The biased estimation is used to suppress the biased estimation. The spurious correlations caused by this eliminate global statistical bias while preserving local features of coding spatial relationships and object boundaries.

5. The target-oriented navigation method according to claim 1, characterized in that, The intervention mechanism based on the counterfactual co-occurrence matrix in S2 specifically involves introducing an object category co-occurrence probability matrix. Construct a counterfactual co-occurrence matrix As a weighted mixture of observational statistics and intervention statistics: (5) In the above formula It is a co-occurrence matrix obtained through class masking or resampling. It balances observational statistics and interventional statistics.

6. The target-oriented navigation method according to claim 1, characterized in that, The specific steps in S2 to generate the causal alignment fusion feature that removes interference from background texture, lighting and other confounding factors are as follows: after concatenating the deobfuscated feature, the modulated target embedding, the presence indicator and the action embedding, cross-modal interaction is achieved through pointwise convolution to generate the causal alignment fusion feature F.

7. The target-oriented navigation method according to claim 6, characterized in that, In step S3, inputting the causal alignment fusion feature into the Long Short-Term Memory network specifically involves using the causal alignment fusion feature F as the input to the LSTM, combined with the hidden state from the previous time step. The formula to update the current hidden state is: (8) In the above formula This represents the feature representation after visual distillation alignment and causal deconfusion. This is the hidden state of the previous time step.

8. The target-oriented navigation method according to claim 1, characterized in that, The "combining semantic graph relationships and temporal features with the cross-attention mechanism" in S3 specifically means that the cross-attention query Q is determined by the current hidden state h of the LSTM. t Linear projection generation: Key K and value V are generated by linear projection from the semantic graph representation. The calculation formula is as follows: (10) In the above formula , and It is a learnable matrix; next, we calculate the attention distribution. and attention representation The formula is as follows: (11) By and By combining these elements, a fusion representation can be obtained: (12) In the above formula It is a learnable projection. Representation layer normalization; fusion vector As input for two critics, and optionally as input for the actor network.

9. The target-oriented navigation method according to claim 1, characterized in that, The conservative value estimation using a dual-critic architecture in S3 specifically involves the dual-critic network calculating the conservative value estimate using parameters. , Estimate the state value function (s) and (s), the conservative value is the minimum of the two, and the calculation formula is: (13) In the above formula Indicates the state of the agent. and respectively by and Parameterization.

10. The target-oriented navigation method according to claim 1, characterized in that, The comprehensive loss function for end-to-end training in S3 is: (20) in For the critics' loss, For the loss of the actors, and It's a hyperparameter.

Images