Target-driven navigation method and device based on context awareness and imitation learning

Abstract

The invention discloses a target-driven navigation method and device based on context awareness and imitation learning, and the method comprises the steps: recognizing an object instance of interest in an image based on a target detector DETR, and constructing an object graph; based on context perception graph reasoning, in the navigation process, dynamic context information such as images, actions and memories serves as guidance, object features are projected to hyperplanes of corresponding contexts by means of a TransH method at each time step, the object relation is dynamically learned, and an intelligent agent can better understand the complex environment. Based on visual representation of Transform, visual features and graph features are fused, and spatial semantic information of the environment is better captured. Based on generative adversarial imitation learning, a new dynamic reward function is designed, and an intelligent agent is helped to avoid a deadlock state in combination with environment rewards. Based on a standard asynchronous dominant actor-commentator algorithm, an effective navigation strategy is trained by using a new reward function, and the navigation success rate and efficiency of the intelligent agent in an unfamiliar environment are improved.

Description

Technical Field

[0001] This invention relates to the field of robot visual navigation, and more specifically, to a target-driven navigation method and apparatus based on context awareness and imitation learning. Background Technology

[0002] Target-driven navigation is a fundamental task in the fields of robotics and artificial intelligence that has received widespread attention in recent years. It aims to guide intelligent agents to find specific target objects in unfamiliar environments based on first-person visual observation.

[0003] Goal-driven navigation requires agents not only to recognize objects but also to reason about their possible locations. Similar to human navigation in unfamiliar environments, the agent needs to memorize the arrangement of objects and construct an object map based on this information to quickly locate the target object. However, due to the complexity of object placement, the agent may fail and get stuck during the search process. Therefore, goal-driven navigation requires agents to possess two key capabilities: first, to learn information-rich visual representations and understand the relationships between objects in the environment to enhance environmental perception; and second, to learn robust navigation strategies to ensure that the agent can effectively avoid obstacles and find the target in complex and unfamiliar environments.

[0004] In implementing this invention, the inventors discovered that visual representation learning is a crucial step in an agent's understanding of its environment during goal-driven navigation. Existing methods primarily enrich visual representations by constructing object graphs. However, during graph reasoning, the constantly changing contextual information between the agent and its environment during navigation is often overlooked. Contextual interaction information, including visual observation, actions, and memories, influences the dynamic changes in object relationships, affecting navigation success rates. Furthermore, navigation strategy learning, another critical step in goal-driven navigation, is primarily based on reinforcement learning in existing methods. However, the reward design suffers from non-discriminatory rewards; the agent typically receives the same negative reward at every step. This singular reward structure fails to provide effective feedback, making the agent prone to local optima or deadlock states, thus impacting navigation efficiency. Summary of the Invention

[0005] The purpose of this invention is to provide a target-driven navigation method and apparatus based on context awareness and imitation learning, so as to solve the problems existing in the prior art and improve the navigation success rate and efficiency of intelligent agents in unfamiliar environments.

[0006] To achieve the above objectives, a first aspect of the present invention provides a target-driven navigation method based on context awareness and imitation learning, comprising: Based on the DETR object detector, object instances of interest in the input image are identified, and an object map is constructed. Based on the constructed object graph, the TransH method is used to dynamically project object features onto a hyperplane defined by the current image, the previous action, and the previous hidden state. The object relationships perceived by the image, the object relationships perceived by the action, and the object relationships perceived by the memory are calculated, and the final object relationships are obtained through weighted fusion. The graph features are obtained by graph encoding the final object relationships through a graph attention layer. The Transformer architecture is used to fuse visual features and graph features. The correspondence between the two is extracted through a multi-head attention mechanism to generate a visual representation for navigation. The visual features are obtained by the residual neural network model after feature extraction of the input image. The obtained visual representation is used as the state. A generator G and a discriminator D are constructed. A dynamic reward signal is designed based on the discriminator D and combined with the environmental reward function to form a new reward function. Effective navigation strategies are trained based on visual representations and novel reward functions to guide agents in finding target objects in unfamiliar environments.

[0007] In one implementation, the nodes of the object graph represent objects, and the edges represent the relationships between objects. The features contained in the nodes are composed of object features, normalized bounding boxes, confidence scores, semantic labels, and one-hot encoded vectors of the target object.

[0008] In one implementation, based on the constructed object graph, the TransH method is used to dynamically project object features onto a hyperplane defined by the current image, the previous action, and the previous hidden state, calculating image-aware object relationships, action-aware object relationships, and memory-aware object relationships, including: The TransH method is used to project each object feature onto the hyperplane defined by the global features of the current image obtained by global average pooling. The connection weights between objects are calculated by scaling dot product attention and softmax operation to obtain the object relationships perceived in the image. Each object feature is projected onto a hyperplane defined by the previous action. The connection weights between objects are calculated by scaling dot product attention and softmax operation to obtain the object relationships for action perception. Each object feature is projected onto a hyperplane defined by the previous hidden state. The connection weights between objects are calculated by scaling dot product attention and softmax operation to obtain the object relationships perceived by memory.

[0009] In one implementation, graph features are obtained by graph encoding the final object relationships through a graph attention layer, including: The graph attention layer uses the ReLU activation function and weight matrix to perform a weighted summation of the features of a node's neighboring nodes, updating the node features to obtain graph features.

[0010] In one implementation, a Transformer architecture is used to fuse visual features and graph features, and a multi-head attention mechanism is used to extract the correspondence between the two to generate a visual representation for navigation, including: The graph features are mapped by a feedforward neural network, and then processed by residual connections and normalization to obtain normalized features. Position embeddings are introduced into visual features and flattened. First, multi-head attention is used to refine the visual features, and then another multi-head attention is used to converge the graph feature information related to the visual features, ultimately generating a visual representation for navigation.

[0011] In one implementation, the new reward function is:

[0012] in The reward function is obtained from generative adversarial learning. For state-action pairs, For the environmental reward function, As a discount factor, Determined by the output of discriminator D, its range is [0,1]. When the value is close to 1, it indicates that the agent's behavior is similar to that of an expert; conversely, when the value is close to 1, it indicates that the agent's behavior is similar to that of an expert. When the value is close to 0, it indicates that the agent's behavior differs from that of the expert.

[0013] In one implementation, an effective navigation policy is trained based on visual representation and a novel reward function to guide the agent to find target objects in unfamiliar environments, including: The visual representation obtained from the Transformer is input into the Long Short-Term Memory network to obtain the hidden state; The hidden state of the LSTM is input into the asynchronous advantage actor-critic algorithm, and the resulting new reward function is used to train an effective navigation policy to guide the agent to find target objects in unfamiliar environments. During the training process, the loss used includes action loss, value loss and generative adversarial imitation loss.

[0014] Based on the same inventive concept, a second aspect of the present invention provides a target-driven navigation device based on context awareness and imitation learning, comprising: The object graph construction module is used to identify instances of objects of interest in the input image based on the DETR object detector and construct an object graph. The context-aware graph reasoning module is used to dynamically project object features onto a hyperplane defined by the current image, the previous action, and the previous hidden state based on the constructed object graph using the TransH method. It calculates the object relationships perceived by the image, the object relationships perceived by the action, and the object relationships perceived by the memory, and obtains the final object relationships through weighted fusion. The graph encoder module is used to encode the final object relationships using a graph attention layer to obtain graph features; The Transformer-based visual representation module is used to fuse visual features and graph features using the Transformer architecture. It extracts the correspondence between the two through a multi-head attention mechanism to generate a visual representation for navigation. The visual features are obtained by extracting features from the input image using a residual neural network model. The Generative Adversarial Imitation Learning module is used to take the obtained visual representation as the state, construct a generator G and a discriminator D, design a dynamic reward signal based on the discriminator D, and combine it with the environmental reward function to form a new reward function. The navigation policy learning module is used to train effective navigation policies based on visual representations and novel reward functions to guide agents to find target objects in unfamiliar environments.

[0015] Based on the same inventive concept, a third aspect of the present invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, provides the target-driven navigation method based on context awareness and imitation learning described in the first aspect.

[0016] Based on the same inventive concept, a fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the target-driven navigation method based on context awareness and imitation learning described in the first aspect.

[0017] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows: (1) Context-aware graph reasoning: By considering the influence of images, actions and memories on the relationships between objects, CGI can dynamically adjust the relationships between objects, enabling the agent to better understand the complex environment.

[0018] (2) Graph encoder: Using graph attention can help the agent focus on valuable object relationships and narrow the search scope.

[0019] (3) Visual representation based on Transformer: By fusing visual features and graph features, the generated visual representation can better capture the spatial semantic information of the environment.

[0020] (4) Generative adversarial learning: GAIL provides clear navigation guidance to the agent through a dynamic reward function, helping it learn effective navigation strategies and avoid deadlock. Attached Figure Description To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart of a target-driven navigation method based on context awareness and imitation learning provided in an embodiment of the present invention; Figure 2 A system framework diagram of the target-driven navigation method based on context awareness and imitation learning provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of projection in context graph-aware reasoning in an embodiment of the present invention; Figure 4 This is a framework diagram of the visual representation based on Transformer in an embodiment of the present invention; Figure 5 This is a block diagram of a target-driven navigation device based on context awareness and imitation learning provided in an embodiment of the present invention. Detailed Implementation

[0022] This invention provides a target-driven navigation method and apparatus based on context-aware graph reasoning and generative adversarial learning. The main concepts are as follows: Based on the DETR object detector, instances of interest in images are identified, and an object graph is constructed. Based on context-aware graph reasoning, dynamic contextual information such as images, actions, and memories guide navigation. At each time step, the TransH method is used to project object features onto the corresponding context hyperplane, dynamically learning object relationships and enabling the agent to better understand complex environments. Based on Transformer-based visual representation, visual features and graph features are fused to better capture the spatial semantic information of the environment. Based on generative adversarial learning, a new dynamic reward function is designed, combined with environmental rewards, to help the agent avoid deadlock states. Based on the standard asynchronous advantage actor-critic algorithm, the new reward function is used to train an effective navigation strategy, improving the success rate and efficiency of navigation in unfamiliar environments.

[0023] Example 1 This embodiment provides a target-driven navigation method based on context awareness and imitation learning. Please refer to [link to relevant documentation]. Figure 1 ,include: S1: Based on the DETR object detector, identify instances of objects of interest in the input image and construct an object map.

[0024] In the object graph, nodes represent objects, and edges represent relationships between objects. The features contained in a node are composed of object features, normalized bounding boxes, confidence scores, semantic labels, and one-hot encoded vectors of the target object.

[0025] In practice, a DETR detector is used to process the input RGB image and identify all objects of interest. For each detected object, its feature vector is extracted, including object features. Bounding box Confidence score and the semantic label with the highest confidence ,in This represents the total number of objects of interest in the image. Furthermore, one-hot encoding of the target object is introduced. The above features are then concatenated to form the final feature vector for each object. Each detected object is treated as a node, forming a node set. Construct a fully connected undirected object graph ,in It is a set of edges that contains all possible node pairs (excluding self-loops).

[0026] S2: Based on the constructed object graph, the TransH method is used to dynamically project object features onto the hyperplane defined by the current image, the previous action, and the previous hidden state. The object relationships perceived by the image, the object relationships perceived by the action, and the object relationships perceived by the memory are calculated, and the final object relationships are obtained through weighted fusion.

[0027] In one implementation, the calculation of object relationships can be achieved in the following way: The TransH method is used to project each object feature onto the hyperplane defined by the global features of the current image obtained by global average pooling. The connection weights between objects are calculated by scaling dot product attention and softmax operation to obtain the object relationships perceived in the image. Each object feature is projected onto a hyperplane defined by the previous action. The connection weights between objects are calculated by scaling dot product attention and softmax operation to obtain the object relationships for action perception. Each object feature is projected onto a hyperplane defined by the previous hidden state. The connection weights between objects are calculated by scaling dot product attention and softmax operation to obtain the object relationships perceived by memory.

[0028] Specifically, please see Figure 3 This is a projection diagram in the context graph-aware reasoning of an embodiment of the present invention.

[0029] Step S21, calculate the object relationships in the image perception at the current time step. Feature maps of the input image Perform global average pooling to obtain global image features. ,like Figure 2 As shown, this invention uses the TransH method to project each object feature onto a graph composed of... In the defined hyperplane:

[0030]

[0031] in, It is the normal vector of the hyperplane. and It is an object and Features and It is to make objects and Feature projection to Features defined after the hyperplane. After projection, this invention uses scaled dot product attention to compute the object. and Relationships between objects perceived in images :

[0032] in, It is a dimension of object characteristics.

[0033] Step S22: Calculate the object relationships for motion perception, and assign the actions from the previous time step to the objects in the previous time step. As context, such as Figure 2 As shown, this invention uses the TransH method to project each object feature onto a graph composed of... In the defined hyperplane:

[0034]

[0035] in, It is the normal vector of the hyperplane. and Object and Feature projection to Features defined after the hyperplane. After projection, this invention also uses scaled dot product attention to compute the object. and Relationship between objects for motion perception :

[0036] Step S23: Calculate the object relationships perceived by memory, using the previous hidden state of LSTM. As a memory context, such as Figure 2 As shown, this invention uses the TransH method to project each object feature onto a graph composed of... In the defined hyperplane:

[0037]

[0038] in, It is the normal vector of the hyperplane. and Object and Feature projection to Features defined after the hyperplane. After projection, this invention also uses scaled dot product attention to compute the object. and Relationship between objects perceived through memory :

[0039] Step S24: After calculating the object relationships perceived by the image, the object relationships perceived by the action, and the object relationships perceived by the memory, this invention uses the previous hidden state of LSTM. To calculate the weights of image, action, and memory context. , and And normalize, so that The final object relationships are obtained by weighting and summing the relationships of the three contexts: .

[0040] S3: Graph features are obtained by graph encoding the final object relationships through a graph attention layer.

[0041] The graph attention layer uses the ReLU activation function and weight matrix to perform a weighted summation of the features of the neighboring nodes of a node, thereby updating the node features and obtaining the graph features.

[0042] In the specific implementation process, based on the context-aware object relationships obtained through reasoning in step S2, this invention designs a graph attention layer as a graph encoder to encode the features of the object graph, thereby obtaining graph features and focusing on the most relevant object relationships for navigation:

[0043] in, It is an object exist Features of the layer It is an object At adjacent nodes in the object graph, Represents the ReLU activation function. express Layer weight matrix.

[0044] S4: The Transformer architecture is used to fuse visual features and graph features. The correspondence between the two is extracted through a multi-head attention mechanism to generate a visual representation for navigation. The visual features are obtained by the residual neural network model after feature extraction of the input image.

[0045] S4 can be implemented in the following way: The graph features are mapped by a feedforward neural network, and then processed by residual connections and normalization to obtain normalized graph features. Position embeddings are introduced into visual features and flattened. First, multi-head attention is used to refine the visual features, and then another multi-head attention is used to converge the graph feature information related to the visual features, ultimately generating a visual representation for navigation.

[0046] In the specific implementation process, such as Figure 4 As shown, the visual representation based on Transformer fuses visual features (image features) and the graph features obtained in step S3. The graph features are then mapped through a feedforward neural network to obtain... And residual connections and normalization are applied. In the visual decoder, in order to emphasize spatial location information, visual features are... Introducing position embedding :

[0047]

[0048] in, It is a channel dimension index, It is the first in position embedding The components of each dimension are encoded using a sine function to encode the row and column coordinates of the visual features. , It is the first in position embedding The components of each dimension are encoded using a cosine function to encode the row and column coordinates of the visual features. .

[0049] The visual features after introducing position embedding are flattened to obtain This invention first uses multi-head attention to extract visual features, and then uses another multi-head attention to gather graph feature information related to the visual features to obtain a richer visual representation.

[0050] S5: Using the obtained visual representation as the state, a generator G and a discriminator D are constructed. A dynamic reward signal is designed based on the discriminator D and combined with the environmental reward function to form a new reward function. In the specific implementation process, the visual representation obtained in step S4 is used as the state. Generative adversarial learning is designed, including a generator G and a discriminator D. G is the agent's standard A3C navigation policy network, responsible for generating navigation actions based on the current state. The discriminator's role is to distinguish the trajectories generated by the generator. and expert trajectory By inputting state-action pairs Output a probability value , representing the probability that the generated action is an expert action. This invention uses Dijkstra's algorithm to generate the shortest path from the initial position to the target object as the expert trajectory. The expert trajectory includes a series of state-action pairs. The generator is trained using a standard A3C navigation policy network, and the discriminator is updated using the following gradients to distinguish between non-expert and expert actions:

[0051] in, Discriminator parameters gradient, It is the discriminator function. Represents the generated trajectory The expected gradient of the logarithm of the probability of being judged as a non-expert. Representing expert trajectory The gradient expectation of the logarithm of the probability of being judged as an expert.

[0052] This invention designs a dynamic imitation reward function. and environmental rewards Combined, a new reward function is formed. Solving the problem of non-discriminatory rewards:

[0053]

[0054] in, It provides a positive imitation reward signal, indicating the degree of similarity between the generated actions and those of the expert, encouraging the navigation strategy to move closer to the expert strategy. It is a discount factor used to balance the effects of imitation rewards and environmental rewards.

[0055] S6: Train an effective navigation strategy based on visual representation and a novel reward function to guide the agent to find target objects in unfamiliar environments.

[0056] S6 can be implemented in the following way: The visual representation obtained from the Transformer is input into the Long Short-Term Memory network to obtain the hidden state; The hidden state of the LSTM is input into the asynchronous advantage actor-critic algorithm, and the resulting new reward function is used to train an effective navigation policy to guide the agent to find target objects in unfamiliar environments. During the training process, the loss used includes action loss, value loss and generative adversarial imitation loss.

[0057] In specific implementation, this invention obtains the current state representation through LSTM. The navigation policy is trained using the standard A3C reinforcement learning algorithm, which includes an action network and a value network, and a new reward function designed in step S5 is used. Training is then performed. The loss functions for actions and values ​​are as follows:

[0058]

[0059] in, For the parameters of the action network, For the parameters of the value network, Indicates the state The following parameters Generate Actions The probability, For the current time step, For the total time step, As a discount factor, For time step The reward This is a state value function.

[0060] After introducing Generative Adversarial Imitation Learning (GAIL), the total loss function is expressed as a weighted sum of the action loss, value loss, and GAIL loss:

[0061] in, , and The weights for action loss, value loss, and GAIL loss are corresponding to the following: GAIL loss is defined as follows:

[0062] Among them, the loss function The goal is to classify the generated trajectory as non-expert behavior, and the loss function is... The goal is to identify expert trajectories as expert behavior.

[0063] Example 2 Based on the same inventive concept, this embodiment discloses a target-driven navigation device based on context awareness and imitation learning. Please refer to [link to relevant documentation]. Figure 5 ,include: The object graph construction module 101 is used to identify instances of objects of interest in the input image based on the object detector DETR and construct an object graph. The context-aware graph reasoning module 102 is used to dynamically project object features onto a hyperplane defined by the current image, the previous action, and the previous hidden state based on the constructed object graph using the TransH method. It calculates the object relationships perceived by the image, the object relationships perceived by the action, and the object relationships perceived by the memory, and obtains the final object relationships through weighted fusion. Graph encoder module 103 is used to perform graph encoding on the final object relationships through a graph attention layer to obtain graph features; The Transformer-based visual representation module 104 is used to fuse visual features and graph features using the Transformer architecture, extract the correspondence between the two through a multi-head attention mechanism, and generate a visual representation for navigation. The visual features are obtained by the residual neural network model after feature extraction of the input image. Generative adversarial learning module 105 is used to take the obtained visual representation as the state, construct a generator G and a discriminator D, design a dynamic reward signal based on the discriminator D, and combine it with the environmental reward function to form a new reward function. The navigation policy learning module 106 is used to train an effective navigation policy based on visual representation and a novel reward function to guide the agent to find target objects in unfamiliar environments.

[0064] Because of the apparatus used in the method of Embodiment 2 of the present invention, those skilled in the art can understand the specific structure and variations of the apparatus based on the method described in Embodiment 1 of the present invention, and therefore will not be described again here. All apparatuses used in the method of Embodiment 1 of the present invention fall within the scope of protection of the present invention.

[0065] Example 3 Based on the same inventive concept, the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the target-driven navigation method based on context awareness and imitation learning of Embodiment 1.

[0066] Since the computer-readable storage medium described in Embodiment 3 of this invention is the same computer-readable storage medium used in implementing the target-driven navigation method based on context awareness and imitation learning in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this computer-readable storage medium based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All computer-readable storage media used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.

[0067] Example 4 Based on the same inventive concept, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in Embodiment 1.

[0068] Since the computer device described in Embodiment 4 of this invention is the same computer device used to implement the target-driven navigation method based on context awareness and imitation learning in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this computer device based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All computer devices used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.

[0069] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0070] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0071] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various modifications and variations to the embodiments of the invention without departing from the spirit and scope of the invention. Thus, if these modifications and variations of the embodiments of the invention fall within the scope of the claims of the invention and their equivalents, the invention also intends to include these modifications and variations.

Claims

1. A goal-driven navigation method based on context-aware and imitation learning, characterized in that, Comprise: Based on the target detector DETR, the object instances of interest in the input image are identified, and an object graph is constructed; Based on the constructed object graph, the object features are dynamically projected onto the hyperplane defined by the current image, the previous action and the previous hidden state using the TransH method, the image-perceived object relationship, the action-perceived object relationship and the memory-perceived object relationship are calculated, and the final object relationship is obtained by weighted fusion; The final object relationship is graph-encoded by a graph attention layer to obtain a graph feature; The visual feature is obtained by a residual neural network model after feature extraction of the input image, and the visual feature and the graph feature are fused by a Transformer architecture, and the corresponding relationship between the two is extracted by a multi-head attention mechanism to generate a visual representation for navigation; The obtained visual representation is taken as a state, a generator G and a discriminator D are constructed, a dynamic reward signal is designed based on the discriminator D, and is combined with an environment reward function to form a new reward function; Based on the visual representation and the new reward function, an effective navigation strategy is trained to guide the agent to find the target object in an unfamiliar environment.

2. The context-aware and imitation learning based goal-driven navigation method of claim 1, wherein, The nodes of the object graph represent objects, and the edges represent the relationships between objects, wherein the features contained in the nodes are spliced from object features, normalized bounding boxes, confidence scores, semantic labels and one-hot encoding vectors of target objects.

3. The context-aware and imitation learning based goal-driven navigation method of claim 1, wherein, Based on the constructed object graph, the object features are dynamically projected onto the hyperplane defined by the current image, the previous action and the previous hidden state using the TransH method, the image-perceived object relationship, the action-perceived object relationship and the memory-perceived object relationship are calculated, including: Each object feature is projected onto the hyperplane defined by the global feature of the current image obtained by global average pooling using the TransH method, the connection weight between objects is calculated by scaled dot-product attention and softmax operation, and the image-perceived object relationship is obtained; Each object feature is projected onto the hyperplane defined by the previous action, the connection weight between objects is calculated by scaled dot-product attention and softmax operation, and the action-perceived object relationship is obtained; Each object feature is projected onto the hyperplane defined by the previous hidden state, the connection weight between objects is calculated by scaled dot-product attention and softmax operation, and the memory-perceived object relationship is obtained.

4. The context-aware and imitation learning based goal-driven navigation method of claim 1, wherein, The final object relationship is graph-encoded by a graph attention layer to obtain a graph feature, including: The graph attention layer uses a ReLU activation function and a weight matrix to perform weighted summation on the features of the neighbor nodes of the node, updates the node features, and obtains the graph features.

5. The context-aware and imitation learning based goal-driven navigation method of claim 1, wherein, The visual feature and the graph feature are fused by a Transformer architecture, and the corresponding relationship between the two is extracted by a multi-head attention mechanism to generate a visual representation for navigation, including: The graph feature is mapped by a feedforward neural network, and a residual connection and normalization are applied for processing to obtain normalized features; The position embedding is introduced into the visual feature and flattened, the visual feature is refined using a multi-head attention first, and then another multi-head attention is used to gather the graph feature information related to the visual feature, and finally a visual representation for navigation is generated.

6. The context-aware and imitation learning based goal-driven navigation method of claim 1, wherein, The new reward function is: wherein is the reward function obtained from the generative adversarial imitation learning, is a state-action pair, is the environment reward function, is a discount factor, is determined by the output of the discriminator D, whose value range is [0, 1], when is close to 1, indicating that the behavior of the agent is similar to the expert behavior, and vice versa, when is close to 0, indicating that the behavior of the agent is different from the expert behavior.

7. The context-aware and imitation learning based goal-driven navigation method of claim 1, wherein, An effective navigation strategy is trained based on the visual representation and the new reward function to guide the agent to find the target object in an unfamiliar environment, including: The visual representation obtained from the Transformer is input into a long short-term memory network to obtain a hidden state; The hidden state of the LSTM is input into an asynchronous advantage actor-critic algorithm, and an effective navigation strategy is trained based on the new reward function obtained to guide the agent to find the target object in an unfamiliar environment, wherein the loss used in the training process includes an action loss, a value loss, and a generative adversarial imitation loss.

8. A goal-driven navigation apparatus based on context-awareness and imitation learning, characterized by, It includes: An object graph construction module for identifying object instances of interest in an input image based on a target detector DETR and constructing an object graph; A context-aware graph reasoning module for dynamically projecting object features onto a hyperplane defined by the current image, the previous action, and the previous hidden state using the TransH method based on the constructed object graph, calculating the image-aware object relationship, the action-aware object relationship, and the memory-aware object relationship, and obtaining the final object relationship through weighted fusion; A graph encoder module for graph encoding the final object relationship through a graph attention layer to obtain a graph feature; A Transformer-based visual representation module for fusing visual features and graph features using a Transformer architecture, extracting the corresponding relationship between the two through a multi-head attention mechanism, and generating a visual representation for navigation, wherein the visual features are obtained by feature extraction on the input image using a residual neural network model; A generative adversarial imitation learning module for using the obtained visual representation as a state, constructing a generator G and a discriminator D, designing a dynamic reward signal based on the discriminator D, and combining it with an environment reward function to form a new reward function; A navigation strategy learning module for training an effective navigation strategy based on the visual representation and the new reward function to guide the agent to find the target object in an unfamiliar environment.

9. A computer-readable storage medium, characterized in that, A computer program is stored thereon, which is executed by a processor to implement the target-driven navigation method based on context awareness and imitation learning according to any one of claims 1-7.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the target-driven navigation method based on context awareness and imitation learning according to any one of claims 1-7.

Images