Method and system for using visual conversion network to assist navigation in intelligent navigation scenarios

Abstract

The present invention provides a method and system for using a visual conversion network to assist navigation in intelligent navigation scenarios. The method uses a visual converter network to encode local and global spatial features in a graphless navigation framework. The method uses the visual converter network to infer the optimal next action for navigation based on the local target information in the current target view, the global information between the current view and the previous frame view, and the knowledge graph to complete the visual navigation task. The present invention proposes a new visual spatial sequence information conversion network SSTNet to extract information feature representations for visual navigation. It encodes local and global targets, actions, memory, and knowledge graph information, and models it as an effective distribution of reinforcement learning state representations, so that the intelligent agent is not affected by changes in scene layout in new indoor navigation environments. The navigation framework has rich navigation semantic knowledge and is successfully generalized to new scenarios.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence robots, and in particular relates to a solution for utilizing visual conversion networks to assist navigation in intelligent navigation scenarios. Background Art

[0002] In visual navigation, an agent needs to extract environmental features to localize itself and leverage spatial relationships across time steps to navigate toward a target. Furthermore, the agent needs to have a good understanding of changes in environmental structure between observation frames or across scenes. The task of visual navigation is to guide the agent to find a given target in a three-dimensional environment based on its current observations. This task can be evaluated by the agent's ability to understand its surroundings and navigate efficiently and safely to the designated target based solely on input from camera sensors. Successful target locating in unfamiliar environments requires strong environmental perception and cross-scenario adaptability. Improving these capabilities requires two key aspects. First, the navigation model enables the agent to perceive large-scale semantic concepts distributed across high-dimensional spaces. This includes understanding the distribution of useful images, semantics, and spatial structures related to navigation goals in unknown environments, thereby improving generalization. Second, the agent needs to memorize sequential information from experience replay in reinforcement learning to improve search efficiency. Current visual perception relies solely on deep convolutional neural networks, such as ResNet and VGG neural networks, to capture global information about the environment. However, navigation environments are complex and diverse, with numerous areas of strong lighting, weak textures, and obstacles, and their structure is also dynamically changing. Representing reinforcement learning states with this global information can introduce a large amount of redundant information and easily lead to dimensionality explosion.

[0003] Recent reinforcement learning visual navigation algorithms have mostly used pre-trained global image features of the environment combined with semantic information about the target to represent the state of the environment. In dynamic and complex navigation environments, using global image features as input introduces a large amount of redundant information, making it difficult to adapt to environmental changes and resulting in low data efficiency. Summary of the Invention

[0004] The present invention provides a solution for using a visual transformation network to assist navigation in intelligent navigation scenarios. For the emerging field of visual navigation, a reinforcement learning state representation method for extracting global and local features of the environment through the visual transformation network is designed and implemented, thereby improving the generalization ability of visual navigation in unknown environments.

[0005] The present invention provides a method for assisting navigation using a visual conversion network in an intelligent navigation scenario. The method uses a graphless navigation framework that uses a visual converter network to encode local and global spatial features. The method performs inference navigation based on the target local information in the current target view, the global information between the current view and the previous frame view, and the knowledge graph to perform the next optimal action, thereby completing the visual navigation task.

[0006] Moreover, the implementation process includes the following steps,

[0007] Step 1: First, use the pre-trained Faster-RCNN object detector to obtain the objects that appear in each frame of the scene through random walks. This co-occurrence relationship is defined as an edge, and a knowledge graph is generated and stored. When a relationship appears more than a preset number of times, it is determined to be common sense.

[0008] Step 2: In the spatial attention module (SAF) of the visual transformer network, the DETR object detector is used to obtain a local feature vector by splicing the local information in the environment. This local feature vector is then fused with the navigation action features of the previous navigation action, the hidden state features, and the graph features to obtain a spatial encoder.

[0009] Step 3: In the sequence conversion network STNet of the visual converter network, for global features, the global image information extracted by the ResNet residual network is used as a sequence decoder, and local features are used as part of the multi-head attention to obtain a global feature vector. Then, the Leaky ReLU activation function is used. The features obtained by the activation function are used as LSTM inputs to store the navigation round experience. The obtained sequence feature descriptor is input as the representation of the reinforcement learning state, and the self-attention mechanism is used to obtain useful information within the current state. Finally, the action strategy and value function are formed.

[0010] Step 4: Based on the knowledge graph, a Markov logic network is used to perform action reasoning and optimize the navigation strategy. In the graph reasoning module GAM, a prediction model is used to consider the spatial context and sequence context and decide the next action.

[0011] Moreover, when generating the knowledge graph, in order to learn the relationship between all objects, the 2D detection box coordinates and confidence scores of all objects are first located and combined into the detection features of the nodes; then the depth map is used to extract deep features through the VGG convolutional neural network, and then spliced through the CNN convolutional neural network to obtain the 3D positioning bounding box in the scene graph.

[0012] Moreover, in the spatial attention module SAF, the input is encoded as local information, and the position features of DETR are added for positioning. After feature decomposition, the environmental features are extracted through multi-head self-attention. Then, the previous input encoding is used to enhance and restore the features through addition and normalization operations. The decoder features are output through forward propagation and addition & normalization operations, and finally a vector matrix is formed for forward propagation.

[0013] Moreover, in the sequence transformation network STNet, the input is the global information of the environment extracted by ResNet. After feature decomposition, the local features are mapped to the global graph through masked self-attention, and the information is restored using addition and normalization operations. The encoder features are spliced with the features obtained in the previous step, and the decoder features are output through linear layers and normalization.

[0014] Moreover, the A3C architecture is adopted as the navigation engine. The navigation strategy takes visual embedding and spatial embedding as input and outputs action distribution and action value. To optimize the strategy, GAM is used to help determine the best next action, which is navigated according to the A3C action distribution.

[0015] On the other hand, the present invention provides a system for utilizing visual conversion network to assist navigation in an intelligent navigation scenario, for implementing the method for utilizing visual conversion network to assist navigation in an intelligent navigation scenario as described above.

[0016] Furthermore, the system includes a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the method of utilizing visual conversion network to assist navigation in an intelligent navigation scenario as described above.

[0017] Alternatively, it includes a readable storage medium having a computer program stored thereon, and when the computer program is executed, it implements the method for assisting navigation by using a visual conversion network in an intelligent navigation scenario as described above.

[0018] The main innovations of the present invention are as follows:

[0019] 1. This paper designs a novel visual-spatial sequence transformation network (SSTNet) to extract informative feature representations for visual navigation. It encodes local and global goals, actions, memory, and knowledge graph information and models them as an efficient distribution of reinforcement learning state representations. This enables intelligent agents to navigate novel indoor environments unaffected by layout changes. This navigation framework incorporates rich navigation semantics and successfully generalizes to new scenarios.

[0020] 2. SSTNet introduces a spatially aware and sequence-aware descriptor, and implements state-action mapping with interpretable visualization.

[0021] 3. Feature extraction is performed on the observed image in the scene, and semantic representation is performed on the target word vector. The image features, semantic features, and spatial information features are then combined into a neural network for memory storage and passed to reinforcement learning for state representation, enabling complete navigation.

[0022] 4. The present invention uses prior knowledge to infer the shortest path based on the target position relationship in the navigation scene, thereby providing reinforcement learning optimization decisions, guiding reinforcement learning navigation, and improving navigation generalization capabilities.

[0023] The solution of the present invention is simple and convenient to implement and has strong practicality. It solves the problems of low practicality and inconvenience in actual application existing in related technologies, can improve user experience, and has important market value. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of the SSTNet motivation proposed in an embodiment of the present invention.

[0025] Figure 2 This is a diagram of the overall framework of using SSTNet visual navigation in an embodiment of the present invention.

[0026] Figure 3 This is a flowchart of establishing a knowledge graph according to an embodiment of the present invention.

[0027] Figure 4 2 is a schematic diagram of the structure of SAF as an encoder and STNet as an encoder in an embodiment of the present invention.

[0028] Figure 5 This is a navigation test result diagram of the state value mapping according to an embodiment of the present invention.

[0029] Figure 6 3 is a convergence curve diagram obtained by comparing the method of the embodiment of the present invention with other experiments. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0031] In response to the problem that existing visual navigation methods cannot accurately and effectively provide spatial information to assist navigation, the present invention proposes a method for constructing spatial sequence information using deep learning methods in intelligent navigation scenarios, as well as a method for improving navigation generalization capabilities using graph neural networks and meta-learning update strategies.

[0032] This paper proposes a universal visual perception representation method for navigation environments, as well as a knowledge representation and reasoning model that incorporates a graph attention mechanism. Based on the relationships between multiple targets in the navigation scenario, the prior knowledge of a common common sense knowledge graph is used to infer target locations. This visual perception representation can improve the navigation success rate and overall navigation effectiveness of intelligent agents navigating new scenarios.

[0033] Specifically, the present invention proposes a feature representation of visual observation information of navigation scenes, and a visual transformation network to represent and reason about spatial structure information and sequence information. Based on the multi-target relationship in the navigation scene, the target position is inferred using the prior knowledge of the general common sense knowledge graph. This navigation framework can improve the navigation success rate of intelligent agents in new scenes and enhance the overall navigation effect. In order to solve these problems, many works have introduced multimodal information to enhance the intelligent agent's understanding of the environment, and at the same time designed more efficient neural networks to enable the intelligent agent to focus on information related to the navigation target area during the navigation process, thereby improving data efficiency.

[0034] With the development of the Transformer, it has been widely and successfully used in natural language processing and computer vision classification and generation tasks. The Visual Transformer was proposed specifically to solve computer vision tasks such as object detection, semantic segmentation, and video understanding. The Visual Transformer is a deep neural network composed of an attention mechanism. It replaces the functions of the CNN and exhibits excellent performance in visual feature extraction and representation. However, research on the Transformer in reinforcement learning is relatively limited, mainly due to the different nature of the problem. For example, the data generated by visual navigation training has high variance, making it difficult for the model to converge. With the advancement of large-scale pre-trained models, large Visual Transformer models can be combined with other complex algorithmic applications. It solves the problem of multimodal fusion visual perception in visual navigation, improves data efficiency, and thus enhances the generalization ability of intelligent navigation.

[0035] Based on this, the present invention mainly studies the application of visual transformation networks in reinforcement learning visual navigation. The task of visual navigation is to guide the agent to find a given target in a 3D environment based on current observations. This task can be described as the ability of an agent to perceive its surrounding environment relying solely on visual sensors, and to navigate the agent to a specified target effectively and safely. This visual perception includes two key technologies: First, the navigation model allows the agent to perceive the high-dimensional spatial distribution of semantic concepts in scale, including the effective use of meta-learning to generalize implicit distribution sequence information in unknown environments. Secondly, the present invention models the environment in the generative navigation trajectory. The modeled joint distribution sequence is the local image of the target, the global image of the environment, the past state of the environment, the previous action of the agent, and the knowledge graph.

[0036] This paper proposes a visual transformation network SSTNet (Spatial Sequential Transformer Network) to extract spatial attention of the observed image and the spatial position of the target, and then models the state, action, knowledge graph local features and global features as informative sequence transformer input. Figure 1As shown, SSTNet encodes local information (local spatial encoding) and decodes global information (global sequence decoding), and fuses environmental features through A3C reinforcement learning to output navigation actions. In particular, the SSTNet of the present invention uses two newly designed spatial perception and sequence perception descriptors as keys and queries in the reinforcement learning state representation, and encodes them to construct an expressive visual representation. Experimental results show that the visual representation learned by the embodiment of the present invention significantly improves the navigation efficiency of the visual navigation system in unknown environments, and is about 15% higher than the path length success weight (SPL), as shown in Figure 2. Figure 5 and 6 shown.

[0037] This paper designs a graphless navigation framework that uses a visual transformer network to encode local and global spatial features. This framework infers the optimal next navigation action based on the local information of the target in the current target view, the global information between the current view and the previous frame, and the knowledge graph. The visual navigation task is to guide an intelligent agent to find a given target through observation. The effective transformation from the agent's observation to the visual representation determines the navigation behavior, thereby promoting more intelligent navigation strategies. In this work, the present invention proposes SSTNet to learn visual representations of information in deep reinforcement learning (DRL). SSTNet consists of SAF and STNet. In reinforcement learning, SAF transforms cross-modal information into visual cues. It encodes the semantic information of the observed objects and the spatial information of their positions, jointly leveraging the inter-image relations. STNet generates (imagines) the next observation and infers the action most relevant to the target. It decodes the relationships within the image. In this way, the agent learns to understand the causal relationship between navigation behavior and the dynamic changes of observations. SSTNet is an autoregressive model based on expected reward, past states, actions, and a knowledge graph. The entire navigation framework considers local and global visual information, as well as temporal order information. Therefore, it allows the agent to efficiently navigate to popular objects. Evaluating the model of the present invention on the AI2THOR framework shows that the method of the present invention has an average success rate of at least 10% higher than most state-of-the-art models.

[0038] See also Figure 2 The embodiment of the present invention provides a method for utilizing visual conversion network-assisted navigation in an intelligent navigation scenario, comprising the following steps:

[0039] Step 1: First, use the pre-trained Faster-RCNN target detector to detect 108 targets in the navigation environment. By random walk, we can obtain the targets that appear in each frame of the scene. This co-occurrence relationship is defined as an edge and stored as a knowledge graph, such as Figure 3The process of establishing a knowledge graph. When a relationship appears more than a preset number of times (for example, 3 times), the relationship is determined to be common sense. Figure 3 As shown, Faster-RCNN encodes the detection target area (size N image dimension d) into d-dimensional features through the feedforward network FFN The detection results converted from the same layer to N×d are obtained, which include the target bounding box, confidence score and category. In order to learn the relationship between all targets, the 2D detection box coordinates (x, y, w, h) and confidence scores of all targets are first located and combined into the detection features of the node. The depth map is then passed through the VGG convolutional neural network to extract the deep features (i.e., z-dimensional features) and the CNN convolutional neural network is used for splicing to obtain the 3D positioning bounding box (x, y, z, w, h) in the scene graph. t-1 And the current target detection result O t Perform pairing (i.e. target matching) to generate the spatial relationship in the scene graph. Get the target O t , the relationship between the targets R t , and the target attribute A t , thus forming the knowledge graph G t .For example, Figure 3 For example, books, refrigerators, etc. constitute the knowledge graph. In the embodiment, in order to obtain the main information in the scene graph, a total of 92 most frequently occurring target types in AI2THOR were counted. The constructed scene graph has 92 nodes, and each node is connected to at least two other nodes, which represents the relationship between visible targets. The relationship labels are set as follows: R = {in, on, over, contain, under, left_of, right_of} respectively represent inside, on top, above, contain, below, left and right. The embodiment uses the GAT graph attention neural network to extract the constructed knowledge graph features, with 3 network layers and a size of 7×7×92.

[0040] Step 2: In the spatial attention module SAF of the embodiment, for local features, the DETR target detector is used to concatenate the local information in the environment, including the Glove word vector (size is 300), the coordinates of the bbox (dimension is 4), and the CS cosine similarity (dimension is 1), to obtain a local feature vector (size is 7×7×512). Then the local feature vector μ i (t), navigation action feature μ of the previous navigation action a (t) (size is 1×6), hidden state feature μ m (t) (size is 7×7×256) and the graph feature μ of the previous step g (t) Perform feature fusion to obtain a spatial encoder of size 7×7×866 as the spatial embedding layer.

[0041] Step 3, in the sequence conversion network STNet of the embodiment, for global features, the global image information extracted by the ResNet residual network (i.e., global features, size of 1×1×2048) is used as a sequence decoder (i.e., sequence conversion layer), and local features are used as part of the multi-head attention to obtain a global feature vector of size 1732, which is then activated using the LeakyReLU activation function to obtain a feature vector of size 1×512. The features obtained by the activation function are used as LSTM inputs, and the navigation round experience is stored with a size of 784×512. The output hidden state size is 1×512 as the representation of the sequence feature descriptor. The sequence feature descriptor is then input as the representation of the reinforcement learning state, and the self-attention mechanism is used to obtain useful information within the current state, respectively by q w 、k w and v w Represents query, key and value, where w is a training parameter. Finally, the action policy π(a|s) and value function are formed.

[0042] Step 4: After building the knowledge graph, in order to optimize the navigation strategy and provide supervision for the navigation action, the embodiment uses Markov logic network (MLN) for action reasoning and optimizes the navigation strategy. In the graph reasoning module GAM (Graph Association Module), the prediction model considers the spatial context and sequence context And decide the next action based on them. It uses bilinear dot product to calculate the probability p of each navigation direction k If two nodes are dependent on each other, embodiments connect them with an edge. In an MLN, all nodes are initialized using transition probabilities to connect to edges. Therefore, the graph reasoning module can view visible targets as the observation matrix in the MLN, and its conditional probabilities with the emission matrix can predict the optimal reasoning path to the target. For example, when the mouse appears in the current viewpoint, the agent is more likely to find the target object, the computer, by turning right. According to the transition matrix, the edge relationship between the mouse and the computer is left_of.

[0043] The embodiment uses a visual transformation network to encode spatial information of an indoor navigation environment and sequence information of reinforcement learning navigation to improve navigation performance.

[0044] The specific method of visual transformation network SSTNet encoding navigation environment information is:

[0045] Reference Figure 1 : The motivation of the embodiment is to fuse and map the multimodal information of the environment into navigation actions.

[0046] Reference Figure 2 :The key components of the visual navigation framework SSTNet proposed in the embodiment can be divided into three parts: the spatial attention module SAF, the sequence transformation network STNet (Sequential Transformer Network) and the navigation strategy graph association model GAM (Graph Association Module). STNet serves as the decoder of the entire framework. It extracts the semantic information of a given image in the scene and effectively encodes the intermediate and high-level semantic information in the image. Then, the spatial position of the top convolutional layer is used to encode the local detection target area and describe it with a vector. The embodiment also uses a spatial representation with DETR target detector features, which encodes the features of the Glove word vector and fuses the bounding box and confidence score. The embodiment uses the target direction feature to infer the relative position of the target. In order to make full use of the current observation information. The conventional state representation in reinforcement learning is to connect visual and spatial representation features. The navigation policy generates an action for the agent at each step, which will be learned based on the state representation. However, how to optimize the navigation policy is a long-standing problem. The embodiment adopts the standard A3C architecture as the navigation engine. The navigation policy takes visual embedding and spatial embedding as input and outputs action distribution and action value.

[0047] To optimize the strategy, the embodiment designs a navigation strategy graph association model (GAM) as a graph association module to help determine the best next action, which is navigated according to the A3C action distribution. The specific steps of the graph association module are as follows:

[0048] 1) Spatial perception descriptor uses DETR to convert d-dimensional features of N-type target encoding from N-type detection results Includes Glove word embeddings, bounding boxes, classes, and confidence scores for the feedforward network.

[0049] The SSTNet spatial attention fusion module of the embodiment locates each convolutional neuron, using i, j = 1, ..., n w and (i,j) th Indicates the i-th row, j-th column position, n w Represents n windows w, corresponding to the detection area in the observation image, and the window at time t is represented by the vector Then, the embodiment uses the window size of the input image as n w ×n w The attention probability distribution represented by is used to emphasize the spatial information of the object. The attention probability distribution consists of four attention units:

[0050] (1) Local image unit, combining target Glove word embedding, bounding box and observation confidence;

[0051] (2) Action attention unit, which considers the last action of the agent;

[0052] (3) memory attention unit, which stores the images seen in the scene;

[0053] (4) Graph attention unit, used to update scene layout and memorize scene priors.

[0054] These four distributions are then fused into a single attention probability distribution and embedded into the spatial embedding layer. t (i,j) represents time t at n w ×n w Fusion probability on sub-windows.

[0055]

[0056] Among them, attention() represents the attention operation, ⊙ represents the dot product operation,

[0057] 2) The sequence-aware descriptor of the embodiment encodes temporal information and SAF for global object localization. STNet also models the sequential data of current and previous information. It consists of stacked self-attention layers with residual connections.

[0058]

[0059] This allows the layer to assign "weights" (maximizing dot products) by implicitly forming state-value associations based on the similarity between the query and key vectors, where softmax represents normalization. is the accumulation operation, Z i represents the output of the self-attention layer, q i is the input of the i-th encoder, k j′＝1 The input of the previous encoder is j ′ And initialize it to the key value vector of 1, For the time t, the current input j-th encoder output sequence features, this formula represents the output of the sequence conversion layer at this moment. In this work, the embodiment designed STNet, in which the self-attention mask is used to extract global features using the visual conversion network to generate a regression sequence model, represented by the sequence (j∈[1,i]). For example, once the target appears on the left side of the view, it should prefer "RotateRight" instead of "RotateLeft". Since each spatially perceived descriptor corresponds to a certain area of observation, it is regarded as and encoded sequence-aware descriptors The attention function of the SSTNet of the embodiment can be expressed as:

[0060]

[0061] The superscript T represents the rotation operation, and d represents the vector dimension.

[0062] The embodiment can use the visual temporal transformation network to model SAF. It normalizes the sequence of local image features μ i (t), current navigation action feature μ a (t), navigation memory feature μ m (t), graph feature μ g (t) is fused to represent the local image features, navigation history action features, memory features, and graph features at time t. This allows the encoded spatial perception descriptor to better represent the environmental perception features while calculating each probability in an asymmetric manner. However, this does not explain the importance of learning. Instead, the embodiment learns the importance of each probability in the LSTM hidden layer at time t, and the embodiment defines

[0063]

[0064] Sequence information can combine all useful spatial information probability distributions into a related distribution, ∝ means equivalent to.

[0065] 3) The embodiment utilizes Markov logic network (MLN) to improve the action reasoning of the optimal navigation strategy.

[0066] In the graph association module, the MLN predictor considers the space and sequence Information, and based on this, decide the next direction. It calculates the probability p of each navigation direction k . Using the bilinear dot product as follows:

[0067]

[0068] Among them, φ k is the action embedding layer, representing the k-th navigation direction, softmax is the normalization operation, h t is the hidden layer vector, W s is the spatial encoder weight, W φ is the sequence decoder weight.

[0069] Reference Figure 5 , the embodiment preferably provides a codec structure with STNet as a decoder and SAF as an encoder as follows:

[0070] Encoder: The input is encoded as local information (including Glove word vectors, bbox coordinates and CS cosine similarity), and the position features of DETR are added for positioning. The position encoding features are then decomposed into q, k, v vectors, and environmental features are extracted through multi-head self-attention. Then, the previous input encoding is used to enhance and restore the features through addition and normalization operations. The decoder features are output through forward propagation and addition & normalization operations; where q, k, v represent the query vector, key value vector and value vector respectively, N = 6 represents the number of layers of the encoder and decoder, and multi-head self-attention represents a process in which the q, k, v vectors of multiple heads are forward propagated separately, and finally the vector matrix is spliced for forward propagation. The number of multiple heads is preferably defined as 8.

[0071] Decoder: The input is the global environment information extracted using ResNet, which is used as the output code. The code features are then decomposed into q, k, and v vectors. Masked self-attention is used to map the local features to the global map. Next, addition and normalization operations are used to restore the information. The encoder features are then concatenated with the features obtained in the previous step. The decoder features are then output through a linear layer and normalization.

[0072] Experimental setup:

[0073] The example uses the AI2THOR simulator as the experimental framework. It is divided into four different room types: bathroom, living room, bedroom, and kitchen. The experimental setting is SAVN. Each room uses 20 scenes for training.

[0074] 5 validation scenes and 5 test scenes.Example All methods are trained until maximum convergence of 200 million frames.

[0075] Kitchen: microwave, toaster, refrigerator, coffee maker, trash can, bowls and boxes;

[0076] Living room: TV, pillows, trash can, laptop, bowls and boxes;

[0077] Bedroom: Indoor plants, lamps, books and alarm clocks;

[0078] Bathroom: Toilet paper, soap bottle, light switch, sink, 21 in total.

[0079] The total number of available objects is 92. To train the model of the embodiment, the embodiment uses the Pytorch framework, uses RMSprop for adaptation optimization, and uses SharedRMSprop.

[0080] Experimental parameter settings:

[0081] In order to process visual perception images, the embodiment uses a pre-trained ResNet18 to extract observation features at each time step. The algorithm uses Glove to generate 300-dimensional semantic embeddings of targets and graph objects, a total of 92 objects. The input of the reinforcement learning actor-critic network of the embodiment is connected to the target object as a 300-dimensional vector, the observation features are used as a 1024-dimensional feature vector, and the knowledge graph of 92 nodes is input to GAT to produce a 92-dimensional vector. At the same time, GAT is also used to perform knowledge reasoning to produce a single value. DETR is used for target detection to obtain local features of the target, with a feature vector size of 7*7*512 dimensions, the spatial embedding layer size is 7*7*866, the sequence transformation layer size is 1732, and finally the MLP-LSTM layer outputs a feature vector of size 512.

[0082] The actor-critic network of this embodiment consists of an LSTM network with 512 hidden states and two FC layers representing the actor-critic. The actor outputs a 6-dimensional action distribution. The critic estimates a value using softmax.

[0083] The value after GAT decomposition is input into the criticism embedding to form a value estimate. In particular, in terms of dynamically updating the knowledge graph and correcting the error prior of the policy network in unknown scenarios, the embodiment adopts a new meta-learning MAML agent. The input of the graph is a 1024-dimensional vector as the node feature. The algorithm uses the glove embedding method to map 512 observation features to 300 to 512 targets through a linear layer. Each layer contains 92 adjacency matrix nodes, for a total of 5 layers, of which 4 layers are edges between objects in the four scenarios, and the other layer is regularized using a self-connection layer.

[0084] In specific implementation, the method proposed in the technical solution of the present invention can be automatically run by those skilled in the art using computer software technology. System devices that implement the method, such as computer-readable storage media that store the corresponding computer program of the technical solution of the present invention and computer equipment that runs the corresponding computer program, should also be within the scope of protection of the present invention.

[0085] In some possible embodiments, a system for network-assisted navigation using visual conversion in an intelligent navigation scenario is provided, comprising a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a method for network-assisted navigation using visual conversion in an intelligent navigation scenario as described above.

[0086] In some possible embodiments, a system for using visual conversion network to assist navigation in an intelligent navigation scenario is provided, including a readable storage medium on which a computer program is stored. When the computer program is executed, a method for using visual conversion network to assist navigation in an intelligent navigation scenario as described above is implemented.

[0087] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Persons skilled in the art may make various modifications, additions, or substitutions to the described specific embodiments without departing from the spirit of the present invention or exceeding the scope of the appended claims.

Claims

1. A method for assisting navigation using a visual conversion network in an intelligent navigation scenario, characterized by: A graphless navigation framework that uses a visual transformer network to encode local and global spatial features. It then uses the local information of the target in the current target view, the global information between the current view and the previous frame, and the knowledge graph to reason and navigate the next optimal action, completing the visual navigation task. Implementation process The following steps are included: Step 1: First, use the pre-trained Faster-RCNN object detector to obtain the objects that appear in each frame of the scene through random walks. This co-occurrence relationship is defined as an edge, and a knowledge graph is generated and stored. When a relationship appears more than a preset number of times, it is determined to be common sense. Step 2: In the spatial attention module (SAF) of the visual transformer network, the DETR object detector is used to obtain a local feature vector by splicing the local information in the environment. This local feature vector is then fused with the navigation action features of the previous navigation action, the hidden state features, and the graph features to obtain a spatial encoder. Step 3: In the sequence conversion network STNet of the visual converter network, for global features, the global image information extracted by the ResNet residual network is used as a sequence decoder, and local features are used as part of the multi-head attention to obtain a global feature vector. Then, the Leaky ReLU activation function is used. The features obtained by the activation function are used as LSTM inputs to store the navigation round experience. The obtained sequence feature descriptor is input as the representation of the reinforcement learning state, and the self-attention mechanism is used to obtain useful information within the current state. Finally, the action strategy and value function are formed. Step 4: Based on the knowledge graph, a Markov logic network is used to perform action reasoning and optimize the navigation strategy. In the graph reasoning module GAM, a prediction model is used to consider the spatial context and sequence context and decide the next action.

2. The method for utilizing visual conversion network-assisted navigation in an intelligent navigation scenario according to claim 1, characterized in that: When generating a knowledge graph, in order to learn the relationship between all objects, the 2D detection box coordinates and confidence scores of all objects are first located and combined into the detection features of the nodes; then the depth map is used to extract deep features through the VGG convolutional neural network, and then spliced through the CNN convolutional neural network to obtain the 3D positioning bounding box in the scene graph.

3. The method for utilizing visual conversion network-assisted navigation in an intelligent navigation scenario according to claim 1, characterized in that: In the spatial attention module (SAF), the input is encoded as local information, and the position features of DETR are added for positioning. After feature decomposition, the environmental features are extracted through multi-head self-attention. Then, the previous input encoding is used to enhance and restore the features through addition and normalization operations. The decoder features are output through forward propagation and addition & normalization operations, and finally a vector matrix is formed for forward propagation.

4. The method for utilizing visual conversion network-assisted navigation in an intelligent navigation scenario according to claim 1, characterized in that: In the sequence transformation network STNet, the input is the global environment information extracted by ResNet. After feature decomposition, the local features are mapped to the global graph through masked self-attention. The information is restored using addition and normalization operations. The encoder features are spliced with the features obtained in the previous step, and the decoder features are output through linear layers and normalization.

5. The method for utilizing visual conversion network-assisted navigation in an intelligent navigation scenario according to claim 1, characterized in that: The A3C architecture is used as the navigation engine. The navigation strategy takes visual embedding and spatial embedding as input and outputs action distribution and action value. To optimize the strategy, GAM is used to help determine the best next action, which is navigated according to the A3C action distribution.

6. A system for assisting navigation using a visual conversion network in an intelligent navigation scenario, characterized by: A method for implementing a visual conversion network-assisted navigation in an intelligent navigation scenario as described in any one of claims 1-5.

7. The system for assisted navigation using visual conversion network in an intelligent navigation scenario according to claim 6, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the method for assisting navigation by using visual conversion network in an intelligent navigation scenario as described in any one of claims 1 to 5.

8. The system for assisted navigation using visual conversion network in an intelligent navigation scenario according to claim 6, characterized in that: It includes a readable storage medium, on which a computer program is stored. When the computer program is executed, it implements the method of using visual conversion network to assist navigation in an intelligent navigation scenario as described in any one of claims 1 to 5.

Images