Image target driven navigation method, apparatus, device, and storage medium

Abstract

Embodiments of the present application provide an image target driven navigation method, device and equipment and a storage medium, the method comprising: obtaining a target image of a navigation target; generating a map node in a topological map and a global node according to the target image and an observation image at a current time; storing short-term memory data in the map node; storing long-term memory data in the global node; obtaining a navigation action output by a navigation model according to the short-term memory data in the map node, the long-term memory data in the global node, the observation image at the current time and the trained navigation model; and the navigation model is used for image target driven navigation. The method of the embodiments of the present application introduces a human brain working memory mechanism, provides more abundant, comprehensive and accurate decision information for image target driven navigation based on long-term memory data, and thus can accurately realize image target driven navigation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual navigation, and in particular to an image target driven navigation method, device, equipment and storage medium. BACKGROUND

[0002] Image target driven navigation is a common task for an agent, which requires the agent to navigate in a three-dimensional scene according to a specified image, autonomously move to a target position without colliding with any object, and has wide application in indoor services of the agent.

[0003] In related technologies, image target driven navigation is still in its infancy, and usually stores and utilizes navigation history information for navigation when navigating. However, only partial and limited information can be obtained for navigation decision-making in the process of navigation based on the navigation history information, and redundant history information and noise information cannot be deleted, resulting in that the success rate of image target driven navigation is still very low. SUMMARY

[0004] In view of the problems in the prior art, the embodiments of the present application provide an image target driven navigation method, device, equipment and storage medium.

[0005] Specifically, the embodiments of the present application provide the following technical solutions:

[0006] In a first aspect, the embodiments of the present application provide an image target driven navigation method, comprising:

[0007] obtaining a target image of a navigation target;

[0008] generating a map node in a topological map and a global node according to the target image and an observation image at a current time; the map node stores short-term memory data; the global node stores long-term memory data; the short-term memory data represents a feature vector of the target image and a feature vector of the observation image; the long-term memory data represents a weighted summation result of the short-term memory data;

[0009] obtaining a navigation action output by a navigation model according to the short-term memory data in the map node, the long-term memory data in the global node, the observation image at the current time and the trained navigation model; the navigation model is used for image target driven navigation.

[0010] Further, the generating a map node in a topological map and a global node according to the target image and an observation image at a current time; the map node stores short-term memory data; the global node stores long-term memory data, comprises:

[0011] According to the first feature vector corresponding to the target image and the second feature vector corresponding to the observation image at the current moment, a map node in a topological map and short-term memory data in the map node are generated;

[0012] The short-term memory data in the plurality of map nodes is weighted and summed to generate a global node in the topological map and long-term memory data in the global node.

[0013] Further, the input of the short-term memory data in the map node, the long-term memory data in the global node, and the observation image at the current moment into the trained navigation model to obtain the navigation action output by the navigation model comprises:

[0014] According to the short-term memory data in the map node and the long-term memory data in the global node, working memory data is generated;

[0015] The working memory data is decoded to obtain a third feature vector related to the target image and a fourth feature vector related to the observation image at the current moment;

[0016] The third feature vector, the fourth feature vector, and the observation image at the current moment are input into the navigation model to obtain the navigation action output by the navigation model.

[0017] Further, the attention scores corresponding to each map node in the topological map are obtained;

[0018] According to the attention scores corresponding to each map node, redundant map nodes in the topological map are determined, and the redundant map nodes are deleted from the topological map.

[0019] Further, the navigation model is trained based on the navigation action and a preset target action; and a loss function of the navigation model is determined according to a log-likelihood of the target action, whether a current position has ever been explored, and a prediction result of a distance between the current position and a navigation target.

[0020] Further, the navigation model is trained based on the navigation action and a reward value of the navigation action; and a loss function of the navigation model is determined according to the reward value of the navigation action, whether a current position has ever been explored, and a prediction result of a distance between the current position and a navigation target.

[0021] In a second aspect, an embodiment of the present application also provides an image target driven navigation device, comprising:

[0022] An acquisition module is configured to acquire a target image of a navigation target;

[0023] The generating module is configured to generate a map node and a global node in a topological map according to the target image and an observation image at a current time; the map node stores short-term memory data; the global node stores long-term memory data; the short-term memory data represents a feature vector of the target image and a feature vector of the observation image; and the long-term memory data represents a weighted summation result of the short-term memory data.

[0024] The navigation module is configured to obtain a navigation action output by a navigation model according to the short-term memory data in the map node, the long-term memory data in the global node, the observation image at the current time, and the trained navigation model; and the navigation model is used for image target driven navigation.

[0025] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the image target driven navigation method according to the first aspect when executing the program.

[0026] In a fourth aspect, a non-transitory computer readable storage medium is provided, which stores a computer program, and the computer program is executable on a processor to implement the image target driven navigation method according to the first aspect.

[0027] In a fifth aspect, a computer program product is provided, which includes a computer program, and the computer program is executable on a processor to implement the image target driven navigation method according to the first aspect.

[0028] The image target driven navigation method, device, electronic device, and storage medium provided by the embodiments of the present application can generate a map node and a global node in a topological map according to a target image and an observation image at a current time; the map node stores short-term memory data; the global node stores long-term memory data; the long-term memory data is obtained by performing weighted summation processing on the short-term memory data generated by the feature vectors of the target image and the observation image at the current time; based on the long-term memory data, more levels and more comprehensive visual features in a navigation scene can be obtained, and information can be transmitted between map nodes corresponding to the visual features at a long distance; the image target driven navigation is provided with more rich, comprehensive, and accurate decision information; and based on the more rich, comprehensive, and accurate decision information, the image target driven navigation can be accurately implemented, and the success rate of the image target driven navigation is improved. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0030] Figure 1 is one of the flowcharts of the image target driven navigation method provided by the embodiments of the present application;

[0031] Figure 2 is another flowchart of the image target driven navigation method provided by the embodiments of the present application;

[0032] Figure 3 is a schematic diagram of the image target driven navigation model training method provided by the embodiments of the present application;

[0033] Figure 4 is a third flowchart of the image target driven navigation method provided by the embodiments of the present application;

[0034] Figure 5 is a structural schematic diagram of the image target driven navigation device provided by the embodiments of the present application;

[0035] Figure 6 is a structural schematic diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0036] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort belong to the protection scope of the present application.

[0037] The method of the embodiments of the present application can be applied in a visual navigation scene to realize image target driven navigation.

[0038] In the related art, image target driven navigation is still in its infancy. Generally, navigation history information is stored and utilized for navigation. However, only partial and limited information can be obtained for navigation decision making in the process of navigation based on the navigation history information, and redundant history information and noise information cannot be deleted, resulting in that the success rate of image target driven navigation is still very low.

[0039] The image target driving navigation method provided in the embodiment of the present application generates a map node and a global node in a topological map according to a target image and an observation image at a current moment; the map node stores short-term memory data; the global node stores long-term memory data; after the short-term memory data generated by the target image and the observation image at the current moment are processed by weighted summation, the long-term memory data is obtained, more levels and more comprehensive visual features in a navigation scene can be obtained based on the long-term memory data, information can be transmitted between the map nodes corresponding to the long-distance visual features, more abundant, comprehensive and accurate decision information is provided for the image target driving navigation, and then the image target driving navigation can be accurately realized based on the more abundant, comprehensive and accurate decision information, and the success rate of the image target driving navigation is improved.

[0040] The technical solutions of the present application will be described in detail below with reference to specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described in some embodiments. Figures 1-6 The technical solutions of the present application will be described in detail below with reference to specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described in some embodiments.

[0041] Figure 1 is a flowchart of an embodiment of the image target driving navigation method provided in the embodiment of the present application. As shown in Figure 1 The method provided in the embodiment includes the following steps.

[0042] Step 101: obtaining a target image of a navigation target;

[0043] Specifically, the image target driving navigation requires an intelligent agent to navigate in a three-dimensional scene according to a specified image, so in order to realize the image target driving navigation, the target image of the navigation target needs to be obtained, and then the navigation is performed according to the obtained target image.

[0044] Step 102: generating a map node and a global node in a topological map according to a target image and an observation image at a current moment; the map node stores short-term memory data; the global node stores long-term memory data; the short-term memory data represents a feature vector of the target image and a feature vector of the observation image; the long-term memory data represents a weighted summation result of the short-term memory data;

[0045] Specifically, the navigation history information is usually stored and utilized for navigation during navigation, but only partial and limited information can be obtained for navigation decision in the process of navigation based on the navigation history information, and redundant history information and noise information cannot be deleted, resulting in that the success rate of image target driven navigation is still low. In order to improve the success rate of image target driven navigation, the working memory mechanism of human brain is introduced in the navigation process in the embodiment of the present application, that is, the panoramic image observed by the agent is encoded into a feature vector (such as 512 dimensions), and is spliced with the feature vector of the target image to generate short-term memory data, which is stored in the map nodes of the topological map, and edges are established between the adjacent map nodes. The short-term memory data is further processed by weighted summation to obtain long-term memory data, which is stored in the global nodes of the topological map, and the global nodes are connected with all the map nodes. The embodiment of the present application simulates the working memory mechanism of human brain for image target driven navigation, and introduces long-term memory data for aggregating the navigation history information stored in the short-term memory data of the agent, that is, by introducing the global nodes connected with all the map nodes on the topological map, the long-term memory data of the agent is stored, and the advantages are that the long-term memory data is obtained by weighted summation processing of the short-term memory data generated by the feature vectors of the target image and the observation image at the current time, so that the long-term memory data can more fully fuse the features of the map nodes of the topological map, extract more levels and more comprehensive visual features in the navigation scene, realize the mutual transmission of information between the map nodes corresponding to the long-distance visual features, expand the receptive field of the graph attention mechanism visual feature, provide more rich, comprehensive and accurate decision information for image target driven navigation, and thus the image target driven navigation can be accurately realized, and the success rate of image target driven navigation is improved. It should be noted that in the embodiment of the present application, the long-term memory data is introduced for aggregating the navigation history information stored in the short-term memory data of the agent by simulating the working memory mechanism of human brain for image target driven navigation, and the long-term memory data can effectively assist navigation decision, that is, the long-term memory data is obtained by weighted summation processing of the short-term memory data generated by the feature vectors of the target image and the observation image at the current time, which can on the one hand fuse the local information (short-term memory data) stored in the map nodes to generate global information about the navigation visual scene, provide a rich source of decision information for image target driven navigation, and on the other hand, can accelerate the information transmission between the map nodes, expand the receptive field of the graph convolution, reduce the number of graph convolution layers, thereby reducing the redundant parameter amount, improving the decision efficiency and navigation success rate of image target driven navigation.

[0046] In step 103, a navigation action output by a navigation model is obtained according to the short-term memory data in the map nodes, the long-term memory data in the global nodes, the observation image at the current time and the trained navigation model; the navigation model is used for image target driven navigation.

[0047] Specifically, after the long-term memory data is obtained by performing the weighted sum processing on the short-term memory data generated by the feature vectors of the target image and the observation image at the current moment, more levels and more comprehensive visual features in the navigation scene can be obtained based on the long-term memory data, and information can be transmitted between the map nodes corresponding to the long-distance visual features, which provides more rich, comprehensive and accurate decision information for the image target driven navigation, so that the image target driven navigation can be accurately realized, and the success rate of the image target driven navigation is improved. That is, based on the short-term memory data in the map node, the long-term memory data in the global node, and the observation image at the current moment, that is, based on more rich, comprehensive and accurate global information and local information, the navigation decision can be effectively assisted, so that the navigation action output by the navigation model can be accurately obtained, and the success rate of the image target driven navigation is improved.

[0048] The method of the above embodiment generates the map node in the topological map and the global node according to the target image and the observation image at the current moment; the short-term memory data is stored in the map node; the long-term memory data is stored in the global node; that is, after the long-term memory data is obtained by performing the weighted sum processing on the short-term memory data generated by the feature vectors of the target image and the observation image at the current moment, more levels and more comprehensive visual features in the navigation scene can be obtained based on the long-term memory data, and information can be transmitted between the map nodes corresponding to the long-distance visual features, which provides more rich, comprehensive and accurate decision information for the image target driven navigation, so that the image target driven navigation can be accurately realized, and the success rate of the image target driven navigation is improved.

[0049] In an embodiment, the map node in the topological map and the global node are generated according to the target image and the observation image at the current moment; the short-term memory data is stored in the map node; the long-term memory data is stored in the global node, including:

[0050] The map node in the topological map and the short-term memory data in the map node are generated according to the first feature vector corresponding to the target image and the second feature vector corresponding to the observation image at the current moment;

[0051] The short-term memory data in the plurality of map nodes is processed by weighted sum, to generate the global node in the topological map and the long-term memory data in the global node.

[0052] Specifically, in order to improve the success rate of image target driven navigation, the working memory mechanism of the human brain is introduced. Optionally, a pre-trained neural network is used to encode the panoramic image observed by the agent into a feature vector, and the feature vector of the target image is spliced together and stored in the map node of the topological map, and then edges are established between the neighboring map nodes, and the feature vectors of the map nodes in the topological map are used as the short-term memory data of the agent; and the short-term memory data in multiple map nodes is weighted and summed to generate a global node in the topological map and long-term memory data in the global node; optionally, the global node of the topological map and all map nodes in the topological map are connected by edges, and then the entire map is processed by a graph attention mechanism to fuse the short-term memory data into long-term memory data, and the long-term memory data is stored in the global node of the topological map.

[0053] The method of the above embodiment generates a map node in the topological map and short-term memory data in the map node according to the first feature vector corresponding to the target image and the second feature vector corresponding to the observation image at the current time, and then performs weighted summation processing on the short-term memory data in multiple map nodes to generate a global node in the topological map and long-term memory data in the global node, that is, by simulating the working memory mechanism of the human brain, the short-term memory data is stored in the map node of the topological map, and the long-term memory data is stored in the global node of the topological map, and then based on the short-term memory data and the long-term memory data, more rich, comprehensive and accurate decision information is provided for image target driven navigation, so that image target driven navigation can be accurately realized and the success rate of image target driven navigation can be improved.

[0054] In the method of an embodiment, the short-term memory data in the map node, the long-term memory data in the global node, and the observation image at the current time are input into the trained navigation model to obtain the navigation action output by the navigation model, including:

[0055] Generating working memory data according to the short-term memory data in the map node and the long-term memory data in the global node;

[0056] Decoding the working memory data to obtain a third feature vector related to the target image and a fourth feature vector related to the observation image at the current time;

[0057] Inputting the third feature vector, the fourth feature vector, and the observation image at the current time into the navigation model to obtain the navigation action output by the navigation model.

[0058] Specifically, in order to provide more rich, comprehensive and accurate decision information for image target driven navigation based on short-term memory data and long-term memory data, in the embodiment of the present application, working memory data is generated according to the short-term memory data in the map node and the long-term memory data in the global node; optionally, the working memory data is generated by splicing the short-term memory data and the long-term memory data together according to the sequence dimension. Further, by decoding the working memory data, a third feature vector related to the target image and a fourth feature vector related to the observation image at the current time are obtained; optionally, the panoramic image and the target image observed by the agent are respectively encoded into features f cur and f goal Then, two decoders based on the attention mechanism respectively utilize the two features f cur and f goal to decode the working memory data, so as to obtain the third feature vector related to the target image and the fourth feature vector related to the observation image at the current time, and thus more rich, comprehensive and accurate decision information is obtained; finally, the third feature vector related to the target image, the fourth feature vector related to the observation image at the current time and the observation image at the current time are input into the navigation model, and accurate decision can be realized based on the rich decision information, so as to accurately realize image target driven navigation and improve the success rate of image target driven navigation.

[0059] The method of the above embodiment generates working memory data according to the short-term memory data in the map node and the long-term memory data in the global node; further, the working memory data is decoded to obtain a third feature vector related to the target image and a fourth feature vector related to the observation image at the current time, so as to obtain more rich, comprehensive and accurate decision information; thus, accurate decision can be realized based on the rich decision information, so as to accurately realize image target driven navigation and improve the success rate of image target driven navigation.

[0060] In an embodiment, the image target driven navigation method further comprises:

[0061] Obtaining attention scores corresponding to each map node in the topological map;

[0062] According to the attention scores corresponding to each map node, determining redundant map nodes in the topological map, and deleting the redundant map nodes from the topological map.

[0063] Specifically, the mechanism of storing and utilizing the navigation history information when navigating in a large scene occupies too much memory and computing resources, and is prone to a deadlock problem of circling in place, and the map node elimination mechanism can flexibly retain the historical navigation information useful to the current navigation target, reduce memory consumption, and weaken noise interference. Optionally, in the process of decoding the feature vectors of all map nodes, an attention score is generated for the feature vector of each map node, and the nodes whose scores are ranked below a specific percentage threshold are eliminated from the topological map. The eliminated map nodes will not participate in subsequent calculations and navigation decisions required for navigation, and the advantage is that it can exclude noise in the short-term memory data of the agent, and only retain map nodes that are beneficial to the current navigation target, which reduces the memory consumption of the topological map and helps the agent to get rid of the deadlock phenomenon of circling in place caused by repeated action execution, improves the success rate of image target driven navigation, and reduces the time length of reaching the navigation target when image target driven navigation is performed.

[0064] The method of the above embodiment determines redundant map nodes in the topological map according to the attention scores of the respective map nodes, and deletes the redundant map nodes from the topological map, which can effectively exclude noise in the short-term memory data of the agent, retain only map nodes that are beneficial to the current navigation target, reduce the memory consumption of the topological map, solve the deadlock phenomenon of circling in place caused by repeated action execution, improve the success rate of image target driven navigation, and reduce the time length of reaching the navigation target when image target driven navigation is performed.

[0065] In an embodiment, the navigation model is trained based on navigation actions and preset target actions; and a loss function of the navigation model is determined according to a log-likelihood of the target actions, whether the current position has ever been explored, and a prediction result of a distance between the current position and the navigation target.

[0066] Specifically, in the embodiments of the present application, the navigation model is trained based on the navigation actions and the preset target actions. Optionally, in order to improve the ability of the agent to navigate using the observed information, the log likelihood of the navigation action output by the obtained navigation model is maximized, and two auxiliary tasks are added, one is to let the agent predict whether the current position has been explored, and the other is to let the agent estimate the distance between the current position and the navigation target, and then based on the global loss function, the trained navigation model can have better performance, so that the navigation action output by the trained navigation model is more close to the preset target action, and at the same time, the trained navigation model can more accurately determine whether the current position has been explored, so that it can make better navigation decisions based on historical experience information, prevent making repeated execution actions, and make the trained navigation model more accurately predict the distance between the current position and the navigation target, so that it can extract and refer to the historical navigation information of the corresponding distance and area, make more accurate navigation decisions, and improve the navigation efficiency and accuracy. Optionally, the global loss function is as follows:

[0067] L all =L bc +αL aux1 +βL aux2 ;

[0068] Wherein, L all represents the global loss function, L bc represents the loss function of the maximum likelihood of the expert action calculated according to the decision result of the obtained navigation model, L aux1 represents the loss function corresponding to the prediction result of whether the current position has been explored, and L aux2 represents the loss function corresponding to the prediction result of the distance between the current position and the navigation target, wherein α and β are hyperparameters, used for the weight ratio between the two auxiliary tasks, and are usually set to 1.0 and 1.0 respectively.

[0069] The method of the above embodiment trains the navigation model based on the navigation action and the preset target action, maximizes the likelihood of the expert action calculated by the navigation model decision result obtained, and realizes the training of the navigation model by taking the prediction result of whether the current position has been explored and the prediction result of the distance between the current position and the navigation target as the loss function, so that the trained navigation model has better performance, the trained navigation model is closer to the preset target action, and the trained navigation model can more accurately determine whether the current position has been explored, so that better navigation decisions can be made based on historical experience information, repeated execution actions are prevented, and the trained navigation model can more accurately predict the distance between the current position and the navigation target, so that historical navigation information of the corresponding distance and region can be extracted and referred to, more accurate navigation decisions can be made, and navigation efficiency and accuracy are improved.

[0070] In an embodiment, the navigation model is trained based on the navigation action and a reward value of the navigation action; and the loss function of the navigation model is determined according to the reward value of the navigation action, the prediction result of whether the current position has been explored, and the prediction result of the distance between the current position and the navigation target.

[0071] Specifically, in the embodiment of the present application, the navigation model can also be trained by the navigation action and the reward value of the navigation action during the training process based on the navigation action and the preset target action; optionally, in order to improve the ability of the agent to navigate using observed information, the navigation model is trained according to the reward value of the navigation action, that is, after the navigation action output by the navigation model is executed, a reward is obtained, and the reward is set as: the agent obtains a reward of -0.001 every time step, and obtains a reward of +10 when reaching the target; and two auxiliary tasks are added, one is to let the agent predict whether the current position has been explored, and the other is to let the agent estimate the distance between the current position and the navigation target, based on the global loss function, the trained navigation model has better performance, the navigation action output by the trained navigation model can obtain a higher reward, and the navigation action output by the navigation model is more accurate and efficient, the trained navigation model can more accurately determine whether the current position has been explored, and better navigation decisions can be made based on historical experience information to prevent repeated execution actions; and the trained navigation model can more accurately predict the distance between the current position and the navigation target, so that historical navigation information of the corresponding distance and region can be extracted and referred to, more accurate navigation decisions can be made, and navigation efficiency and accuracy are improved. Optionally, the global loss function is as follows:

[0072] L all =L rl +αL aux1+ βL aux2 ;

[0073] wherein, L all represents a global loss function, L rl represents a loss function corresponding to the reward value of the navigation action output by the navigation model, L aux represents a loss function corresponding to the prediction result of whether the current position has been explored, and L aux2 represents a loss function corresponding to the prediction result of the distance between the current position and the navigation target, wherein α and β are hyperparameters for the weight ratio between the two auxiliary tasks, and are usually set to 1.0 and 1.0, respectively.

[0074] The method of the above embodiment trains the navigation model based on the navigation action and the reward value of the navigation action, trains the navigation model through the reward value of the navigation action, and realizes the training of the navigation model through the prediction result of whether the current position has been explored and the prediction result of the distance between the current position and the navigation target as loss functions, so that the trained navigation model has better performance, so that the navigation action output by the trained navigation model can obtain higher rewards, so that the navigation action output by the navigation model is more accurate and efficient; at the same time, the trained navigation model can more accurately determine whether the current position has been explored, so that more optimal navigation decisions can be made based on historical experience information, and repeated execution actions can be prevented; and the trained navigation model can more accurately predict the distance between the current position and the navigation target, so that historical navigation information of the corresponding distance and region can be extracted and referred to, and more accurate navigation decisions can be made to improve navigation efficiency and accuracy.

[0075] Exemplarily, the image target driven navigation method is as shown in Figure 2 , the map nodes in the topological map and the global nodes are generated according to the target image and the observation image (the panoramic image observed by the agent) of each time step, and the update of the topological map is realized; further, the working memory data is generated according to the short-term memory data in the map nodes and the long-term memory data in the global nodes; optionally, the short-term memory data is stored in the map nodes of the topological map and the long-term memory data is stored in the global nodes of the topological map by imitating the working memory mechanism of the human brain, and then based on the short-term memory data and the long-term memory data, more rich, comprehensive and accurate decision information can be provided for the image target driven navigation, and then based on the rich, comprehensive and accurate decision information, the image target driven navigation can be accurately realized, and the success rate of the image target driven navigation can be improved.

[0076] Then according to the attention scores corresponding to each map node in the topological map, redundant map nodes in the topological map are determined, the redundant map nodes are deleted from the topological map, the noise in the short-term memory data of the agent is excluded, only the map nodes beneficial to the current navigation target are retained, the memory consumption of the topological map is reduced, the deadlock phenomenon of turning in place caused by repeated action execution is solved, the success rate of image target driven navigation is improved, and the time length of reaching the navigation target during image target driven navigation is reduced.

[0077] Finally, the working memory data is decoded to obtain a third feature vector related to the target image and a fourth feature vector related to the observation image at the current moment, so that more rich, comprehensive and accurate decision information is obtained, and accurate decision can be made based on the rich decision information, the image target driven navigation is accurately realized, and the success rate of image target driven navigation is improved.

[0078] The map node elimination mechanism can flexibly retain the historical navigation information useful to the current navigation target, reduce memory consumption, and weaken noise interference.

[0079] For example, the image target driven navigation training method is as shown in Figure 3 Optionally, first, the target images of multiple navigation targets (such as 3) are specified in sequence, the position of the agent (the starting position) is initialized, and then the agent is navigated to these targets in sequence, that is, the agent is specified with multiple target images in the training and testing stage, and is navigated to the regions indicated by the target images in sequence, and the training and testing method can verify whether the agent can return to the target located in the passed region according to the navigation history memory, so as to improve the success rate of image target driven navigation through testing and training.

[0080] For example, the image target driven navigation method is as shown in Figure 4As shown, optionally, in a deep neural network framework, first, the panoramic color image observed by the agent at the current time is taken as input, a pre-trained convolutional neural network is used to compress the input into a feature vector, and the feature vector is stored as a map node in the topological map. During navigation, the agent generates multiple such feature vectors and stores them in the map nodes of the topological map, to temporarily store the navigation history observation information. A global node and all map nodes are connected by edges, and the feature information of the map nodes is fused through graph convolution operation. An attention mechanism-based decoder decodes all map node feature vectors using the feature vector of the target image, and generates an attention score for each feature vector. Nodes with scores ranking below a certain percentage threshold are removed from the topological map. Finally, only the feature vectors of the retained map nodes and the global node are used as input to make navigation decisions. The steps include:

[0081] Step S0, initialize the agent, place it in an initial position in the environment, then specify a photo (such as 256x256x3 pixels) at a certain place in the scene as the agent's navigation target, and initialize the agent's topological map as a topological map structure without nodes and edges.

[0082] Step S1, during navigation, encode the panoramic image observed by the agent at each time step into a feature vector using a pre-trained neural network, and concatenate the feature vector with the feature vector of the target image, store it in the map node of the topological map, then establish edges between neighboring map nodes, and the feature vectors of all map nodes are used as the agent's short-term memory data M ST (size Lxd).

[0083] Step S2, a global node M LT (size 1xd) stores long-term memory data, and establishes edges with newly added topological map nodes, then the entire map is processed by a graph attention mechanism to fuse the feature vectors of the short-term memory data into long-term memory data. The processed short-term memory data and long-term memory data are M ST and M LT , respectively, which are concatenated according to the sequence dimension to generate working memory M' (size (1+L)xd).

[0084] Step S3, use a convolutional neural network to encode the panoramic image observed by the agent at the current time and the target image into features f cur and f goal (size d), respectively, then two attention mechanism-based decoders Dec cur and Dec goal decode the working memory M' using the two features, respectively, to obtain the decoded features: e cur = Deccur (f cur ,M'),e goal =Dec goal (f goal ,M').

[0085] Step S4, a rejection mechanism based on decoder Dec goal Calculate an attention score for each map node, and remove map nodes that rank below a set percentage threshold from the topology map. Unless the agent returns to the area represented by the removed map node or completes the current navigation objective, the removed map node will not participate in the calculations required for subsequent navigation.

[0086] Step S5, decode feature e cur e goal and f cur These actions are combined and used as input to a navigation module, which in turn generates navigation actions (discrete action space: forward, left turn, right turn, and stop).

[0087] Step S6: After the agent completes the navigation action, it receives a reward and stores the current state information in the experience pool for use during reinforcement learning training. The reward is set as follows: the agent receives a negative reward for each time step and a positive reward upon reaching the target; alternatively, the reward is set as follows: the agent receives a reward of -0.001 for each time step and a reward of +10 upon reaching the target.

[0088] Step S7: The agent trains on a single-target navigation dataset. Navigation is successful when it enters the vicinity of the target location. The model training process consists of two stages: the first stage is imitation learning, which maximizes the log-likelihood of the expert action output by the agent's policy module; the second stage is reinforcement learning, which fine-tunes the model using a proximal policy. To improve the agent's ability to navigate using observed information, two auxiliary tasks are added to both stages: one is for the agent to predict whether it has been to the current location before, and the other is for the agent to estimate the distance to the target. The global loss in the first stage is L. all =L bc +αL aux1 +βL aux2 The second stage global loss L all =L rl +αL aux +βL aux2 , where α and β are hyperparameters used to adjust the loss of the policy module and the weight ratio between the two auxiliary tasks, and are usually set to 1.0 and 1.0 respectively.

[0089] Step S8, the intelligent agent collects the trajectories for training online, and calculates the loss function according to the trajectories, and then adopts the back propagation algorithm and the stochastic gradient descent method to reduce the global loss L all To train the model, 10 million trajectories are collected to train the model, and the final navigation model is obtained through multiple iterations of training. all To train the model, 10 million trajectories are collected to train the model, and the final navigation model is obtained through multiple iterations of training.

[0090] Step S9, the test phase, the trained navigation model is used for multi-target continuous navigation. First, a plurality of target images are specified in sequence, the position of the intelligent agent is initialized, and then the intelligent agent is navigated to the targets in sequence. Optionally, first, four random target images are specified in sequence, the position of the intelligent agent is initialized, and then the intelligent agent is navigated to the targets in sequence. The intelligent agent is tested 700 times, the navigation success rate and the path weighted navigation success rate are counted, and the navigation performance evaluation of the intelligent agent is completed.

[0091] The above image target driven navigation method based on the working memory mechanism of the human brain introduces long-term memory data to aggregate the navigation history information stored in the short-term memory data of the intelligent agent, uses the rejection mechanism to retain the information in the short-term memory data that is beneficial to the current target, and uses the multi-target navigation task to test the navigation performance of the intelligent agent.

[0092] The image target driven navigation device provided by the present application is described below, and the image target driven navigation device described below can be correspondingly referred to the image target driven navigation method described above.

[0093] Figure 5 The structure diagram of the image target driven navigation device provided by the present application is shown. The image target driven navigation device provided by the present embodiment comprises:

[0094] The acquisition module 510 is configured to acquire the target image of the navigation target.

[0095] The generating module 520 is configured to generate a map node in the topological map and a global node according to the target image and the observation image at the current moment; the map node stores short-term memory data; the global node stores long-term memory data; the short-term memory data represents a feature vector of the target image and a feature vector of the observation image; and the long-term memory data represents a weighted sum result of the short-term memory data.

[0096] The navigation module 530 is configured to obtain a navigation action output by a navigation model according to the short-term memory data in the map node, the long-term memory data in the global node, the observation image at the current moment, and the trained navigation model; and the navigation model is used for image target driven navigation.

[0097] Optionally, the generating module 520 is specifically configured to generate the map node in the topological map and the short-term memory data in the map node according to a first feature vector corresponding to the target image and a second feature vector corresponding to the observation image at the current moment.

[0098] The short-term memory data in the plurality of map nodes is subjected to weighted sum processing to generate the global node in the topological map and long-term memory data in the global node.

[0099] Optionally, the navigation module 530 is specifically configured to generate working memory data according to the short-term memory data in the map node and the long-term memory data in the global node.

[0100] The working memory data is decoded to obtain a third feature vector related to the target image and a fourth feature vector related to the observation image at the current moment.

[0101] The third feature vector, the fourth feature vector, and the observation image at the current moment are input into the navigation model to obtain a navigation action output by the navigation model.

[0102] Optionally, the navigation module 530 is further configured to obtain an attention score corresponding to each map node in the topological map.

[0103] According to the attention score corresponding to each map node, a redundant map node in the topological map is determined, and the redundant map node is deleted from the topological map.

[0104] Optionally, the navigation model is trained based on the navigation action and a preset target action; and a loss function of the navigation model is determined according to a log-likelihood of the target action, a prediction result of whether a current position has been explored, and a prediction result of a distance between the current position and a navigation target.

[0105] Optionally, the navigation model is trained based on the navigation action and a reward value of the navigation action; and a loss function of the navigation model is determined according to the reward value of the navigation action, a prediction result of whether the current position has been explored, and a prediction result of a distance between the current position and the navigation target.

[0106] The device of the embodiment of the application is used to execute the method in any of the foregoing method embodiments, and has similar implementation principles and technical effects, which will not be described herein again.

[0107] Figure 6 An example of a schematic diagram of a physical structure of an electronic device is shown, which can include a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 complete mutual communication through the communications bus 640. The processor 610 can invoke a logical instruction in the memory 630 to execute an image target driving navigation method, which includes: acquiring a target image of a navigation target; generating a map node in a topological map and a global node according to the target image and an observation image at a current time; storing short-term memory data in the map node; storing long-term memory data in the global node; the short-term memory data representing a feature vector of the target image and a feature vector of the observation image; the long-term memory data representing a weighted summation result of the short-term memory data; obtaining a navigation action output by a navigation model according to the short-term memory data in the map node, the long-term memory data in the global node, the observation image at the current time, and the trained navigation model; and the navigation model being used for image target driving navigation.

[0108] In addition, the logical instruction in the memory 630 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of the present application essentially or partly, or parts of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0109] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer-readable storage medium, and the computer program comprises program instructions, when the program instructions are executed by a computer, the computer can execute the image target driven navigation method provided by the above method, and the method comprises: obtaining a target image of a navigation target; generating a map node in a topological map and a global node according to the target image and an observation image at a current time; storing short-term memory data in the map node; storing long-term memory data in the global node; the short-term memory data represents a feature vector of the target image and a feature vector of the observation image; the long-term memory data represents a weighted summation result of the short-term memory data; obtaining a navigation action output by a navigation model according to the short-term memory data in the map node, the long-term memory data in the global node, the observation image at the current time and the trained navigation model; and the navigation model is used for image target driven navigation.

[0110] In another aspect, the present application also provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the image target driven navigation method provided by the above method, and the method comprises: obtaining a target image of a navigation target; generating a map node in a topological map and a global node according to the target image and an observation image at a current time; storing short-term memory data in the map node; storing long-term memory data in the global node; the short-term memory data represents a feature vector of the target image and a feature vector of the observation image; the long-term memory data represents a weighted summation result of the short-term memory data; obtaining a navigation action output by a navigation model according to the short-term memory data in the map node, the long-term memory data in the global node, the observation image at the current time and the trained navigation model; and the navigation model is used for image target driven navigation.

[0111] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement without creative labor.

[0112] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0113] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image target-driven navigation method, characterized in that, include: Obtain the target image of the navigation target; Based on the target image and the observed image at the current moment, map nodes and global nodes in the topology map are generated; the map nodes store short-term memory data. The global node stores long-term memory data; the short-term memory data represents the feature vectors of the target image and the observed image; the long-term memory data represents the weighted sum of the short-term memory data. Based on the short-term memory data in the map nodes, the long-term memory data in the global nodes, the observed image at the current moment, and the trained navigation model, the navigation action output by the navigation model is obtained; the navigation model is used for image target-driven navigation.

2. The image target-driven navigation method according to claim 1, characterized in that, The process involves generating map nodes and global nodes in the topology map based on the target image and the observed image at the current moment. The map nodes store short-term memory data; The global node stores long-term memory data, including: Based on the first feature vector corresponding to the target image and the second feature vector corresponding to the observed image at the current moment, map nodes in the topological map and short-term memory data in the map nodes are generated. The short-term memory data from multiple map nodes are weighted and summed to generate global nodes and long-term memory data in the global nodes of the topology map.

3. The image target-driven navigation method according to claim 2, characterized in that, The step of obtaining the navigation action output by the navigation model based on the short-term memory data in the map nodes, the long-term memory data in the global nodes, the observed image at the current moment, and the trained navigation model includes: Working memory data is generated based on the short-term memory data in the map nodes and the long-term memory data in the global nodes; The working memory data is decoded to obtain a third feature vector related to the target image and a fourth feature vector related to the observed image at the current moment; The third feature vector, the fourth feature vector, and the current observation image are input into the navigation model to obtain the navigation action output by the navigation model.

4. The image target-driven navigation method according to claim 3, characterized in that, Also includes: Obtain the attention score for each map node in the topology map; Based on the attention scores corresponding to each map node, redundant map nodes in the topology map are identified and deleted from the topology map.

5. The image target-driven navigation method according to any one of claims 1-4, characterized in that, The navigation model is trained based on the navigation action and the preset target action; the loss function of the navigation model is determined based on the log-likelihood of the target action, whether the current position has been explored before, and the predicted distance between the current position and the navigation target.

6. The image target-driven navigation method according to claim 5, characterized in that, Also includes: The navigation model is trained based on the navigation action and the reward value of the navigation action; The loss function of the navigation model is determined based on the reward value of the navigation action, whether the current location has been explored before, and the predicted distance between the current location and the navigation target.

7. An image target-driven navigation device, characterized in that, include: The acquisition module is used to acquire the target image of the navigation target; The generation module is used to generate map nodes and global nodes in the topological map based on the target image and the observed image at the current time; the map nodes store short-term memory data; the global nodes store long-term memory data; the short-term memory data represents the feature vector of the target image and the feature vector of the observed image; the long-term memory data represents the weighted sum of the short-term memory data; The navigation module is used to obtain the navigation action output by the navigation model based on the short-term memory data in the map nodes, the long-term memory data in the global nodes, the observed image at the current moment, and the trained navigation model; the navigation model is used for image target-driven navigation.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the image target-driven navigation method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the image target-driven navigation method as described in any one of claims 1 to 6.

10. A computer program product having executable instructions stored thereon, characterized in that, When executed by the processor, this instruction causes the processor to implement the image target-driven navigation method as described in any one of claims 1 to 6.

Images