Indoor target driving vision navigation method and device, electronic equipment and storage medium

Abstract

This application provides an indoor target-driven visual navigation method, device, electronic device, and storage medium. The method includes: acquiring an observation image and a target word at the current moment; inputting the observation image and target word into a global feature network to determine a global feature vector; inputting the observation image and target word into a local feature network to determine a local feature vector; in the embedding space, processing the global and local feature vectors into a joint vector through a fully connected layer; and outputting the selected action and corresponding reward at the current moment based on an asynchronous dominant actor-critic network. This scheme can improve the generalization ability and navigation capability for unknown scenes and targets.

Description

Technical Field

[0001] This invention belongs to the field of visual navigation technology, and specifically relates to an indoor target-driven visual navigation method, device, electronic device and storage medium. Background Technology

[0002] Traditional navigation methods, i.e., map-based methods, mainly rely on Simultaneous Localization and Mapping (SLAM) and path planning. A prominent problem is that these methods are susceptible to the accumulation of sensor noise, which propagates along the process from mapping and localization to path planning, resulting in large cumulative errors.

[0003] To reduce accumulated errors, an increasing number of researchers are focusing their time and efforts on end-to-end visual navigation based on deep reinforcement learning (DRL), evaluating navigation performance by maximizing the rewards generated as the AI ​​agent learns its policies. Goal-driven visual navigation is a novel approach that guides an agent to a specific target location based on first-person visual information.

[0004] Indoor target-driven visual navigation is a technology where an intelligent agent navigates to a designated target in an indoor environment, using the currently observed image and the target as input. This requires the agent to fully understand its visual input, infer its current position, and rationally plan the optimal path to the target. Currently, indoor target-driven visual navigation technology has two major problems: first, its generalization ability to unknown scenes and targets is weak; second, its navigation performance decreases as the exploration space increases. Summary of the Invention

[0005] The purpose of the embodiments in this specification is to provide an indoor target-driven visual navigation method, device, electronic device, and storage medium.

[0006] To solve the above-mentioned technical problems, the embodiments of this application are implemented in the following ways:

[0007] In a first aspect, this application provides an indoor target-driven visual navigation method, the method comprising:

[0008] Obtain the observed image and target word at the current moment;

[0009] The observed image and target word are input into the global feature network to determine the global feature vector;

[0010] The observed image and target word are input into a local feature network to determine local feature vectors;

[0011] In the embedding space, the global feature vector and the local feature vector are processed into a joint vector through a fully connected layer;

[0012] The joint vector, based on the asynchronous advantage actor-critic network, outputs the action selected at the current moment and the corresponding reward.

[0013] In one embodiment, the observed image and the target word are input into a global feature network to determine a global feature vector, including:

[0014] The observed image is encoded using the spatial location of the top convolutional layer of the first ResNet-18 network to obtain a global embedding vector;

[0015] The target word is embedded using the Word2Vec model to obtain word embedding vectors;

[0016] The global embedding vector and word embedding vector are activated by the softmax function to obtain the spatial attention probability distribution;

[0017] The spatial attention probability distribution and the global embedding vector are multiplied point by point to obtain the global feature vector.

[0018] In one embodiment, the observed image and the target word are input into a local feature network to determine local feature vectors, including:

[0019] The observed image is input into YOLO-v3 for image detection to obtain local embedding variables, the center coordinates of the observed image, and the bounding box size.

[0020] The target word is embedded using the Word2Vec model to obtain word embedding vectors;

[0021] The context feature vector is obtained based on the local embedding variables, the center coordinates of the observed image, the bounding box size, and the word embedding vector;

[0022] Based on the context feature vector, the local feature vector is obtained.

[0023] In one embodiment, a contextual feature vector is obtained based on local embedding variables, the center coordinates of the observed image, the bounding box size, and the word embedding vector, including:

[0024] Based on the observed image and the target word, determine whether the target word is in the observed image;

[0025] Based on local embedding variables and word embedding vectors, determine the cosine similarity between the observed image and the target word;

[0026] Whether the target word is in the observed image, the center coordinates of the observed image, the size of the bounding box, and the cosine similarity constitute the contextual feature vector.

[0027] In one embodiment, a local feature vector is obtained based on the context feature vector, including:

[0028] Flatten the context feature vector and use it as a local feature vector.

[0029] In one embodiment, the method further includes optimization actions, including:

[0030] Acquire the first observation image at the current moment and the second observation image at the previous moment;

[0031] The first feature is obtained by using a second ResNet-18 network to extract features from the first observed image.

[0032] The second observation image is used to extract features using a third ResNet-18 network to obtain the second features;

[0033] The first feature is fed into at least one convolutional layer for processing to obtain the first convolution result;

[0034] The second feature is fed into at least one convolutional layer for processing to obtain the second convolution result;

[0035] The first and second convolution results are sent to the fully connected layer for processing to obtain the action prediction value of the previous time step.

[0036] Get the truth value of the action at the previous moment;

[0037] The objective function is determined based on the predicted action value and the true action value of the previous moment.

[0038] The action that minimizes the objective function is the optimal action.

[0039] In one embodiment, the reward function in the asynchronous dominant actor-critic network is:

[0040] When a special termination signal is issued and the target word becomes visible, the reward is determined to be the preset reward for achieving the target;

[0041] When no special termination signal is issued, the reward is determined to be similarity.

[0042] Otherwise, the reward will be determined as a time penalty.

[0043] Secondly, this application provides an indoor target-driven visual navigation device, the device comprising:

[0044] The acquisition module is used to acquire the observed image and target word at the current moment;

[0045] The first determination module is used to observe the image and target word input into the global feature network to determine the global feature vector;

[0046] The second determination module is used to observe the image and target word input local feature network to determine local feature vectors;

[0047] The processing module is used to process global and local feature vectors into a joint vector through a fully connected layer in the embedding space.

[0048] The output module is used to output the action selected at the current moment and the corresponding reward based on the asynchronous dominant actor-critic network.

[0049] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the indoor target-driven visual navigation method as described in the first aspect.

[0050] Fourthly, this application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the indoor target-driven visual navigation method as described in the first aspect.

[0051] As can be seen from the technical solutions provided in the embodiments of this specification above, the solution fully integrates the spatial and semantic information of the scene through spatial attention probability distribution and context vector, so that the navigation model can be well extended to unknown scenes and objects.

[0052] Furthermore, this application uses action prediction as an auxiliary task to evaluate key differences between sequential states. Through training, it can predict the difference between the current state and the target, and improve the reward function through semantic similarity measurement. These two aspects can solve the sparse reward problem in deep reinforcement learning.

[0053] This application can improve the generalization ability to unknown scenarios and targets, solve the sparse reward problem, accelerate the convergence speed of training, and thus improve navigation performance. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A flowchart illustrating the indoor target-driven visual navigation method provided in this application;

[0056] Figure 2 Another flowchart illustrating the indoor target-driven visual navigation method provided in this application;

[0057] Figure 3 A schematic diagram of the action prediction network provided in this application;

[0058] Figure 4 A schematic diagram of the structure of the indoor target-driven visual navigation device provided in this application;

[0059] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0060] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0061] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0062] Various modifications and variations can be made to the specific embodiments described in this application without departing from the scope or spirit of this application, as will be apparent to those skilled in the art. Other embodiments derived from this application will be obvious to those skilled in the art. This application specification and embodiments are merely exemplary.

[0063] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.

[0064] Unless otherwise specified, "parts" in this application refers to parts by weight.

[0065] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0066] Reference Figure 1This document illustrates a flowchart of an indoor target-driven visual navigation method applicable to embodiments of this application. It is understood that this indoor target-driven visual navigation method can be executed by an intelligent agent. Consider a first-person perspective observation image S = {s0,…,s...} n}, and the target word O = {o1, ..., o} of the target object category. n}, through strategy π θ (·|s0) Learning. Strategy π θ (·|s0) First, in the first round, the agent selects an action from the action set A = {MoveAhead, MoveRight, MoveLeft, RotateRight, RotateLeft, Lookup, LookDown, DONE} for the initial scene s0. In subsequent rounds, the action is iteratively sampled. At time t, the agent observes the scene s0. t Select action a t This continues until the target object (i.e., the target word) is observed, such as when the target object appears in the currently observed image and is within 1 meter of the agent. Each action is defined as follows:

[0067] MoveAhead, MoveRight, MoveLeft: Move forward / right / left respectively;

[0068] RotateRight, RotateLeft: Rotate the agent 45 degrees to the right and left respectively;

[0069] Lookup, LookDown: Tilt the camera up / down by 30 degrees respectively;

[0070] DONE: A special type of termination marker.

[0071] like Figure 1 As shown, the indoor target-driven visual navigation method may include:

[0072] S110. Obtain the observed image and target word at the current moment.

[0073] Specifically, the observed image refers to an image taken from the perspective of the intelligent agent's first-person vision. The target word is the object to be navigated to; that is, the indoor target-driven visual navigation method ends navigation when the target word is observed in the observed image.

[0074] S120. The observed image and target word are input into the global feature network to determine the global feature vector.

[0075] Specifically, the global feature network aims to learn the spatial attention probability distribution on the observed image.

[0076] In one embodiment, the observed image and the target word are input into a global feature network to determine a global feature vector, including:

[0077] The observed image is encoded using the spatial location of the top convolutional layer of the first ResNet-18 network to obtain a global embedding vector;

[0078] The target word is embedded using the Word2Vec model to obtain word embedding vectors;

[0079] The global embedding vector and word embedding vector are activated by the softmax function to obtain the spatial attention probability distribution;

[0080] The spatial attention probability distribution and the global embedding vector are multiplied point by point to obtain the global feature vector.

[0081] Specifically, the first ResNet-18 network was preprocessed by training on ImageNet.

[0082] The dimension of the top convolutional layer of the first ResNet-18 network is n. v ×n v ×d v The spatial location (i,j) of each convolutional neuron in the topmost convolutional layer. th This corresponds to a region in the observed image, where i,j=1,…,n v . (i,j) th The region represented by a neuron at a given location is called (i,j). th The sub-window, with each step's sub-window vector (i.e., the global embedding vector) being v. i t ,j Then, attention probability distribution is used to emphasize the spatial information of objects in the relevant sub-windows.

[0083] The target word is embedded using the Word2Vec model, resulting in a feature vector (i.e., a word embedding vector) of type v. g .

[0084] The feature vector v g and the sub-window vector at each step Embedded into d-dimensional space. Let W V A sub-window vector in d-dimensional space Trainable parameters, W g Embedding vector v for words in the same space g The trainable parameters for each sub-window i,j=1,…,n v The word embedding vector v at step t g with sub-window vector The interaction is achieved by calculating their inner product:

[0085]

[0086] The corresponding spatial attention probability distribution is obtained by using the softmax activation function:

[0087]

[0088] Finally, the global feature vector of the image is obtained through point-by-point multiplication:

[0089]

[0090] Dimensions and They have the same dimensions.

[0091] S130. The observed image and target word are input into the local feature network to determine the local feature vector.

[0092] Specifically, local feature networks aim to learn the context vectors of observed objects in an image.

[0093] In one embodiment, the observed image and the target word are input into a local feature network to determine local feature vectors, including:

[0094] The observed image is input into YOLO-v3 for image detection to obtain local embedding variables, the center coordinates of the observed image, and the bounding box size.

[0095] The target word is embedded using the Word2Vec model to obtain word embedding vectors;

[0096] The context feature vector is obtained based on the local embedding variables, the center coordinates of the observed image, the bounding box size, and the word embedding vector;

[0097] Based on the context feature vector, the local feature vector is obtained.

[0098] In one embodiment, a contextual feature vector is obtained based on local embedding variables, the center coordinates of the observed image, the bounding box size, and the word embedding vector, including:

[0099] Based on the observed image and the target word, determine whether the target word is in the observed image;

[0100] Based on local embedding variables and word embedding vectors, determine the cosine similarity between the observed image and the target word;

[0101] Whether the target word is in the observed image, the center coordinates of the observed image, the size of the bounding box, and the cosine similarity constitute the contextual feature vector.

[0102] In one embodiment, the local feature vector is obtained based on the context feature vector, including:

[0103] Flatten the context feature vector and use it as a local feature vector.

[0104] Specifically, YOLO-v3 is used for local feature representation. First, the currently observed RGB image (i.e., the observed image) is input into YOLO-v3 for image detection. Considering that the target object may exist in the currently observed image, a 5-dimensional context feature vector v is defined. x =[f,x i ,y i ,S b The first element f is a binary number indicating whether the target object can be detected in the current frame of the currently observed image; the element x i ,y i and S b represents the center coordinates and bounding box size of each observed image, both normalized relative to the image size; CS represents the cosine similarity between the current observed image and the target object. The target word obtains its word embedding vector v using pre-trained Word2Vec. g Similarly, word embedding features for the detected object category are obtained, i.e., using the detected target category g and category o. i The word embedding features v are then used to calculate the cosine similarity between the two vectors using the following formula:

[0105]

[0106]

[0107] Finally, the five-dimensional context vector is flattened and used as a local feature vector.

[0108] S140. In the embedding space, the global feature vector and the local feature vector are processed into a joint vector through a fully connected layer.

[0109] S150, the joint vector is based on the asynchronous advantage actor-critic (A3C) network, which outputs the action selected at the current moment and the corresponding reward.

[0110] Specifically, the joint vector is first passed as input to the LSTM (Long Short Term Memory Network), and finally outputs the action selected at the current time (i.e., policy π) and the corresponding reward (i.e. value V) through the A3C network.

[0111] The indoor target-driven visual navigation method provided in the embodiments of this application, such as Figure 2As shown, a global feature network and a local feature network are used. Both networks take the RGB image and the target word at time t as input. The two networks obtain the full feature vector and the local feature vector respectively. The two vectors are processed into a joint vector in the embedding space. The joint vector is passed as input to the LSTM network. Finally, the policy π and the value V are output through the A3C network.

[0112] This application embodiment fully integrates the spatial and semantic information of the scene by using spatial attention probability distribution and context feature vectors, so that the navigation method can be well extended to unknown scenes and objects.

[0113] To obtain the optimal action, this application employs action prediction as an auxiliary task. The action prediction method uses the current observed image x t The observed image x from the previous moment t-1 Given the input, predict the action a at the current last moment. t-1 The algorithm generates a probability distribution and outputs a vector of 8 elements, with each position corresponding to a specific action. The action with the highest probability is 1, and the rest are 0.

[0114] In one embodiment, the indoor target-driven visual navigation method further includes optimizing actions, including:

[0115] Acquire the first observation image at the current moment and the second observation image at the previous moment;

[0116] The first feature is obtained by using a second ResNet-18 network to extract features from the first observed image.

[0117] The second observation image is used to extract features using a third ResNet-18 network to obtain the second features;

[0118] The first feature is fed into at least one convolutional layer for processing to obtain the first convolution result;

[0119] The second feature is fed into at least one convolutional layer for processing to obtain the second convolution result;

[0120] The first and second convolution results are sent to the fully connected layer for processing to obtain the action prediction value of the previous time step.

[0121] Get the truth value of the action at the previous moment;

[0122] The objective function is determined based on the predicted action value and the true action value of the previous moment.

[0123] The action that minimizes the objective function is the optimal action.

[0124] like Figure 3As shown, for example, an action prediction network is used to predict the value of the action at the previous moment.

[0125] Figure 3 In the middle, the action prediction network takes two 224×224 RGB images as input, namely the first observation image x at the current time. t The second observation image x from the previous moment t-1 First, features are extracted using ResNet-18 networks (including a second and a third ResNet-18 network). These features are then fed into five convolutional layers, each with 512, 128, 16, 16, and 16 filters respectively, a 3×3 kernel, and a span of 1. ReLU is used as the activation function between each layer. Finally, a fully connected layer with 256 hidden units and ReLU activation is applied, outputting an 8-element vector where each position represents the corresponding action, with the highest probability action being 1 and the rest being 0. An additional optimization objective function defined using cross-entropy loss is shown below:

[0126]

[0127] i is the index of the action, a (i.e., ...) Figure 1 (π) and The true value and predicted value of the action at time t-1 (i.e. the previous time).

[0128] The smaller the value of the additional optimization objective function defined by the cross-entropy loss, the greater the corresponding reward and the better the action.

[0129] In one embodiment, the reward function in an asynchronous dominant actor-critic network is:

[0130] When a special termination signal is issued and the target word becomes visible, the reward is determined to be the preset reward for achieving the target;

[0131] When no special termination signal is issued, the reward is determined to be similarity.

[0132] Otherwise, the reward will be determined as a time penalty.

[0133] Specifically, the goal of this application's indoor target-driven visual navigation is to minimize the distance between the initial position and the target. The purpose of improving the reward function through semantic similarity measurement is to bring the agent closer to the object with the highest similarity to the target. A special signal, DONE, is used to prompt the agent to learn the task. The proposed new reward function is as follows:

[0134]

[0135] When the DONE signal is issued and the target is visible, the task for this round is completed, and a reward of 10 is given for reaching the target. If the DONE signal is not issued, to accelerate convergence, the agent is encouraged to approach objects that are semantically closer to the target, and a reward of 10 is given. similarity, That is, the object with the highest semantic similarity in each step, k∈(0,1) is the scaling factor, for example k=0.1; in addition, a small time penalty (-0.01) is added as an immediate reward.

[0136] The indoor target-driven visual navigation method provided in this application uses action prediction as an auxiliary task to evaluate key differences between sequential states. Through training, it can predict the difference between the current state and the target, and improve the reward function through semantic similarity measurement. These two aspects can solve the sparse reward problem in deep reinforcement learning. This application can improve the generalization ability to unknown scenes and targets, solve the sparse reward problem, accelerate the convergence speed of training, and thus improve navigation performance.

[0137] Reference Figure 4 The diagram illustrates a structural schematic of an indoor target-driven visual navigation device according to an embodiment of this application.

[0138] like Figure 4 As shown, the indoor target-driven visual navigation device 400 may include:

[0139] The acquisition module 410 is used to acquire the observed image and target word at the current moment;

[0140] The first determining module 420 is used to observe the image and target word input into the global feature network to determine the global feature vector;

[0141] The second determining module 430 is used to observe the image and the target word input local feature network to determine the local feature vector;

[0142] Processing module 440 is used to process global feature vectors and local feature vectors into a joint vector through a fully connected layer in the embedding space;

[0143] Output module 450 is used to output the action selected at the current moment and the corresponding reward based on the asynchronous advantage actor-critic network of the joint vector.

[0144] Optionally, the first determining module 420 is also used for:

[0145] The observed image is encoded using the spatial location of the top convolutional layer of the first ResNet-18 network to obtain a global embedding vector;

[0146] The target word is embedded using the Word2Vec model to obtain word embedding vectors;

[0147] The global embedding vector and word embedding vector are activated by the softmax function to obtain the spatial attention probability distribution;

[0148] The spatial attention probability distribution and the global embedding vector are multiplied point by point to obtain the global feature vector.

[0149] Optionally, the second determining module 430 is also used for:

[0150] The observed image is input into YOLO-v3 for image detection to obtain local embedding variables, the center coordinates of the observed image, and the bounding box size.

[0151] The target word is embedded using the Word2Vec model to obtain word embedding vectors;

[0152] The context feature vector is obtained based on the local embedding variables, the center coordinates of the observed image, the bounding box size, and the word embedding vector;

[0153] Based on the context feature vector, the local feature vector is obtained.

[0154] Optionally, the second determining module 430 is also used for:

[0155] Based on the observed image and the target word, determine whether the target word is in the observed image;

[0156] Based on local embedding variables and word embedding vectors, determine the cosine similarity between the observed image and the target word;

[0157] Whether the target word is in the observed image, the center coordinates of the observed image, the size of the bounding box, and the cosine similarity constitute the contextual feature vector.

[0158] Optionally, the second determining module 430 is also used for:

[0159] Flatten the context feature vector and use it as a local feature vector.

[0160] Optionally, the device also includes a motion optimization module for optimizing motion, including:

[0161] Acquire the first observation image at the current moment and the second observation image at the previous moment;

[0162] The first feature is obtained by using a second ResNet-18 network to extract features from the first observed image.

[0163] The second observation image is used to extract features using a third ResNet-18 network to obtain the second features;

[0164] The first feature is fed into at least one convolutional layer for processing to obtain the first convolution result;

[0165] The second feature is fed into at least one convolutional layer for processing to obtain the second convolution result;

[0166] The first and second convolution results are sent to the fully connected layer for processing to obtain the action prediction value of the previous time step.

[0167] Get the truth value of the action at the previous moment;

[0168] The objective function is determined based on the predicted action value and the true action value of the previous moment.

[0169] The action that minimizes the objective function is the optimal action.

[0170] Optionally, the reward function in an asynchronous advantage actor-critic network is:

[0171] When a special termination signal is issued and the target word becomes visible, the reward is determined to be the preset reward for achieving the target;

[0172] When no special termination signal is issued, the reward is determined to be similarity.

[0173] Otherwise, the reward will be determined as a time penalty.

[0174] This embodiment provides an indoor target-driven visual navigation device that can execute the above-described method. Its implementation principle and technical effects are similar, and will not be repeated here.

[0175] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 5 The diagram shows a structural schematic of an electronic device 300 suitable for implementing embodiments of this application.

[0176] like Figure 5 As shown, the electronic device 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage section 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the device 300. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0177] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. Drive 310 is also connected to I / O interface 306 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0178] In particular, according to embodiments of this disclosure, the above references Figure 1 The described process can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing the above-described indoor target-driven visual navigation method. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311.

[0179] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0180] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be located in a processor. The names of these units or modules do not necessarily constitute a limitation on the unit or module itself.

[0181] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, a mobile phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0182] In another aspect, this application also provides a storage medium, which may be the storage medium included in the aforementioned apparatus in the above embodiments; or it may be a standalone storage medium not assembled into the device. The storage medium stores one or more programs, which are used by one or more processors to execute the indoor target-driven visual navigation method described in this application.

[0183] Storage media, including both permanent and non-permanent, removable and non-removable media, can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0184] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0185] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. An indoor target-driven visual navigation method, characterized in that, The method includes: Obtain the observed image and target word at the current moment; The observed image and the target word are input into a global feature network to determine a global feature vector, including: encoding the observed image through the spatial position of the top convolutional layer of a first ResNet-18 network to obtain a global embedding vector; embedding the target word through a Word2Vec model to obtain a word embedding vector; using the global embedding vector and the word embedding vector through a softmax activation function to obtain a spatial attention probability distribution; and multiplying the spatial attention probability distribution and the global embedding vector point-by-point to obtain the global feature vector. The observed image and the target word are input into a local feature network to determine local feature vectors; In the embedding space, the global feature vector and the local feature vector are processed into a joint vector through a fully connected layer; The joint vector is based on the asynchronous dominant actor-critic network, which outputs the action selected at the current moment and the corresponding reward for that action.

2. The method according to claim 1, characterized in that, The observed image and the target word are input into a local feature network to determine local feature vectors, including: The observed image is input into YOLO-v3 for image detection to obtain local embedding variables, the center coordinates of the observed image, and the bounding box size. The target word is embedded using the Word2Vec model to obtain a word embedding vector; The context feature vector is obtained based on the local embedding variables, the center coordinates of the observed image, the bounding box size, and the word embedding vector; Based on the context feature vector, the local feature vector is obtained.

3. The method according to claim 2, characterized in that, The step of obtaining the context feature vector based on the local embedding variables, the center coordinates of the observed image, the bounding box size, and the word embedding vector includes: Based on the observed image and the target word, determine whether the target word is in the observed image; Based on the local embedding variables and the word embedding vector, the cosine similarity between the observed image and the target word is determined; The context feature vector is composed of whether the target word is in the observed image, the center coordinates of the observed image, the size of the bounding box, and the cosine similarity.

4. The method according to claim 2, characterized in that, The step of obtaining the local feature vector based on the context feature vector includes: The context feature vector is flattened to become the local feature vector.

5. The method according to any one of claims 1-4, characterized in that, The method further includes optimizing the action, including: Acquire the first observation image at the current moment and the second observation image at the previous moment; The first feature is obtained by using a second ResNet-18 network to extract features from the first observed image. The second observed image is used to extract features using a third ResNet-18 network to obtain the second feature; The first feature is fed into at least one convolutional layer for processing to obtain the first convolution result; The second feature is fed into at least one convolutional layer for processing to obtain the second convolution result; The first convolution result and the second convolution result are sent to the fully connected layer for processing to obtain the action prediction value of the previous time step; Obtain the truth value of the action at the previous moment; Based on the predicted action value and the true action value of the previous moment, determine the optimization objective function; The action corresponding to the minimum objective function is the optimal action.

6. The method according to any one of claims 1-4, characterized in that, The reward function in the asynchronous dominant actor-critic network is: When a special termination signal is issued and the target word becomes visible, the reward is determined to be the preset reward for achieving the target; When the special termination signal is not issued, the reward is determined to be similarity. Where k is the scaling factor and i is the index of the action; Otherwise, the reward is determined to be a time penalty.

7. An indoor target-driven visual navigation device, characterized in that, The device comprises: The acquisition module is used to acquire the observed image and target word at the current moment; The first determining module is used to input the observed image and the target word into a global feature network to determine a global feature vector, including: encoding the observed image through the spatial position of the top convolutional layer of a first ResNet-18 network to obtain a global embedding vector; embedding the target word through a Word2Vec model to obtain a word embedding vector; using the global embedding vector and the word embedding vector through a softmax activation function to obtain a spatial attention probability distribution; and multiplying the spatial attention probability distribution and the global embedding vector point-by-point to obtain the global feature vector. The second determining module is used to input the observed image and the target word into a local feature network to determine local feature vectors; The processing module is used to process the global feature vector and the local feature vector into a joint vector through a fully connected layer in the embedding space; The output module is used to output the action selected at the current moment and the corresponding reward based on the asynchronous dominant actor-critic network of the joint vector.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the indoor target-driven visual navigation method as described in any one of claims 1-6.

9. A readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the indoor target-driven visual navigation method as described in any one of claims 1-6.

Images