A zero-shot indoor robot visual navigation method based on class-agnostic network
By using the Class Independent Network (CIRN) method, which utilizes target detection information and semantic similarity as states, navigation capability is decoupled from target features, solving the navigation problem of unseen targets in visual navigation and achieving high-success-rate cross-scene and cross-target navigation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NORTHWESTERN POLYTECHNICAL UNIV
- Filing Date
- 2023-11-28
- Publication Date
- 2026-04-24
AI Technical Summary
Existing reinforcement learning-based visual navigation methods struggle to navigate when faced with unfamiliar targets, preventing robots from finding new targets in testing environments and hindering the achievement of 'train once, apply everywhere'.
The class-independent network (CIRN) method is adopted, which does not use the original visual information as the state. Instead, it uses the relative semantic similarity between the target detection information and the navigation target as the state. The relationship between objects is learned through graph convolutional networks, decoupling the navigation capability from the features of the navigation target, and constructing a state representation that does not depend on a specific category.
It achieves high-success-rate navigation in different scenarios and with different targets, improves the robot's navigation generalization ability, and enables accurate navigation in unseen targets and scenarios.
Smart Images
Figure CN117685950B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of visual navigation technology, and specifically relates to a visual navigation method for indoor robots. Background Technology
[0002] Visual navigation is a fundamental problem in robotics and artificial intelligence. Goal-driven visual navigation tasks aim to command robots to navigate to specific targets in unfamiliar environments using only visual information. In recent years, visual navigation has attracted widespread attention in the fields of artificial intelligence and computer vision, demonstrating broad applications in areas such as automating home services, warehouse management, and the hospitality industry.
[0003] Due to the success of reinforcement learning-based methods in robotic tasks, many map-free visual navigation methods based on reinforcement learning have emerged. Unlike traditional map-based visual navigation methods, these reinforcement learning-based methods are end-to-end solutions that directly map the combination of visual information and navigation objectives to specific navigation actions. Therefore, they require minimal manual engineering and serve as the foundation for a variety of next-generation AI-driven robotic tasks.
[0004] Currently, although researchers have made good progress in target-driven visual navigation, these methods have significant limitations because the range of navigation targets depends on the types of navigation targets learned during training, and they cannot navigate to targets that have not been learned. This means that the robot needs to be trained on how to find specific classes before it can find those classes during testing. However, in the real world, there are countless classes, making it impossible for a robot to learn how to navigate to every specific class. We hope to have an algorithm that can be "trained once, used everywhere" to solve this problem. Therefore, research on zero-shot target-driven visual navigation (ZSON) tasks has become necessary, the goal of which is to enable robots to find classes that were not trained on during the training phase in a testing environment.
[0005] Previous work typically combined the features of the navigation target with the current visual information as the state for reinforcement learning. While this design allows the robot to clearly understand the navigation target, it also causes the agent to simultaneously learn the features of the navigation target and how to navigate to it. Specifically, this approach couples the agent's navigation capabilities with the features of the navigation target during training. Therefore, during testing, when the agent's task is to find new targets, the combination of the new target and the current visual information forms a completely new state, making it difficult for the agent to understand its navigation target. Therefore, we believe the key to solving zero-shot target-driven visual navigation tasks is to decouple the robot's learned navigation capabilities from the features of the navigation target during training, allowing the robot to learn pure navigation capabilities rather than the ability to navigate to a specific trained target during the training phase. Currently, almost all methods have not yielded good results in addressing the zero-shot target-driven visual navigation problem. Summary of the Invention
[0006] To overcome the shortcomings of existing technologies, this invention provides a zero-shot indoor robot visual navigation method (CIRN) based on class-independent networks. Instead of using raw visual information as the state for reinforcement learning, it uses the target detection information under the current vision as the state, representing different objects using the relative semantic similarity between the object and the navigation target, and sorting their information in descending order based on the relative semantic similarity of multiple objects. Compared to the preset number of object categories and fixed positions of categories in the state matrix, this invention's method can be applied to all scenarios, regardless of the specific category.
[0007] The technical solution adopted by the present invention to solve its technical problem includes the following steps:
[0008] Step 1: Set up the experimental environment;
[0009] Download the AI2-THOR virtual environment package. AI2-THOR contains 120 different 3D virtual rooms, which are divided into kitchen, living room, bedroom and bathroom. Each room type has 30 different rooms, and each type of room is divided into 20 training environments and 10 testing environments.
[0010] During training, a randomly given navigation target is randomly selected from a random location in a training room before each task begins. During testing, a given test target is randomly selected from a random test location in a test room. The training target and the test target are different.
[0011] During training, the room type is fixed, and during testing, a different room type is selected than that used during training.
[0012] Step 2: Construct the network model;
[0013] The system extracts target detection information from visual information, sorts the target detection information of each target in descending order of semantic similarity with the navigation target to form the current features, and then passes the current features to a graph convolutional network to encapsulate the relational features of global and local semantic information. The policy network module uses an LSTM network and two MLP networks to input the policy function and Q function, and determines the action at the current time step based on the policy function.
[0014] Step 3: Evaluation Indicators;
[0015] The evaluation metrics are the average success rate (SR), which represents the success rate of all trained navigation targets; and the successful path weight (SPL), which is the ratio of the shortest path to the actual path multiplied by the success rate. The calculation formulas are as follows:
[0016]
[0017]
[0018] In the formula, S i N represents the total number of test cases, and L represents the number of successful test cases. i P represents the optimal path. i Indicates the actual path;
[0019] Step 4: Training Process;
[0020] At the start of a training trajectory, the experimental environment randomly places the robot at any location in the environment and assigns the robot's current navigation target as the task objective. At each moment, the robot receives the current image and selects the corresponding action based on the image. After a trajectory is completed, the network is updated based on the current trajectory.
[0021] Step 5: Network testing;
[0022] For each saved model, the navigation target is tested; during testing, each test case has a fixed starting position and navigation target; the maximum step size during testing is 50 steps, and the success rate and successful path weight are determined based on the number of successful results after testing.
[0023] Preferably, a navigation task in the AI2-THOR virtual environment includes a scene S, an initial point p, and a navigation target o; the robot's goal is to find the navigation target o in the environment from the initial position p within a given number of steps; the robot's action space is limited to six actions: forward, left turn, right turn, look down, look up, and complete; in each step, the robot receives an image from the current camera in the scene, with a size of 300*300*3, and acts in one of the following two ways: 1. select an action and move accordingly, or 2. select to complete the action and terminate the task; the set of all steps from the start to the end of the task is called a trajectory episode; the task is considered successful if the following four conditions are met: 1. the robot selects to complete the action; 2. the robot is no more than 1 meter away from the given target object; 3. the target object is within the robot's field of vision; 4. the robot has not exceeded the maximum number of steps limit.
[0024] Preferably, the training in step 4 is performed 900,000 times.
[0025] The beneficial effects of this invention are as follows:
[0026] 1. This invention proposes a CIRN model to solve the problem of zero-shot visual navigation.
[0027] 2. The present invention achieves optimal accuracy on existing task settings.
[0028] 3. This invention proposes a more challenging task—cross-scene and cross-target visual navigation—while still achieving a high success rate. Attached Figure Description
[0029] Figure 1 This is a conceptual diagram of the class-independent relational network navigation of the present invention.
[0030] Figure 2 This is a structural diagram of the class-independent relational network model of the present invention. Detailed Implementation
[0031] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0032] The purpose of this invention is to overcome the shortcomings of existing technologies. For navigation tasks, target detection information is sufficient for the robot to locate the spatial position of the target. Compared with other methods that directly embed the word or visual features of the object into the state, this invention uses the relative category of the object and the navigation target to distinguish the relationship between different targets. Specifically, it uses the semantic similarity between the object and the navigation target, aiming to eliminate features belonging to specific categories from the state representation. It is this category feature that leads to the coupling between the robot's navigation ability and the navigation target. The target detection information from the current perspective is sorted according to the semantic similarity between the object and the navigation target, forming a matrix of target detection information, and this matrix is used as the current state. Regardless of its specific category, the unique identifier that distinguishes objects in this state is their semantic similarity with the navigation target. Based on the above idea, a Class-Independent Relationship Network (CIRN) is proposed. Figure 1 As shown, the key idea of class-independent relational networks is to ensure that the information received by the agent does not change significantly due to changes in the navigation target and the scene. Specifically, class-independent relational networks use object detection information and semantic similarity with the navigation target as states to effectively decouple the feature information of the navigation target from the navigation decision-making process. This enables the agent to navigate more flexibly and accurately when faced with new navigation targets.
[0033] This invention provides a zero-shot indoor robot visual navigation method (CIRN) based on class-independent networks. The key idea of this method is to ensure that the information received by the agent does not change significantly due to changes in the navigation target and the scene. Regarding visual information processing, it is considered that the original visual information contains too much background color, texture, and other information irrelevant to the navigation task, which can easily lead to overfitting of the network model. In navigation tasks, target detection information is sufficient for the robot to locate the spatial position of the target, and target detection information does not contain irrelevant information such as texture, making it very suitable for state representation in zero-shot visual navigation. Therefore, CIRN does not use the original visual information as the state for reinforcement learning, but instead uses the target detection information under the current vision as the state. Regarding how to define the navigation target, it prefers to use relative classes rather than using word vectors or visual features to represent the object class. This is because word vectors and visual features contain unique features that uniquely identify the navigation target, and the goal is to eliminate these features belonging to a specific category from the state representation. Therefore, the relative semantic similarity between objects and navigation targets is used to represent different objects. Then, the information of multiple objects is sorted in descending order based on their relative semantic similarity. Compared to the pre-defined number of object categories and their fixed positions in a state matrix, the method of this invention can be applied to all scenarios, regardless of the specific category. Therefore, the unique identifier that distinguishes objects is their semantic similarity to the navigation target, regardless of their category. Figure 1In the examples shown, the red-marked objects represent the navigation target, while the other yellow-marked objects represent other objects in the environment. The numbers on the bounding boxes indicate the semantic similarity between the object and the navigation target. For example, in the left image, the navigation target is a toaster. In the test environment, although the test environment and test target differ from the training environment, the CIRN state set does not contain features of the objects or environment. Therefore, the difference between the two lies only in spatial location and semantic similarity.
[0034] The technical solution adopted in this invention includes the following steps:
[0035] (1) Virtual environment preparation:
[0036] The method of this invention was trained and tested in the AI2-THOR virtual environment. AI2-THOR contains 120 different 3D virtual rooms, categorized by type as kitchen, living room, bedroom, and bathroom, with different items in each room type. A navigation task in the AI2-THOR virtual environment includes a scene S, an initial point p, and a navigation target o. The agent's goal is to find the navigation target o from the initial position p within a given number of steps. The agent's action space is limited to six actions: forward, left turn, right turn, look down, look up, and Done. In each step, the robot receives an image from the current camera, with a size of 300*300*3, and can act in one of two ways: 1. Select an action and move accordingly, or 2. Select the Done action to terminate the task. The set of all steps from the start to the end of the task is called an episode (trajectory). The task is considered successful if the following four conditions are met: 1. The robot selects to complete the action; 2. The robot is no more than 1 meter away from the given target object; 3. The target object is within the robot's field of vision; 4. The robot does not exceed the maximum number of steps.
[0037] Regarding the division of training and testing environments, each room type is divided into 20 training environments and 10 testing environments, with different training and testing environments selected based on the experimental setup. Navigation objectives are divided into training objectives and testing objectives, which do not overlap.
[0038] (2) Model Architecture Design
[0039] Because visual data contains a large amount of navigation-irrelevant information, such as color and texture, this information is not only useless for navigation but may also lead to overfitting of the model to the environment or specific objects. Therefore, to eliminate the differences in visual information between different environments and objects, any visual data is excluded from the state representation, and object detection information is used instead to represent the current state. The proposed CIRN architecture is as follows: Figure 2 As shown, the state of the network is defined as a 20x5 matrix, where each row represents the target detection information r of object i. i =[h,s,x c ,y c [area]. The first element, h, indicates whether the object is the target; if yes, h = 1, otherwise h = 0. The second element, s, represents the semantic similarity between the object and the navigation target. The third and fourth elements, x... c and y c The first element represents the center coordinates of the detection box, and the fifth element, `area`, represents the area of the detection box. The rows are sorted in descending order based on the value of `s`, and the first 20 object detection boxes are selected. If the number of object detection boxes in the current visual data is less than twenty, the remaining rows are set to zero. Semantic similarity is obtained by calculating the cosine similarity of the word embeddings between the object and the target, using the following formula:
[0040]
[0041] Where g i G represents the word embedding of object i. t The word embeddings represent the navigation target. It can be observed that the state information does not contain any visual information. The spatial location information of objects in the state is used to identify their positions, while semantic similarity is used to distinguish different objects. Furthermore, the class information of objects is relative to the navigation target; if an object has a high semantic similarity to the target, its order in the matrix is prioritized, without including detailed information about any specific object. Therefore, during testing, when faced with a new navigation target, the information contained in the state is consistent with the information during training and does not change due to changes in the target category.
[0042] After constructing the state, a Graph Convolutional Network (GCN) is used to learn relationships between objects based on their cosine similarity. In navigation tasks, state information can be viewed as a graph, where each object corresponds to a node, and semantic similarity and spatial location information constitute node features. The GCN performs multiple convolutional operations, progressively updating and integrating the features of each node, enabling the node representation to encapsulate global and local semantic information relationships. The role of the GCN is to learn the relationships between different objects, thereby enhancing the expressive power of the state representation. By learning the connection patterns between nodes in the graph, the model can capture the semantic connections and spatial relationships between objects, which is particularly important for navigation tasks because they need to consider the layout and features of the surrounding environment, not just individual object information. Subsequently, the GCN-processed state is input into a policy network (including LSTM and Actor-Critic networks). The policy network then outputs the policy π corresponding to the current state.
[0043] Example:
[0044] (1) Experimental setup
[0045] First, download the AI2-THOR virtual environment package. AI2-THOR contains 120 different 3D virtual rooms, which are divided into kitchen, living room, bedroom and bathroom types. Each room type has 30 different rooms, and each type of room is divided into 20 training environments and 10 testing environments.
[0046] In the zero-shot cross-target experimental setup, during training, a randomly given navigation target is randomly selected from a random location in a training room before each task begins. During testing, a given test target is randomly selected from a random test location in a test room. The training target and the test target are not the same.
[0047] In the zero-shot cross-target cross-scenario experimental setup, the room type was fixed during training, and a different room type was selected during testing to test the robot's generalization ability to different room types.
[0048] (2) Network Model
[0049] Network models such as Figure 2 As shown, target detection information is first extracted from visual information. The current features are then formed by sorting the target detection information of each target in descending order of semantic similarity to the navigation target. These current features are then passed to a graph convolutional network to encapsulate the relationship between global and local semantic information. The policy network module uses one LSTM network and two MLP networks as inputs for the policy function and Q function, and determines the action at the current time step based on the policy function.
[0050] (3) Evaluation indicators
[0051] The evaluation metrics are the average success rate (SR), which represents the success rate across all trained navigation targets, and the Successful Path Weight (SPL), which is the success rate multiplied by the ratio of the shortest path to the actual path. The formulas are as follows:
[0052]
[0053]
[0054] (4) Training process
[0055] At the start of a training trajectory, the environment randomly places the robot in an arbitrary location and assigns it the current navigation target as its task objective. At each moment, the robot receives the current image and selects the corresponding action based on it. After a trajectory is completed, the network is updated based on the current trajectory. Training was conducted a total of 900,000 times.
[0056] (5) Network Testing
[0057] For each saved model, its navigation target is tested. During testing, each test case has a fixed starting position and navigation target. The maximum step size during testing is 50 steps. After testing, the success rate and successful path weight are determined based on the number of successful attempts.
[0058] Tables 1 to 5 show the comparison results of the method of the present invention with other methods and the experimental results of the method of the present invention in different scenarios.
[0059] Table 1 Comparison of the present invention with other methods in a zero-sample cross-target experimental setting.
[0060]
[0061] Table 2. Results of training and testing of the method of the present invention in the kitchen and bedroom.
[0062]
[0063] Table 3. Test results of the method of the present invention in the kitchen and living room.
[0064]
[0065] Table 4. Results of training and testing of the method of the present invention in the living room and the bathroom.
[0066]
[0067] Table 5. Results of training and testing of the method of the present invention in the living room and kitchen.
[0068]
Claims
1. A zero-shot indoor robot visual navigation method based on class-independent networks, characterized in that, Includes the following steps; Step 1: Set up the experimental environment; Download the AI2-THOR virtual environment package. AI2-THOR contains 120 different 3D virtual rooms, which are divided into kitchen, living room, bedroom and bathroom. Each room type has 30 different rooms, and each type of room is divided into 20 training environments and 10 testing environments. During training, a randomly given navigation target is randomly selected from a random location in a training room before each task begins. During testing, a given test target is randomly selected from a random test location in a test room. The training target and the test target are different. During training, the room type is fixed, and during testing, a different room type is selected than that used during training. Step 2: Construct the network model; Target detection information is extracted from visual information. The target detection information of each target is sorted in descending order according to the semantic similarity between the target detection information and the navigation target to form the current feature. Then, the current feature is handed over to a graph convolutional network to encapsulate the relationship features of global and local semantic information. The policy network module uses an LSTM network and two MLP networks to take the policy function and Q function as inputs, and determines the action at the current time step based on the policy function. Step 3: Evaluation Indicators; The evaluation metrics are the average success rate (SR), which represents the average success rate of all trained navigation targets; and the success path weight (SPL), which represents the success rate multiplied by the ratio of the shortest path to the actual path. The calculation formula is as follows: In the formula, S i N represents the total number of test cases, and L represents the number of successful test cases. i P represents the optimal path. i Indicates the actual path; Step 4: Training Process; At the start of a training trajectory, the experimental environment randomly places the robot at any location in the environment and assigns the robot's current navigation target as the task objective. At each moment, the robot receives the current image and selects the corresponding action based on the image. After a trajectory is completed, the network is updated based on the current trajectory. Step 5: Network testing; For each saved model, the navigation target is tested; during testing, each test case has a fixed starting position and navigation target; the maximum step size during testing is 50 steps, and the success rate and successful path weight are determined based on the number of successful results after testing.
2. The zero-shot indoor robot visual navigation method based on class-independent networks according to claim 1, characterized in that, A navigation task in the AI2-THOR virtual environment includes a scene S, an initial point p, and a navigation target o. The robot's goal is to find the navigation target o from the initial position p within a given number of steps. The robot's action space is limited to six actions: forward, left turn, right turn, look down, look up, and complete. In each step, the robot receives an image from the current camera, with a size of 300*300*3, and acts in one of two ways: 1) selects an action and moves accordingly, or 2) selects to complete the action and terminates the task. The set of all steps from the start to the end of the task is called a trajectory episode. The task is considered successful if the following four conditions are met simultaneously: 1) the robot selects to complete the action; 2) the robot is no more than 1 meter away from the given target object; 3) the target object is within the robot's field of vision; 4) the robot has not exceeded the maximum number of steps.
3. The zero-shot indoor robot visual navigation method based on class-independent networks according to claim 1, characterized in that, The training in step 4 is performed 900,000 times.