A visual target-driven navigation system for robots in unknown indoor environments

Abstract

The present invention discloses a visual target driven navigation system for robots in unknown indoor environments, including a feature preprocessing module, a neural network module and a training interaction module. The feature preprocessing module is used to preprocess the original sensor data of the robot's current environment and navigation target, and to reconstruct the virtual scene layout and structural information; the neural network module is used to organize the memory reasoning model, the perception model and the planning decision model in an orderly manner; the training interaction module is used to be responsible for the orderly management and operation of the entire platform based on the configuration file, the main function entry, the component initialization, the environment, the optimizer, the loss function and the intelligent agent, so as to support the rapid realization of visual target driven navigation in unknown indoor environments and narrow the gap between virtual environments and real physical environments. The present invention can effectively improve the training and testing efficiency in the construction of visual target driven navigation processes in unknown indoor environments.

Description

Technical Field

[0001] The invention belongs to the field of robot intelligent navigation and artificial intelligence control, and in particular relates to a visual target driven navigation system used in an indoor unknown environment. Background Art

[0002] Advanced machine learning theory focuses on breakthroughs in theoretical methods such as adaptive learning and autonomous learning, to achieve artificial intelligence with high explainability and strong generalization capabilities.

[0003] Traditional indoor navigation methods rely heavily on predetermined maps, or on-the-fly mapping, which is expensive in terms of time and manpower. In addition, due to changes in the environment, maps are only accurate at the time of collection. We believe that this strict requirement for obtaining high-quality maps fundamentally limits the feasibility of robotic systems in our dynamic world. The visual target-driven navigation of robots in unknown indoor environments involved in this patent is an emerging indoor navigation technology solution, which will provide more feasible solutions for robot navigation methods and application scenarios, and provide impetus for the integrated innovation of new-generation information technology and manufacturing equipment.

[0004] Visual target-driven navigation of robots in unknown indoor environments closely links artificial intelligence technology with robot navigation technology. Its ultimate goal is to endow robots with artificial intelligence information encoding, processing, memory and reasoning capabilities, so that robots can perform navigation tasks based solely on visual information in scenarios without additional auxiliary positioning information such as maps and satellite signals. This poses a higher challenge to the robot's ability to observe the current environment and recognize, remember, reason about navigation targets, and make decisions for the next step. However, (1) it is very difficult to train a navigation agent in the real physical world because training in real scenarios consumes a lot of time and is dangerous, and the robot's movements are difficult to control and the experimental conditions are difficult to reproduce. (2) Using deep learning to preprocess a variety of raw data requires strong computing power, and the robot's onboard control terminal does not have the ability to carry out large-scale training and testing of artificial intelligence algorithms.

[0005] Therefore, there is an urgent need for an experimental research system that integrates artificial intelligence methods and robot visual navigation. The system has the ability to simulate robot navigation behavior and train and test artificial intelligence algorithms. The emergence of this system can provide researchers with convenient, flexible and fast verification of robot navigation effects, and provide a reliable experimental reference for further generalization of robot navigation systems to the real world. Summary of the invention

[0006] In order to solve the above-mentioned deficiencies in the prior art, the present invention proposes a visual target driven navigation research system for indoor unknown environments.

[0007] In order to achieve the above-mentioned purpose, the technical solution of the present invention provides a visual target driven navigation system for robots in unknown indoor environments, including a feature preprocessing module, a neural network module and a training interaction module.

[0008] The feature preprocessing module is used to preprocess the original sensor data of the robot's current environment and navigation target, and to reconstruct the virtual scene layout and structure information;

[0009] The neural network module is used to organize the memory reasoning model, the perception model and the planning decision model in an orderly manner;

[0010] The training interaction module is responsible for the orderly management and operation of the entire platform based on the configuration file, main function entry, component initialization, environment, optimizer, loss function and intelligent agent, so as to support the rapid realization of visual target driven navigation in indoor unknown environments and narrow the gap between virtual environments and real physical environments.

[0011] Moreover, it supports adding required network modules and algorithm models. When the data formats of the observation environment and navigation targets change, the feature extraction network for processing the observation environment and navigation targets can be flexibly replaced and modified.

[0012] Furthermore, to achieve automatic indoor robot navigation, the following steps are performed,

[0013] Step 1: Determine the simulation environment, including using a simulation environment publicly available in the field as the training environment for the robot, and perform multi-threaded encapsulation of scene layout and structural information through environmental reconstruction to streamline environmental data;

[0014] Step 2: Extracting navigation target features, including pre-extracting features from the original sensor data, and extracting features using different feature extraction methods based on the data format of prior knowledge;

[0015] Step 3: feature fusion, including fusing the features extracted from the simulation environment with the features of the navigation target to form a navigation feature vector;

[0016] Step 4: Strategy training, including determining the basic algorithm of reinforcement learning, determining components and interaction parameters, including network model, navigation logic network part, and providing object dictionary and action dictionary;

[0017] Step 5: Hyperparameter determination, including determining the maximum segment length, the maximum number of steps that the agent can execute, and the learning rate;

[0018] Step 6: Component fine-tuning, including fine-tuning the parameters and components of the network model according to the training results to achieve better results;

[0019] Step 7: Implement efficient automatic indoor robot navigation based on the results obtained in step 6.

[0020] The beneficial effects of the present invention are:

[0021] (1) Mapless navigation: The navigation system involved in this patent is a new system of mapless navigation paradigm, which supports intelligent agents to explore and find targets indoors without pre-drawn maps.

[0022] (1) Simulation properties: Compared with traditional navigation simulation systems, the system involved in this patent can support the rapid training of navigation agents, and support the import of mainstream environments and self-made environments to train navigation agents;

[0023] (3) High modularity: The navigation system involved in the present invention supports researchers to quickly get started with related research on indoor visual target-driven navigation, and helps researchers to reproduce existing algorithms in the field. Whether it is the design of a new algorithm or the reproduction of an existing algorithm, the modular framework and detailed parameter settings allow researchers to implement the entire reinforcement learning interaction process without having to start from scratch, and to control the variables to a minimum during comparative experiments, making it easier for experimenters to compare algorithm effects, and can support the rapid implementation of automatic navigation.

[0024] The solution of the present invention is simple and convenient to implement, has strong practicality, solves the problems of low practicality and inconvenience in actual application existing in related technologies, can improve user experience, and has important market value. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a system framework diagram of an embodiment of the present invention.

[0026] Figure 2 The figure is a flowchart of an implementation of an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0028] See also Figure 1, an embodiment of the present invention provides a visual target driven navigation system for a robot in an unknown indoor environment, the system is composed of three modules, namely a feature preprocessing module, a neural network module and a training interaction module. The feature preprocessing module preprocesses the original sensor data of the robot's current environment and navigation target, as well as reconstructs the virtual scene layout and structural information. The neural network module is responsible for organizing the memory reasoning model, the perception model and the planning decision model in an orderly manner. The configuration file, the main function entry, the component initialization, the environment, the optimizer, the loss function and the intelligent agent in the training interaction module are responsible for the orderly management and operation of the entire system. The present invention can effectively improve the construction, training and testing efficiency of the visual target driven navigation process (which can be embodied as an algorithm) in an unknown indoor environment by scientific researchers, and narrow the gap between the virtual environment and the real physical environment.

[0029] This visual target-driven navigation system for robots in unknown indoor environments can be flexibly modified to add required network modules and algorithm models. When the data format of the observation environment and navigation targets changes, the feature extraction network for processing the observation environment and navigation targets can be flexibly replaced and modified without affecting the use of other modules.

[0030] Moreover, its parameter module can flexibly adjust the functions of the system, such as training or testing.

[0031] The embodiment of the present invention is a visual target driven navigation system for robots in unknown indoor environments. The system can help researchers quickly build, train and test the performance of the robot's indoor visual target driven navigation process to achieve efficient automatic indoor robot navigation. The embodiment flow chart is shown in FIG. Figure 2 shown.

[0032] The specific implementation steps are as follows:

[0033] 1. Determine the simulation environment. Use the simulation environment disclosed in the field as the training environment for the robot, and use the environment reconstruction module of the system described in this patent to perform multi-threaded encapsulation of scene layout and structure information to streamline environmental data. In specific implementation, existing simulation frameworks such as AI2-THOR, Habitat, House 3D, and Gibson can be used.

[0034] 2. Navigation target feature extraction. The feature pre-extraction function of the system of the present invention is used to pre-extract features from the original sensor data (e.g., RGB / D). At the same time, for the prior knowledge provided by other data sets, different feature extraction methods can be used to extract features according to the data format of the prior knowledge.

[0035] 3. Feature fusion: The features extracted from the simulation environment and the features of the navigation target are fused to form a navigation feature vector.

[0036] 4. Strategy training. Strategy training requires determining the basic algorithm of reinforcement learning, including: model output, loss function type and related parameters of reinforcement learning algorithm itself. The next step is to determine the components and interaction parameters, including: network model design, navigation logic network part. Object dictionary and action dictionary need to be provided in the process.

[0037] 5. Hyperparameter determination: Before the program runs, you need to determine the maximum segment length, the maximum number of steps that the agent can execute, and the learning rate.

[0038] 6. Component fine-tuning. In this step, it is usually necessary to fine-tune the parameters and components of the network model according to the training results to achieve better results.

[0039] 7. Based on the results obtained in step 6, efficient automatic indoor robot navigation is achieved.

[0040] During specific implementation, those skilled in the art may use software technology to realize the automatic operation of the above process.

[0041] The visual target driven navigation research system for robots in unknown indoor environments provided by the embodiments of the present invention is based on Python and Pytorch architecture. In addition to supporting the import of various simulation environments, the biggest function of this system is to lower the research threshold and streamline the research process, thereby accelerating process development and comparative testing. In order to achieve this, the system forms different independent and complete modules, and forms navigation processes and corresponding complete experiments and tests by calling the components of the modules. The modularly organized code is very flexible.

[0042] The generation, training and testing of the navigation process (which can be expressed as an algorithm) are all completed by writing a configuration file. The parameters in the configuration file include both the selection of module components and the setting of various hyperparameters. The completion of a configuration file is roughly divided into three steps: 1. Determination of the basic reinforcement learning algorithm; 2. Determination of components and interaction parameters; 3. Determination of hyperparameters. In the current research field, 2 often includes the acceleration step of data preprocessing.

[0043] 1. Selection of basic reinforcement learning algorithms

[0044] The reason why the choice of basic algorithm is put first is that no matter how the network model is modified or how deep learning techniques are added, some basic input and output quantities, network structure, basic elements of loss function and hyperparameters have been determined when selecting the basic reinforcement learning algorithm.

[0045] In this system, the basic method of reinforcement learning is selected through different main function entrances, but at the same time, the configuration file must contain the corresponding hyperparameters and components. There are different entrances depending on whether it is testing or training, single model method or multi-model method. The main function is responsible for reading the configuration file, generating the experiment folder, generating the record file, and calling the trainer or tester according to the configuration file to start a specific test or training. Data will be output regularly and the model will be saved regularly.

[0046] 2. Determination of components and interaction parameters

[0047] A major feature of this framework is that it modularizes the structures and mechanisms required for a complete algorithm study, which can be easily combined and partially adjusted, and defines clear interfaces for customizing and expanding components. After the basic reinforcement learning algorithm is selected, it is necessary to further select or customize the corresponding components in the module according to the needs of navigation. This part of the design configuration file writing is mainly composed of three parts.

[0048] Network model design: The network model is the core of the navigation process. This framework divides the network into three parts: (1) perception input and preprocessing part; (2) navigation logic network part; (3) network output layer.

[0049] The perception input and preprocessing part needs to solve the problem of generalized navigation of the robot in an unfamiliar environment. The form and amount of information of the input sensor data need to be suitable and sufficient to support the intelligent agent to make effective decisions. In this system, some widely used perception input processing networks have been pre-integrated, which can be easily combined into information-rich input feature vectors, and even pre-processed and extracted, thereby accelerating the training process. After determining the input amount, you only need to specify the file name of the required data for the object dictionary and the target dictionary in the configuration file. In the generated environment, these data will be automatically obtained and passed to the neural network according to the string representation in the object dictionary and the target dictionary.

[0050] The function of the navigation logic network is to judge and process the state feature vector and finally output the optimal action. The architecture and parameters of this part of the network will directly determine the quality of the strategy. The network with multi-layer perceptron (MLP) and long short-term memory neural network (LSTM) structure can be pre-implemented as routines to facilitate development and implementation. By analyzing the characteristics and logic of the specific problem of navigation, different structural designs can be inspired.

[0051] The role of the network output layer is determined by the selected reinforcement learning algorithm. According to different algorithms, value function data of different dimensions are output.

[0052] In this system, the embodiment provides a process for secondary packaging of the original environment, which includes the data preprocessing part. The data after preprocessing is called secondary observation data. Therefore, the design of the neural network in the present invention can be determined directly from the secondary observation data - for example, a network that requires target detection results, resnet50 feature vectors, and glove for feature extraction of navigation targets, as long as the preprocessing operation is completed, the file name of the secondary observation data can be directly specified in the object dictionary and the target dictionary, and the network code does not need to call or reproduce the preprocessing network part. Except for some linear layers of dimensional scaling, the network design can almost start directly from the navigation logic network part.

[0053] 3 Interaction parameters and other components

[0054] Interaction parameters refer to a series of parameter protocols required for the agent to interact with the environment. They include an object dictionary and a target dictionary, which are used to specify the data provided by the environment to the agent; an action dictionary, which is used to specify the actions that the agent can send to the environment and needs to be written according to certain rules to facilitate environmental recognition; and a reward dictionary, which is used to specify how the environment provides rewards to the agent under what circumstances.

[0055] The object dictionary and target dictionary have been determined above; the object dictionary should be defined according to the actions supported by the environment, such as the AI2-THOR discrete data set, which supports 45-degree rotation and 30-degree pitch; the reward dictionary uses the string of the event as the key value and the value of the reward as the value. For example, the reward for finding the target adopted by multiple tasks is 10 points, and the collision deducts 0.1 points. Each step deducts 0.01 points, which can be written as: reward dictionary = {collision: -0.1, number of steps: -0.01, target successfully found: 10, target failed to find: 0}. When a new event is needed, it can be modified in the environment code.

[0056] The next step is to determine the components. The environment to be used can be determined and replaced in advance; the loss function has been determined when the basic reinforcement learning algorithm is selected, but you can choose more finely whether the loss function has entropy and GAE; the trainer has been basically determined when the reinforcement learning algorithm is selected, but if you want to use a deep learning method such as meta-learning, you need to add a new trainer; the executor will be different depending on the parallel situation of the model, for example, a3c and a2c require different executors; the optimizer can be freely selected. When the model needs to synchronize parameters, it is recommended to use a shared series optimizer, which will use the same optimizer and save the same state to optimize the model of each thread, thereby accelerating convergence; it is necessary to determine the information that the agent records and needs to pass to the model, such as the previous action, the distribution of the previous action, and other historical information.

[0057] Following are the definitions of these modules.

[0058] Agent: An agent can be considered as an object with independent intelligence that can interact with the environment. Conceptually, it receives information from the environment, including its own information such as its own history, which is generally called observation. Then, based on these observations, it selects an action in a discrete or continuous action space and feeds it back to the environment.

[0059] Environment: The environment is an object that the agent is in or needs to interact with. It is not necessarily a physical space, but can also be a system that the agent needs to solve. All the information about the environment is often not or even impossible to be given to the agent, but when the environment is affected by the action of the agent, it may change the state of the environment. The agent itself may also obtain different information from the previous environment due to the change of state, thereby inferring the overall picture and rules of the learning environment.

[0060] The implementation and encapsulation of the above two concepts in the tool does not distort or modify their concepts.

[0061] Model: The model refers to the neural network model, which is built based on the pytorch framework. The agent is the encapsulation of the model, so that the input and output of the data conform to the concept of reinforcement learning interaction and record some self-information. However, when looking at the model, we can still regard it as a simple input and output system. The model receives dictionary data, and the key value is the user-defined data label. When writing the model, different data in the dictionary are processed accordingly, and finally the data required by the reinforcement learning loss function is output, such as the policy distribution pi and the value v.

[0062] Interaction process: Reinforcement learning algorithms are still algorithms that learn from data, but the data is generated from interactions. Therefore, this tool has a separate class to represent and perform the interaction process, that is, the environment and the agent interact in the executor. The executor is also responsible for recording the experience and test results used for learning, and packaging them and passing them to the loss function calculation.

[0063] Training process: A function used to train an agent. The training process will include a complete training process of one thread (in a multi-threaded multi-model approach), from class initialization, to calling the executor to generate n-step experience, to loss function calculation, gradient clipping, calling the optimizer for backpropagation, recording test results, and so on.

[0064] Test process: A function used to test the model. It can be considered a simplified version of the trainer, but with more detailed data recording and processing code. The single model multi-environment method currently does not include a tester.

[0065] Auxiliary tools: various auxiliary functions, most of which can be called freely.

[0066] 4 Hyperparameter Determination

[0067] Finally, some hyperparameter settings are required during the training phase. This framework summarizes a large number of algorithms, extracts all hyperparameters that can be modified and affect the training results, and integrates them into the configuration file, which is very convenient for adjustment and modification during the research and development process. These parameters have been set to default values ​​without modification. If they need to be modified, they can be overwritten by simply specifying them in the configuration file.

[0068] In specific implementation, the technical solution of the present invention can be realized by those skilled in the art using computer software modularization technology, and the corresponding devices for implementing the solution, such as a computer-readable storage medium storing a computer program corresponding to the technical solution of the present invention and a computer device for running the corresponding computer program, should also be within the scope of protection of the present invention.

[0069] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A visual target driven navigation system for robots in unknown indoor environments, characterized in that: Including feature preprocessing module, neural network module and training interaction module, The feature preprocessing module is used to preprocess the original sensor data of the robot's current environment and navigation target, and to reconstruct the virtual scene layout and structure information; The neural network module is used to organize the memory reasoning model, the perception model and the planning decision model in an orderly manner; The training interaction module is responsible for the orderly management and operation of the entire platform based on the configuration file, main function entry, component initialization, environment, optimizer, loss function and agent, so as to support the rapid realization of visual target driven navigation in indoor unknown environment and narrow the gap between virtual environment and real physical environment; To achieve automatic indoor robot navigation, the following steps are performed, Step 1: Determine the simulation environment, including using a simulation environment publicly available in the field as the training environment for the robot, and perform multi-threaded encapsulation of scene layout and structural information through environmental reconstruction to streamline environmental data; Step 2: Extracting navigation target features, including pre-extracting features from the original sensor data, and extracting features using different feature extraction methods based on the data format of prior knowledge; Step 3: feature fusion, including fusing the features extracted from the simulation environment with the features of the navigation target to form a navigation feature vector; Step 4: Strategy training, including determining the basic algorithm of reinforcement learning, determining components and interaction parameters, including network model, navigation logic network part, and providing object dictionary and action dictionary; Step 5: Hyperparameter determination, including determining the maximum segment length, the maximum number of steps that the agent can execute, and the learning rate; Step 6: Component fine-tuning, including fine-tuning the parameters and components of the network model according to the training results to achieve better results; Step 7: Implement efficient automatic indoor robot navigation based on the results obtained in step 6.

2. The visual target driven navigation system for robots in unknown indoor environments according to claim 1, characterized in that: It supports adding required network modules and algorithm models. When the data format of the observation environment and navigation targets changes, the feature extraction network for processing the observation environment and navigation targets can be flexibly replaced and modified.

Images