A method and system for autonomously constructing and maintaining indoor environment models
By combining scene graphs and recurrent reinforcement learning algorithms, a three-dimensional scene graph environment model is constructed, which solves the problem of insufficient dynamic updating of the environment model and realizes the rapid deployment and long-term autonomy of robots in complex environments.
Patent Information
- Application Number
- CN202310339805.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-31
- Publication Date
- 2025-09-19
- Estimated Expiration
- 2043-03-31
AI Technical Summary
Existing environment models based on scene graphs lack the ability to dynamically and actively update, making it difficult to achieve long-term autonomy and effective deployment of robots in dynamic and complex environments.
Combining scene graphs and partially observable recurrent reinforcement learning algorithms, by constructing a three-dimensional scene graph environment model, and adopting hierarchical path planning and active exploration modules, the autonomous construction and maintenance of mobile robots in indoor environments are realized.
It enables rapid deployment and long-term operation of robots in complex and unknown environments, and improves the efficiency and automation of dynamic updates of environmental models.
Smart Images
Figure CN116358555B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of mobile robot technology, and in particular to a method and system for autonomously constructing and maintaining an indoor environment model. Background Art
[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.
[0003] A scene graph is a data structure that represents the contents of a scene. A three-dimensional scene graph environment model is a scene graph-based environmental representation method that can be used to improve the performance of autonomous robot navigation. Currently, research on scene graph-based environmental models is limited. These models are often constructed manually and lack the ability to dynamically and proactively update, hindering the long-term operation of robots in dynamic and complex environments. Summary of the Invention
[0004] To address the above-mentioned issues, this paper proposes a method and system for autonomously constructing and maintaining indoor environment models. By combining scene graphs with a partially observable recurrent reinforcement learning algorithm, the method achieves the goal of long-term autonomous model construction for mobile robots, actively explores indoor environments, continuously updates models and scene graphs, and promotes the rapid deployment and long-term operation of robots in complex and unknown environments.
[0005] According to some embodiments, the present disclosure adopts the following technical solutions:
[0006] A method for autonomously constructing and maintaining an indoor environment model, comprising:
[0007] Use graph structures at different levels to describe indoor scenes and build a three-dimensional scene graph environment model for mobile robots;
[0008] Based on the 3D scene graph environment model, a hierarchical path planning framework is used to generate the global navigation path and the local navigation path of the neighborhood.
[0009] Determine the target point to be explored, calculate the information gain and path cost from the current position of the mobile robot to the target point to be explored, select the optimal target point, search for a collision-free path from the current position to the optimal target point, and continuously iterate until the conditions are met to complete the autonomous construction of the indoor environment model;
[0010] A partially observable recurrent reinforcement learning algorithm is used to solve the optimal action of the mobile robot under the current observation, obtain the changes of multiple objects in the indoor environment, and maintain and update the indoor environment model.
[0011] According to some embodiments, the present disclosure adopts the following technical solutions:
[0012] An autonomous indoor environment model construction and maintenance system is based on a mobile robot. The control platform of the mobile robot includes:
[0013] 3D scene graph environment construction module: used to describe indoor scenes using graph structures at different levels and build a 3D scene graph environment model for mobile robots;
[0014] Path planning module: used to generate global navigation paths and local navigation paths in the neighborhood based on a three-dimensional scene graph environment model using a hierarchical path planning framework;
[0015] The active exploration module is used to determine the target point to be explored, calculate the information gain and path cost from the current position of the mobile robot to the target point to be explored, select the optimal target point, search for a collision-free path from the current position to the optimal target point, and continuously iterate until the conditions are met to complete the autonomous construction of the indoor environment model;
[0016] The autonomous patrol module is used to use a partially observable recurrent reinforcement learning algorithm to solve the optimal action of the mobile robot under the current observation, obtain changes in multiple objects in the indoor environment, and maintain and update the indoor environment model.
[0017] According to some embodiments, the present disclosure adopts the following technical solutions:
[0018] A non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the method for autonomously constructing and maintaining an indoor environment model is implemented.
[0019] According to some embodiments, the present disclosure adopts the following technical solutions:
[0020] An electronic device includes: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the method for autonomously constructing and maintaining an indoor environment model.
[0021] Compared with the prior art, the present invention has the following beneficial effects:
[0022] This disclosure proposes a method and system for autonomously constructing and maintaining indoor environment models. This system uses a wheeled mobile robot to autonomously construct, update, and maintain indoor environment models. The system includes a three-dimensional scene graph environment construction module, a path planning module, an active exploration module, an autonomous patrol module, and the robot itself. The system's three-dimensional scene graph environment construction module incrementally constructs collected RGBD images and inertial sensor information into a three-dimensional hierarchical scene graph in real time. This facilitates providing tailored information to the robot's different task requirements (dialogue level, task level, navigation level, etc.). This information can be used for loop detection, task planning, hierarchical path planning, and navigation.
[0023] The path planning module can construct a collision-free, safe path between the starting and ending points as needed. The active exploration module guides the robot to explore the map boundary in the direction with the highest information gain until the indoor map is fully known. The autonomous patrol module guides the robot to move and observe between different rooms or locations indoors to actively update the environmental model. This is more efficient and automated than manual updates of the environmental model.
[0024] The present disclosure can use mobile robots to perceive and model the environment, plan robot movement strategies in complex indoor environments, and achieve long-term robot autonomy and autonomous updating and self-maintenance of environmental models. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The accompanying drawings, which constitute a part of the present disclosure, are used to provide a further understanding of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation to the present disclosure.
[0026] Figure 1 This is a framework diagram of the indoor environment model autonomous construction and maintenance system disclosed in this disclosure;
[0027] Figure 2 A system simulation diagram generated for the three-dimensional scene graph environment model disclosed herein;
[0028] Figure 3 A schematic diagram of the hierarchical path planning disclosed in the present invention. DETAILED DESCRIPTION
[0029] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.
[0030] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present disclosure belongs.
[0031] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.
[0032] Example 1
[0033] In one embodiment of the present disclosure, a method for autonomously constructing and maintaining an indoor environment model is provided, comprising:
[0034] Step 1: Use graph structures at different levels to describe the indoor scene and build a 3D scene graph environment model for the mobile robot;
[0035] Step 2: Based on the 3D scene graph environment model, a hierarchical path planning framework is used to generate a global navigation path and a local navigation path for the neighborhood.
[0036] Step 3: Determine the target point to be explored, calculate the information gain and path cost from the current position of the mobile robot to the target point to be explored, select the optimal target point, search for a collision-free path from the current position to the optimal target point, and continuously iterate until the conditions are met to complete the autonomous construction of the indoor environment model;
[0037] Step 4: Use a partially observable recurrent reinforcement learning algorithm to solve the optimal action of the mobile robot under the current observation, obtain the changes of multiple objects in the indoor environment, and maintain and update the indoor environment model.
[0038] As an example, in step 1, a hierarchical environment representation is employed, using graph structures at different levels to describe the indoor scene. The nodes in the scene graph structure represent objects, locations, rooms, and buildings in the indoor environment, and the edges of the graph structure represent the conceptual attribute relationships between nodes. The disclosed 3D scene graph generation algorithm can be deployed on a robot and can generate high-precision scene graphs in real time.
[0039] Based on a mobile robot, the mobile robot body carries multiple sensors such as a three-dimensional laser camera, a color depth camera, an inertial sensor measurement unit, an odometer, etc., which are used to obtain external environment images and the robot's posture.
[0040] In step 2, the graph structures at different levels are respectively a scene graph layer, a navigation layer and a dynamic layer, and the three-layer structure of the scene graph layer, the navigation layer and the dynamic layer constitutes a layered three-dimensional scene graph environment model.
[0041] As an embodiment, a method for constructing a scene graph layer is as follows: indoor environment images and the position and posture of a mobile robot are acquired through an RGBD camera and an inertial measurement unit, respectively; the acquired indoor environment images are passed into a semantic segmentation module for recognition; the recognized objects are converted into point clouds through an open source PCL point cloud library; the point clouds are clustered into different object nodes; the identified obstacles are expanded to separate room nodes; the objects and room nodes are organized through a scene graph through a scene graph generation method; and the connecting edges in the scene graph are divided into passable paths and inpassable paths according to the obstacle relationships in the navigation layer.
[0042] The navigation layer is based on an incremental grid map and is static. After the mobile robot determines the target to be explored, it can use the A* path planning algorithm and a heuristic function to calculate the priority of each node in the grid map, traverse the optimal node, and obtain the shortest path. This shortest path is used as the global navigation path; the dynamic layer is used to characterize the dynamic factors in the environment. The dynamic factors are used as the dynamic level, and a local navigation path is generated based on the dynamic layer, where the local represents the maximum range that the robot can currently perceive. The DWA and other algorithms are used to generate a local path within the perception range, and this path will be directly used for the navigation of the mobile robot.
[0043] A hierarchical path planning framework is used to achieve path planning in dynamic environments with large indoor crowd flow and large object position change rates. Based on the static navigation layer, after the robot determines the target point to be explored, it uses path planning algorithms such as A* to generate a global navigation path.
[0044] The construction of the navigation layer is based on an incremental grid map, which is provided by the Simultaneously Localization and Mapping (SLAM) module. Common laser SLAM algorithms such as Gmapping can be used to construct the grid map.
[0045] The robot uses algorithms such as DWA based on the dynamic layer to generate a local navigation path. This local path takes into account the dynamic obstacles reflected in the dynamic layer, ensuring that the robot can flexibly avoid obstacles in a dynamic and complex environment.
[0046] As an embodiment, in order to enable a mobile robot to quickly explore the entire unknown environment, a method for quickly exploring an unknown environment for a mobile robot is designed. The robot needs to move in a direction with a high comprehensive benefit to explore the unknown boundary.
[0047] In step 3, the method for determining the target point to be explored is:
[0048] For the raster map based on the navigation layer, first extract the boundary of the raster map, and then aggregate the boundary points. Aggregation can use point query method to traverse the found boundary points and search their neighborhood. If its neighborhood point is still a boundary point, continue to search it. Repeat this process until no boundary point is found. All the found boundary points will be aggregated into a class, and the center point of the class is determined as the target point to be explored.
[0049] After determining the target point to be explored, the information gain and path cost from the current position of the mobile robot to the target point to be explored are calculated. The evaluation function is used to evaluate multiple target points to be explored, the optimal target point is selected, and the path planning algorithm is used to search for a collision-free path from the current position to the optimal target point. This process is repeated. During the movement, the real-time positioning and mapping algorithm is used to continuously obtain environmental information and realize the positioning of the mobile robot in the environment. Until there are no boundary points in the entire navigation layer, the preliminary construction of the entire environmental model is completed.
[0050] Specifically, the information gain I and path cost C from the robot's current position to the target point to be explored are calculated. The information gain I is calculated by searching for all boundary points belonging to the target point class near the target point to be explored, assigning a probability of 0.5, and calculating their information entropy, which is used as the information gain. C can be obtained by path planning algorithms such as A*. The evaluation function g = Ce is used. λI To evaluate multiple target points, where λ is an adjustable coefficient.
[0051] A more optimal target point is selected, and the A* path planning algorithm is used to search for a collision-free path from the current position to the optimal target point in the grid map. This process is repeated. During the movement, the robot uses a simultaneous localization and mapping algorithm (SLAM) (such as existing algorithms such as Gmapping) to update the environment model through environmental information obtained by its own sensors and realize the robot's positioning in the environment map. This is done until there are no boundary points in the entire navigation layer, completing the initial construction of the entire environment model.
[0052] As an embodiment, in step 4, a partially observable recurrent reinforcement learning algorithm is used to solve the optimal action of the mobile robot under the current observation, obtain the changes of multiple objects in the indoor environment, and maintain and update the indoor environment model.
[0053] After a mobile robot's indoor environment model is constructed, its environment model may continue to change during the robot's long-term service. A mobile robot cannot continuously observe objects in all rooms—this would waste computing resources and introduce errors in object recognition. Consequently, uncertainty increases as the robot's service progresses. This paper proposes a POMDP (Partially Observable Markov Decision Process) framework and algorithm to control the robot's patrol strategy between different rooms and locations indoors. This allows the robot to patrol rooms or locations with frequently changing objects, ensuring rapid and stable updates of the environment model over the long term.
[0054] Based on a partially observable Markov decision process, the distribution and update frequency of objects in the environment are abstracted into a state space for modeling. A belief function and state transition function are constructed, and a Monte Carlo tree search method is used to solve the problem. This strategy controls the robot's strategy for updating the environment model while patrolling the room. The POMDP planning process assumes that the system parameters are known. The POMDP module simplifies the environment model, accepts the robot's object detection observations, and updates the belief value of the actual state to determine the order in which the robot patrols the rooms or locations.
[0055] Specifically, by constructing a three-dimensional scene graph, analyzing the changes in the positions of object nodes and the update time and frequency, establishing the state space S, belief state B, action set A, state transition function T, observation function O, reward function R, discount factor γ, and using a partially observable recurrent reinforcement learning algorithm to solve the optimal action under the current observation, the robot can observe more object changes, which helps to quickly update the environmental model.
[0056] The state space S is divided into two systems, including the state change S of objects in each room in the room room and robot posture state S robot The action set A refers to the actions that the robot can take when patrolling indoors, A=(Move i ,Observe), where Move i Represents moving to the i-th room, and Observe represents observing the room. t It is defined as the distribution and update of objects observed by the robot through sensors in the current room at time t and the robot's position. Due to distance, occlusion and sensor intensive reading problems, the robot's observation results contain noise. The observation function can be expressed as
[0057] O(a t ,S t+1 ,O t+1 )=Pro(o t+1 |S t+1 ,a t )+Err,
[0058] Among them, O(S t+1 ,a t ,O t+1 ) indicates that the robot takes action a at time t t , the observation quantity at time t+1 is O t+1 And the environmental state is S t+1 The probability of S t+1 and a t The sum of the conditional probability and the observation error probability under the given conditions. Err is the sensor noise, which is inversely proportional to the distance between the robot and the object. The state transition function is defined as T(s,a,s')=P(s'|s,a). When the Move action or the Observe action is executed, the state will change. Belief state B t+1 =Pro(S t+1 |B t ,o t ,a+t), represents the posterior probability distribution of each state. It has Markov properties and is calculated based on historical observations and actions. The reward function R represents the reward the robot receives for taking action a in state s. If the robot observes an object change, it receives a reward of +10. To prevent the robot from observing only one room, a threshold is set to give the robot a chance to search other rooms or locations where object changes may occur.
[0059] Example 2
[0060] In one embodiment of the present disclosure, a system for autonomously constructing and maintaining an indoor environment model is provided. The system is based on a mobile robot, and a control platform of the mobile robot includes:
[0061] 3D scene graph environment construction module: used to describe indoor scenes using graph structures at different levels and build a 3D scene graph environment model for mobile robots;
[0062] Path planning module: used to generate global navigation paths and local navigation paths in the neighborhood based on a three-dimensional scene graph environment model using a hierarchical path planning framework;
[0063] The active exploration module is used to determine the target point to be explored, calculate the information gain and path cost from the current position of the mobile robot to the target point to be explored, select the optimal target point, search for a collision-free path from the current position to the optimal target point, and continuously iterate until the conditions are met to complete the autonomous construction of the indoor environment model;
[0064] The autonomous patrol module is used to use a partially observable recurrent reinforcement learning algorithm to solve the optimal action of the mobile robot under the current observation, obtain changes in multiple objects in the indoor environment, and maintain and update the indoor environment model.
[0065] The mobile robot body also carries multiple sensors such as a three-dimensional laser camera, a depth camera, an inertial sensor measurement unit, and an odometer.
[0066] The system's 3D scene graph environment model construction module can incrementally construct collected RGBD images and inertial sensor information into a 3D hierarchical scene graph in real time. The path planning module can construct a collision-free and safe path between the starting and ending points as needed. The active exploration module can guide the robot to explore in the direction of maximum information gain at the map boundary until the entire indoor map is known. The autonomous patrol module can guide the robot to move and observe between different rooms or locations indoors to achieve the purpose of actively updating the environmental model. Mobile robots need to navigate indoor environments with dense crowds, dynamic changes, and high object density. The ability to quickly and smoothly plan paths is not only the basis for the robot to traverse multiple navigation points, but also a prerequisite for autonomous exploration of the environment and active patrol.
[0067] Example 3
[0068] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, which is used to store computer instructions. When the computer instructions are executed by a processor, the steps of the method for autonomously constructing and maintaining an indoor environment model are implemented.
[0069] Example 4
[0070] In one embodiment of the present disclosure, an electronic device is provided, comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device performs the steps of the method for autonomously constructing and maintaining an indoor environment model.
[0071] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0073] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Those skilled in the art should understand that, based on the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.
Claims
1. A method for autonomously constructing and maintaining an indoor environment model, characterized in that: include: Use graph structures at different levels to describe indoor scenes and build a three-dimensional scene graph environment model for mobile robots; Based on the 3D scene graph environment model, a hierarchical path planning framework is used to generate the global navigation path and the local navigation path of the neighborhood. Determine the target point to be explored, calculate the information gain and path cost from the current position of the mobile robot to the target point to be explored, select the optimal target point, search for a collision-free path from the current position to the optimal target point, and continuously iterate until the conditions are met to complete the autonomous construction of the indoor environment model; A partially observable recurrent reinforcement learning algorithm is used to solve the optimal action of the mobile robot under the current observation, obtain the changes of multiple objects in the indoor environment, and maintain and update the indoor environment model.
2. The method for autonomously constructing and maintaining an indoor environment model according to claim 1, wherein: The nodes in the graph structure represent information about objects, places, rooms, and buildings in an indoor environment, and the edges of the graph structure represent conceptual attribute relationships between the nodes.
3. The method for autonomously constructing and maintaining an indoor environment model according to claim 1, wherein: The graph structures at different levels are respectively a path layer, a navigation layer and a dynamic layer, and the three-layer structure of the path layer, the navigation layer and the dynamic layer constitutes a layered three-dimensional scene graph environment model.
4. The method for autonomously constructing and maintaining an indoor environment model according to claim 3, wherein: The path layer is constructed by acquiring indoor environment images and mobile robot postures, identifying the acquired indoor environment images, converting the identified objects into point clouds, clustering the point clouds into different object nodes, performing expansion operations on identified obstacles as separate room nodes, dividing communicable path points into location nodes, connecting room nodes to building nodes, extracting robot IMU information as paths and storing them as path layers.
5. The method for autonomously constructing and maintaining an indoor environment model according to claim 3, wherein: The navigation layer is based on an incremental grid map. The navigation layer is static. After the mobile robot determines the target to be explored, a path planning algorithm can be used to generate a global navigation path. The dynamic layer is used to describe the dynamic factors in the environment. The dynamic factors are used as dynamic levels, and a local navigation path of the neighborhood is generated based on the dynamic layer.
6. The method for autonomously constructing and maintaining an indoor environment model according to claim 1, wherein: The method for determining the target point to be explored is: based on the grid map of the navigation layer, first extracting the boundary of the grid map, and then aggregating the boundary points to determine the target point to be explored.
7. The method for autonomously constructing and maintaining an indoor environment model according to claim 1, wherein: After determining the target point to be explored, the information gain and path cost from the current position of the mobile robot to the target point to be explored are calculated. The evaluation function is used to evaluate multiple target points to be explored, the optimal target point is selected, and the path planning algorithm is used to search for a collision-free path from the current position to the optimal target point. This process is repeated. During the movement, the real-time positioning and mapping algorithm is used to continuously obtain environmental information and realize the positioning of the mobile robot in the environment. Until there are no boundary points in the entire navigation layer, the preliminary construction of the entire environmental model is completed.
8. An indoor environment model autonomous construction and maintenance system, characterized in that: Based on a mobile robot, the control platform of the mobile robot includes: 3D scene graph environment construction module: used to describe indoor scenes using graph structures at different levels and build a 3D scene graph environment model for mobile robots; Path planning module: used to generate global navigation paths and local navigation paths in the neighborhood based on a three-dimensional scene graph environment model using a hierarchical path planning framework; The active exploration module is used to determine the target point to be explored, calculate the information gain and path cost from the current position of the mobile robot to the target point to be explored, select the optimal target point, search for a collision-free path from the current position to the optimal target point, and continuously iterate until the conditions are met to complete the autonomous construction of the indoor environment model; The autonomous patrol module is used to use a partially observable recurrent reinforcement learning algorithm to solve the optimal action of the mobile robot under the current observation, obtain changes in multiple objects in the indoor environment, and maintain and update the indoor environment model.
9. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the method for autonomously constructing and maintaining an indoor environment model according to any one of claims 1 to 7 is implemented.
10. An electronic device, characterized in that: include: A processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement an autonomous construction and maintenance method for an indoor environment model as described in any one of claims 1 to 7.