Autonomous exploration method and system for fusing environmental mutual information in unknown environment

Abstract

The invention discloses an autonomous exploration method and system for fusing environmental mutual information in an unknown environment, and relates to the field of autonomous exploration environments, and the method comprises the steps: obtaining dense three-dimensional point cloud data and image data of the environment in real time; constructing a dense three-dimensional point cloud map containing semantic mutual information according to dense three-dimensional point cloud data and the image data; determining a target subspace according to the dense three-dimensional point cloud map; determining an optimal target subspace sequence and a guiding path according to the target subspace; adopting a fast expansion random tree algorithm to construct a fast expansion random graph according to the dense three-dimensional point cloud map; determining a local optimal exploration path according to the guiding path based on the fast expansion random graph; and the mobile robot moves along a local optimal exploration path until the optimal target subspace sequence in the whole region is explored, and an environment map of the whole region is finally obtained according to all dense three-dimensional point cloud data and image data. According to the invention, the coverage range and exploration efficiency of the scene are improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous exploration of environments, and in particular to an autonomous exploration method and system for integrating environmental mutual information in an unknown environment. Background Art

[0002] Currently, mobile robots are moving towards autonomy and intelligence, and the autonomous exploration technology of mobile robots in unknown environments has broad application prospects. In recent years, mobile robot technology has been used in disaster relief, search and rescue environmental surveys, and underground mines where it is inconvenient for people to enter. It has also been widely used in many fields such as bomb disposal and smart agriculture. However, robots often lack prior information about the environment and have a limited perception range, which increases the difficulty of decision-making during the exploration process. Due to the limitations of the robot's endurance and the urgency of the task, there is a need for a method to drive the robot to quickly complete environmental exploration. Therefore, in-depth research on autonomous exploration (AE) technology is of great significance and urgent need.

[0003] Autonomous exploration refers to using robots to autonomously explore unknown environments within a reasonable time, search for information and preliminarily establish an environmental model to facilitate the robot's navigation in the environment and achieve subsequent work tasks. Autonomous exploration problems must handle two tasks at the same time: one is to update the environmental model online in real time to discover areas that need to be explored, and the other is to plan a continuous feasible path to guide exploration movement. At present, the main research direction of autonomous exploration is exploration strategy, that is, how to plan the robot's exploration goals and optimal paths, and traverse the entire unknown environment as completely as possible under limited time or resources to establish a map model of the entire environment. This poses a huge challenge to the exploration strategy.

[0004] Most existing autonomous exploration methods only consider the geometric mutual information in the environment, such as the frontier boundary point and the next optimal viewpoint, and adopt a greedy strategy, which often lacks the guidance of semantic mutual information. When the algorithm only focuses on short-term geometric mutual information, such as moving towards the frontier boundary point with the greatest current benefit, the robot is prone to short-sightedness and ignores the long-term goal, and backtracking will occur. Fully utilizing the semantic information of the environment can not only improve the exploration efficiency but also increase the coverage of the scene. The specific design of a decision-making strategy affects the selection of target points, exploration efficiency and exploration behavior. Summary of the invention

[0005] The purpose of this application is to provide an autonomous exploration method and system that integrates environmental mutual information in an unknown environment, so that the mobile robot can independently and efficiently plan the robot's exploration goals and optimal paths in an unknown and complex environment under the conditions of lack of environmental prior information and limited perception range, traverse the entire unknown environment as completely as possible within limited time or resources to establish a map model of the entire environment, improve exploration efficiency and increase the coverage of the scene.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides an autonomous exploration method for integrating environmental mutual information in an unknown environment, including:

[0008] Acquire dense three-dimensional point cloud data and image data of the environment in real time; the dense three-dimensional point cloud data is obtained by scanning the environment with a laser radar carried by a mobile robot; the image data is obtained by photographing with a camera carried by a mobile robot;

[0009] Constructing a dense three-dimensional point cloud map containing semantic mutual information according to the dense three-dimensional point cloud data and the image data;

[0010] Determine a target subspace according to the dense three-dimensional point cloud map;

[0011] Determine an optimal target subspace sequence and a guide path according to the target subspace; the optimal target subspace sequence is a sequence of target subspaces that the mobile robot will sequentially go to; the guide path is a path connecting each target subspace;

[0012] Using a fast expanding random tree algorithm, constructing a fast expanding random graph according to the dense three-dimensional point cloud map;

[0013] Based on the rapidly expanding random graph, determining a local optimal exploration path according to the guided path;

[0014] The mobile robot moves along the local optimal exploration path until the optimal target subspace sequence in the entire area is explored, and finally obtains an environmental map of the entire area based on all the dense three-dimensional point cloud data and all the image data.

[0015] Optionally, constructing a dense three-dimensional point cloud map containing semantic information according to the dense three-dimensional point cloud data and the image data specifically includes:

[0016] Extracting semantic information of the environment using a fully convolutional neural network based on the image data;

[0017] According to the semantic information of the environment, using the fully convolutional neural network, all points in the point cloud of the dense three-dimensional point cloud data are given category labels;

[0018] Construct a dense 3D point cloud map with semantic information based on point clouds with category labels.

[0019] Optionally, determining the target subspace according to the dense three-dimensional point cloud map specifically includes:

[0020] Downsampling the dense three-dimensional point cloud map to obtain a global sparse octree map;

[0021] The exploration area in the global sparse octree map is evenly divided into a plurality of cubic subspaces; the spatial attributes of the cubic subspaces include exploration states; the exploration states include unexplored, exploring, and explored;

[0022] The cube subspace in the exploration state is taken as the target subspace.

[0023] Optionally, after determining the local optimal exploration path based on the rapidly expanding random graph and according to the guided path, the autonomous exploration method for integrating environmental mutual information in an unknown environment further includes:

[0024] According to the local optimal exploration path, a speed instruction is calculated by a pure tracking algorithm to achieve tracking control of the mobile robot on the local optimal exploration path;

[0025] According to the offline trajectory library and the passability map, the mobile robot performs real-time collision detection in the local optimal exploration path to achieve dynamic obstacle avoidance; the offline trajectory library is a forward simulated cubic spline curve generated by MATLAB, and the passability map is obtained by superimposing multiple frames of dense three-dimensional point cloud data to analyze the drivable area of ​​the ground around the mobile robot.

[0026] Optionally, a fast expanding random tree algorithm is used to construct a fast expanding random graph according to the dense three-dimensional point cloud map, specifically including:

[0027] Using an octree to convert the dense three-dimensional point cloud map into a local dense three-dimensional occupancy map;

[0028] A rapidly expanding random tree algorithm is adopted in combination with random sampling and semantic mutual information to construct a rapidly expanding random graph according to the local dense three-dimensional occupancy map.

[0029] Optionally, based on the rapidly expanding random graph and according to the guided path, determining a local optimal exploration path specifically includes:

[0030] Determining a shortest path in the rapidly expanding random graph according to the guided path;

[0031] The exploration information gain function is used to calculate the exploration gain for all shortest paths, and the shortest path with the maximum exploration gain is taken as the local optimal exploration path.

[0032] Optionally, the exploration information gain function is:

[0033]

[0034] Among them, δ i is the i-th shortest path, δ exp Represents a length and δ i Same, direction is determined by δ i The starting point points to δ i The straight line path of the end point, w1 and w2 are weight values, is the measure δ i and path δ exp The evaluation function of trajectory similarity is represents the path length, represents the first node of the i-th shortest path, represents the jth node of the i-th shortest path, Represents the spatial gain of the i-th path. The spatial gain is calculated by the equal-length ray model. Gain() represents each node on the path. Volume gain at, m i Represents the path δ i The number of nodes in the middle, S() represents the current planned path δ i and a pseudo-straight line exploration path of the same length δ exp The cumulative Euclidean distance between .

[0035] In a second aspect, the present application provides an autonomous exploration system for integrating environmental mutual information in an unknown environment, including: a hardware platform, a base layer, and an autonomous detection layer; the autonomous detection layer includes a global guidance layer and a local exploration layer;

[0036] The hardware platform is used to obtain dense three-dimensional point cloud data and image data of the environment in real time; the dense three-dimensional point cloud data is obtained by scanning the environment with a laser radar carried by a mobile robot; the image data is obtained by shooting with a camera carried by a mobile robot;

[0037] The base layer is used to construct a dense three-dimensional point cloud map containing semantic mutual information according to the dense three-dimensional point cloud data and the image data;

[0038] The global guidance layer is used to: determine the target subspace according to the dense three-dimensional point cloud map;

[0039] Determine an optimal target subspace sequence and a guide path according to the target subspace; the optimal target subspace sequence is a sequence of target subspaces that the mobile robot will sequentially go to; the guide path is a path connecting each target subspace;

[0040] The local exploration layer is used to: adopt a fast expanding random tree algorithm to construct a fast expanding random graph according to the dense three-dimensional point cloud map;

[0041] Based on the rapidly expanding random graph, determining a local optimal exploration path according to the guided path;

[0042] The hardware platform is also used for the mobile robot to move along the local optimal exploration path until the optimal target subspace sequence in the entire area is explored, and finally an environmental map of the entire area is obtained based on all the dense three-dimensional point cloud data and all the image data.

[0043] Optionally, the autonomous exploration system for integrating environmental mutual information in an unknown environment further includes a navigation layer; the navigation layer includes:

[0044] A path tracking module, used to calculate a speed instruction according to the local optimal exploration path through a pure tracking algorithm, so as to achieve tracking control of the mobile robot on the local optimal exploration path;

[0045] The dynamic obstacle avoidance module is used to perform real-time collision detection in the local optimal exploration path of the mobile robot according to the offline trajectory library and the passability map to achieve dynamic obstacle avoidance; the offline trajectory library is a forward simulated cubic spline curve generated by MATLAB, and the passability map is obtained by analyzing the drivable area of ​​the ground around the mobile robot by superimposing multiple frames of dense three-dimensional point cloud data.

[0046] Optionally, in the aspect of adopting a rapidly expanding random tree algorithm to construct a rapidly expanding random graph according to the dense three-dimensional point cloud map, the local exploration layer includes:

[0047] A local map conversion module, used to convert the dense three-dimensional point cloud map into a local dense three-dimensional occupancy map using an octree;

[0048] A fast expanding random graph construction module is used to construct a fast expanding random graph according to the local dense three-dimensional occupancy map by using a fast expanding random tree algorithm combined with random sampling and semantic mutual information.

[0049] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0050] The present application provides an autonomous exploration method and system for integrating environmental mutual information in an unknown environment, so as to enable a mobile robot to integrate semantic mutual information to plan the robot's exploration goal and optimal path in an unknown complex environment, design a hierarchical framework for constructing a dual-resolution map, generate an inspiring rough trajectory at the global level to guide the approximate direction of the robot's movement, and propose a rapidly expanding random graph (RRG) generation strategy that combines semantic mutual information and random sampling, so as to plan an optimal path for quickly exploring the local space at the local level, thereby completely traversing the entire unknown environment to establish a map model of the entire environment, thereby improving the coverage and exploration efficiency of the scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 A flowchart of an autonomous exploration method for integrating environmental mutual information in an unknown environment in one embodiment of the present application;

[0053] Figure 2 A schematic diagram of the composition of an autonomous exploration system integrating environmental mutual information in an unknown environment provided by an embodiment of the present application;

[0054] Figure 3 A schematic diagram of a planning process of a global guidance layer of an autonomous exploration system integrating environmental mutual information in an unknown environment provided by an embodiment of the present application;

[0055] Figure 4 A schematic diagram of the overall process of an autonomous exploration system for integrating environmental mutual information in an unknown environment provided in one embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] Explanation of terms involved in this application:

[0058] (1) Mutual Information: includes the geometric mutual information and semantic mutual information of the environment;

[0059] (2) Simultaneous Localization and Mapping (SLAM): Mapping the environment and estimating the robot’s positioning data.

[0060] (3) Semantic SLAM: The map created for the environment contains semantic information, such as which part of the point cloud corresponds to the table and which part corresponds to the door.

[0061] (4) Traveling Salesman Problem (TSP): Given a series of cities and the distances between each pair of cities, find the shortest path that visits each city once and returns to the starting city.

[0062] The existing solutions mainly include robot environment exploration methods based on boundary detection strategy and robot environment exploration methods based on viewpoint sampling strategy.

[0063] Existing Solution 1: A robot environment exploration method based on boundary detection strategy. This method uses sensors to construct a two-dimensional occupancy grid map of the environment of interest. It identifies the boundaries between known and unknown spaces in the two-dimensional occupancy grid map, clusters the grids at the boundary to obtain the boundary center, and evaluates it. It selects the boundary center point with the closest Euclidean distance as the exploration target, which is used to drive the environmental exploration.

[0064] Existing Solution 2: A robot environment exploration method based on a viewpoint sampling strategy, which uses random sampling to randomly sample exploration viewpoints in the environment map, and then evaluates the exploration utility of these viewpoints to drive the exploration. Specifically, the environment exploration method based on the viewpoint sampling strategy generally uses the Rapidly-Exploring Random Trees (RRT) algorithm and its variants to sample the robot sensor poses in the environment map, and uses the exploration characteristics of the RRT itself to sample in the environment map to obtain viewpoints that can cover unknown areas. Since the RRT algorithm is used to sample viewpoints, the path from each random sampling point to the root node of the tree is also determined, thereby eliminating the path planning process.

[0065] However, most existing autonomous exploration methods only consider the geometric mutual information in the environment, such as the frontier boundary point and the next optimal viewpoint, and adopt a greedy strategy, which often lacks the guidance of semantic mutual information. When the algorithm only focuses on short-term geometric mutual information, such as moving towards the frontier boundary point with the greatest current benefit, the robot is prone to short-sightedness and ignores the long-term goal, and backtracking will occur. Fully utilizing the semantic information of the environment can not only improve the exploration efficiency but also improve the stability of the exploration method. The specific design of a decision strategy affects the selection of target points, exploration efficiency and exploration behavior.

[0066] In addition, for indoor or underground environments with multiple corridors, multiple branches, and narrow passages, the traditional RRT sampling viewpoint-based method has the problem of difficulty in passing through narrow passages and repeatedly exploring the same area. Since the sampling method requires continuous repetition of the tree construction process and it is difficult to jump out of the narrow space, it is easy for the exploration to fall into the local optimum, and the efficiency is low in large-scale or independent subspace scene exploration tasks. How to choose the evaluation indicators and the design of the utility function, and even how to achieve the corresponding evaluation indicators, all affect the improvement of exploration efficiency.

[0067] In addition, traditional methods based on frontier boundary points often need to maintain a large-scale map. The algorithm will slow down as the robot's exploration range expands, reducing the exploration efficiency. The RRT sampling viewpoint-based method has a slow growth of the RRT tree in the later stages of exploration of large-scale scenes, and some corners of the scene cannot be covered.

[0068] This application proposes an autonomous exploration method and system for integrating environmental mutual information in an unknown environment to solve the above problems.

[0069] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0070] In view of the problem that traditional algorithms only maintain an overall map and the computational efficiency decreases as the exploration scope expands, and the problem that the RRT tree grows slowly in the late exploration of large-scale scenes based on RRT random sampling, and some corners of the scene cannot be covered. A hierarchical framework for building dual-resolution maps is designed. The global guidance layer is based on the global sparse octree map for heuristic rough guidance. The global guidance path is used to connect each subspace. The local exploration layer plans an optimal path to quickly explore the local space based on the local dense map.

[0071] In an exemplary embodiment, Figure 1 As shown, an autonomous exploration method for integrating environmental mutual information in an unknown environment is provided, including:

[0072] Acquire dense three-dimensional point cloud data and image data of the environment in real time; the dense three-dimensional point cloud data is obtained by scanning the environment with a laser radar equipped on a mobile robot; and the image data is obtained by photographing with a camera equipped on a mobile robot.

[0073] A dense three-dimensional point cloud map containing semantic mutual information is constructed according to the dense three-dimensional point cloud data and the image data.

[0074] A target subspace is determined according to the dense three-dimensional point cloud map.

[0075] An optimal target subspace sequence and a guide path are determined according to the target subspace; the optimal target subspace sequence is a sequence of target subspaces to which the mobile robot will sequentially go; and the guide path is a path connecting each target subspace.

[0076] A fast expanding random tree algorithm is adopted to construct a fast expanding random graph according to the dense three-dimensional point cloud map.

[0077] Based on the rapidly expanding random graph, determining a local optimal exploration path according to the guided path;

[0078] The mobile robot moves along the local optimal exploration path until the optimal target subspace sequence in the entire area is explored, and finally obtains an environmental map of the entire area based on all the dense three-dimensional point cloud data and all the image data.

[0079] This embodiment constructs a hierarchical framework of dual-resolution maps. The global guidance layer uses a global sparse octree map for heuristic guidance, and the local exploration layer uses a local dense three-dimensional occupancy map to plan a local optimal exploration path that quickly explores the local subspace, so that the robot explores the environment in a sequence jointly determined by the global and local, reducing the amount of calculation while improving the coverage and exploration efficiency of the scene.

[0080] In another exemplary embodiment of the present application, constructing a dense three-dimensional point cloud map containing semantic information according to the dense three-dimensional point cloud data and the image data specifically includes:

[0081] Based on the image data, a fully convolutional neural network is used to extract semantic information of the environment.

[0082] According to the semantic information of the environment, the fully convolutional neural network is used to give category labels to all points in the point cloud of the dense three-dimensional point cloud data.

[0083] Construct a dense 3D point cloud map with semantic information based on point clouds with category labels.

[0084] Most existing autonomous exploration methods only consider the geometric mutual information in the environment (such as frontier boundary points), and the greedy strategy or the strategy of maximizing information gain leads to short-sightedness and backtracking. Inspired by the law of human perception - the law of closure, a subspace global guidance strategy integrating semantic mutual information is proposed to generate an inspiring rough trajectory at the global level to guide the approximate direction of the robot's movement.

[0085] In another exemplary embodiment of the present application, determining the target subspace according to the dense three-dimensional point cloud map specifically includes:

[0086] The dense three-dimensional point cloud map is downsampled to obtain a global sparse octree map.

[0087] The exploration area in the global sparse octree map is evenly divided into a plurality of cubic subspaces. The spatial attributes of the cubic subspaces include exploration states; the exploration states include unexplored, exploring, and explored.

[0088] The cube subspace in the exploration state is taken as the target subspace.

[0089] Inspired by the law of closure, a law of human perception, this embodiment proposes a subspace global guidance strategy that integrates semantic mutual information, by dividing the environment into subspaces for the exploration area, taking into account the exploration state and semantic information of the environment. The heuristic decision strategy is used to comprehensively consider the semantic mutual information and topological structure information of the environment, reducing short-sightedness and backtracking in the exploration process.

[0090] In another exemplary embodiment of the present application, Figure 1 As shown, after determining the local optimal exploration path based on the fast expanding random graph and the guide path, the autonomous exploration method of integrating environmental mutual information in an unknown environment further includes:

[0091] According to the local optimal exploration path, a speed instruction is calculated through a pure tracking algorithm to achieve tracking control of the mobile robot on the local optimal exploration path.

[0092] According to the offline trajectory library and the passability map, the mobile robot performs real-time collision detection in the local optimal exploration path to achieve dynamic obstacle avoidance; the offline trajectory library is a forward simulated cubic spline curve generated by MATLAB, and the passability map is obtained by superimposing multiple frames of dense three-dimensional point cloud data to analyze the drivable area of ​​the ground around the mobile robot.

[0093] For indoor or underground environments with multiple corridors, multiple branches and narrow passages, the traditional RRT random sampling method has the problem of difficulty in passing through narrow passages and repeatedly exploring the same area. A rapidly expanding random graph (RRG) generation strategy combining semantic mutual information and random sampling, as well as a local path planning method introducing trajectory similarity evaluation, is proposed.

[0094] In another exemplary embodiment of the present application, a fast expanding random tree algorithm is used to construct a fast expanding random graph according to the dense three-dimensional point cloud map, specifically including:

[0095] An octree is used to convert the dense three-dimensional point cloud map into a local dense three-dimensional occupancy map.

[0096] A rapidly expanding random tree algorithm is adopted in combination with random sampling and semantic mutual information to construct a rapidly expanding random graph according to the local dense three-dimensional occupancy map.

[0097] This embodiment introduces a method combining semantic mutual information and random sampling to generate a fast-expanding random graph, effectively solving the problem of robot exploration in complex environments such as multi-branch, multi-corridor, and narrow passages. This strategy provides more effective nodes when constructing a fast-expanding random graph, improves the robot's path planning effect in complex environments, and increases its ability to pass through narrow paths.

[0098] In another exemplary embodiment of the present application, based on the rapidly expanding random graph and according to the guided path, determining a local optimal exploration path specifically includes:

[0099] A shortest path is determined in the rapidly expanding random graph based on the guided path.

[0100] The exploration information gain function is used to calculate the exploration gain for all shortest paths, and the shortest path with the maximum exploration gain is taken as the local optimal exploration path.

[0101] In another exemplary embodiment of the present application, the exploration information gain function is:

[0102]

[0103] Among them, δ i is the i-th shortest path, δ exp Represents a length and δ i Same, direction is determined by δ i The starting point points to δ i The straight line path of the end point, w1 and w2 are weight values, To measure the current feasible path δi and path δ exp The evaluation function of trajectory similarity is represents the path length, represents the first node of the i-th shortest path, represents the jth node of the i-th shortest path, Represents the spatial gain of the i-th path. The spatial gain is calculated by the equal-length ray model. Gain() represents each node on the path. The volume gain at the node is the cumulative volume of the unmapped area that the robot can explore at this node, m i Represents the path δ i The number of nodes in the path, S() represents the path similarity function, that is, the current planned path δ i and a pseudo-straight line exploration path of the same length δ exp The cumulative Euclidean distance between .

[0104] This embodiment introduces a local path planner for trajectory similarity evaluation, which comprehensively considers the path length, exploration space gain and trajectory similarity of the path. Through reasonable trade-offs, it avoids the intermittent reciprocating motion of the robot in a specific area, and effectively solves the problem of suboptimal paths when the robot encounters a fork in the road in an indoor or underground environment with multiple corridors, multiple branches and narrow passages. A better path can be found in a multi-fork environment.

[0105] Based on the same inventive concept, the embodiment of the present application also provides an autonomous exploration system for fusion environment mutual information in an unknown environment for implementing the autonomous exploration method for fusion environment mutual information in an unknown environment involved above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the autonomous exploration system for fusion environment mutual information in one or more unknown environments provided below can be referred to the limitations of the autonomous exploration method for fusion environment mutual information in an unknown environment above, and will not be repeated here.

[0106] In an exemplary embodiment, an autonomous exploration system for integrating environmental mutual information in an unknown environment is provided, which consists of four layers, such as Figure 2 As shown, it includes: hardware platform, basic layer, navigation layer and autonomous detection layer.

[0107] The hardware platform includes a mobile robot, which is based on a tracked differential chassis and is equipped with a lidar, camera and industrial computer. The camera and lidar are used to obtain environmental perception data, while the industrial computer runs algorithms to process data, plan paths and control motion, and finally sends the data to the tracked differential chassis for execution.

[0108] The navigation layer provides terrain passability analysis, dynamic obstacle avoidance, and path tracking functions, and calculates control instructions based on the path points sent by the autonomous exploration layer. Passability analysis analyzes the ground around the robot for drivable areas by superimposing multiple frame point clouds, dynamic obstacle avoidance relies on the offline trajectory library for real-time collision detection, and path tracking is achieved through a pure tracking algorithm.

[0109] The autonomous exploration layer provides top-level decisions and paths for the robot, generates an inspiring rough trajectory in the global guidance layer to guide the approximate direction of the robot's movement, and then the local exploration layer plans a more precise optimal trajectory to quickly explore and map the robot's surrounding environment.

[0110] The autonomous detection layer includes a global guidance layer and a local exploration layer. The hardware platform is used to obtain dense three-dimensional point cloud data and image data of the environment in real time; the dense three-dimensional point cloud data is obtained by scanning the environment with a laser radar equipped on a mobile robot; the image data is obtained by shooting with a camera equipped on a mobile robot.

[0111] The base layer is used to construct a dense three-dimensional point cloud map containing semantic mutual information based on the dense three-dimensional point cloud data and the image data.

[0112] The global guidance layer is used to determine a target subspace according to the dense three-dimensional point cloud map.

[0113] An optimal target subspace sequence and a guide path are determined according to the target subspace; the optimal target subspace sequence is a sequence of target subspaces to which the mobile robot will sequentially go; and the guide path is a path connecting each target subspace.

[0114] The local exploration layer is used to: adopt a fast expanding random tree algorithm to construct a fast expanding random graph according to the dense three-dimensional point cloud map.

[0115] Based on the rapidly expanding random graph, determining a local optimal exploration path according to the guided path;

[0116] The hardware platform is also used for the mobile robot to move along the local optimal exploration path until the optimal target subspace sequence in the entire area is explored, and finally an environmental map of the entire area is obtained based on all the dense three-dimensional point cloud data and all the image data.

[0117] In another exemplary embodiment of the present application, the autonomous exploration system for integrating environmental mutual information in an unknown environment further includes a navigation layer; the navigation layer includes:

[0118] The path tracking module is used to calculate the speed instruction according to the local optimal exploration path through a pure tracking algorithm, so as to realize the tracking control of the mobile robot on the local optimal exploration path.

[0119] The dynamic obstacle avoidance module is used to perform real-time collision detection in the local optimal exploration path of the mobile robot according to the offline trajectory library and the passability map to achieve dynamic obstacle avoidance; the offline trajectory library is a forward simulated cubic spline curve generated by MATLAB, and the passability map is obtained by the passability analysis module in the navigation layer, specifically by superimposing multiple frames of dense three-dimensional point cloud data to analyze the drivable area of ​​the ground around the mobile robot.

[0120] In another exemplary embodiment of the present application, in the aspect of using a fast expanding random tree algorithm to construct a fast expanding random graph according to the dense three-dimensional point cloud map, the local exploration layer includes:

[0121] The local map conversion module is used to convert the dense three-dimensional point cloud map into a local dense three-dimensional occupancy map by using an octree.

[0122] A fast expanding random graph construction module is used to construct a fast expanding random graph according to the local dense three-dimensional occupancy map by using a fast expanding random tree algorithm combined with random sampling and semantic mutual information.

[0123] like Figure 4 As shown, the overall workflow of the autonomous exploration system for integrating environmental mutual information in an unknown environment in the above embodiment is as follows:

[0124] Step 1: The mobile robot performs semantic SLAM in real time through lidar and camera during movement, and inputs the map data into the passability analysis module of the navigation layer to analyze the passability of the terrain. The lidar scans the environment in real time to obtain dense 3D point cloud data of the environment. At the same time, the camera's image data is input into the fully convolutional neural network to extract the semantic information of the environment. The network gives category labels to all points in the 3D point cloud, so that when building the map, the labeled point cloud is used. Semantic SLAM specifically uses the SuMa++ algorithm to update the dense 3D point cloud map of the environment containing semantic information and the position data of the mobile robot in real time.

[0125] Step 2: The specific process of the global guidance layer is as follows Figure 3As shown in the figure, a subspace global guidance strategy that integrates semantic mutual information is used in the global guidance layer to generate an inspiring global rough path at the global level to guide the approximate orientation of the robot's movement. First, downsampling is performed on the dense three-dimensional point cloud map containing semantic information that is updated in real time to obtain a global sparse octree map (i.e., an incremental sparse octree map). Then the area to be explored is evenly divided into an even number of cubic subspaces, each of which stores the explored environmental surfaces. The cubic subspace attributes consist of exploration status and semantic mutual information. There are three types of exploration status: "unexplored", "exploring" or "explored". If the cubic subspace does not contain any covered or uncovered surfaces, the status is "unexplored"; if a cubic subspace only contains covered surfaces, the status is "explored"; if a cubic subspace contains any uncovered surfaces, the status is "explored". Semantic mutual information only considers doors and forks in the road. The above two are defined as semantic objects of interest, which are incrementally identified and located by the efficient semantic SLAM algorithm. The cube subspace semantic mutual information includes "objects with semantic interest" and "objects without semantic interest". In the global planning, only the cube subspace in the "exploring" state is considered. If the cube subspace state is "exploring", the mobile robot will go to these target subspaces in turn during the subsequent exploration process.

[0126] During the exploration process, several target subspaces will appear, and the optimal target subspace sequence is obtained by the heuristic decision strategy. The heuristic decision strategy comprehensively considers the semantic mutual information of the environment and the geometric mutual information of the environment. Semantic mutual information can provide high-level exploration guidance information for the mobile robot. During the exploration process, similar to human perception, the mobile robot always tends to explore rooms rather than corridors. In general, the strategies in the two cases are as follows:

[0127] Case 1: If an object of semantic interest is detected, it means that the corresponding target subspace has a room or a fork in the road. The mobile robot will traverse these target subspaces according to the distance from the current position, and the exploration will become a Travel Salesman Problem (TSP). When a new room appears in the process, the robot will replan the guidance path every time it completes the exploration of a room.

[0128] Case 2: If there is no semantically interesting object, it means that the corresponding target subspace has no room or fork in the road. In this case, the optimal target subspace sequence of all target subspaces is also obtained by solving TSP.

[0129] The actual path cost between the target subspaces is obtained by the hybrid A* algorithm, and then a distance matrix is ​​constructed, which is input into the TSP solver in this strategy, and the TSP solver is solved based on the ant colony algorithm. Therefore, the heuristic decision strategy takes into account the mutual information of environmental geometry and the mutual information of environmental semantics.

[0130] Step 3: In the local exploration layer, combine semantic mutual information and random sampling to construct a fast-expanding random graph. The following are the specific steps for constructing a fast-expanding random graph:

[0131] Firstly, a local space of fixed size is delineated with the mobile robot as the center, and the corresponding dense 3D point cloud map is obtained from semantic SLAM, which is then converted into a local dense 3D occupancy map using an octree.

[0132] Take the current position of the mobile robot as the root node X of RRT root , define the area corresponding to the local space as a growth space D, set the maximum number of nodes and the maximum number of edges, initialize the node set V, the edge set E, and put the current position of the mobile robot into the set V.

[0133] Take a point X in the traversable domain of the growth space D. rand , click X rand There are two sources for generating : one is random sampling in the passable domain; since the sampling probability of the sampled viewpoints in narrow channels is very small, it is not easy to grow to narrow channels. Therefore, additional viewpoints are generated by combining the predicted semantic mutual information, and several viewpoints are sampled within a certain radius r of the semantic interest object. These viewpoints and the viewpoints obtained by random sampling constitute a feasible node set.

[0134] Find the distance from the existing RRT rand The nearest point X nearest , for X nearest To X rand Check the lines between them for collision. If there is no collision, X rand The collision-free lines are added to sets V and E as edges.

[0135] Delineated by X rand A circle with radius δ as the center is formed by connecting all nodes in set V and X rand Connect, only connect the edges without collision, and add them to RRT.

[0136] The above process is iterated repeatedly until the number of nodes or edges in the local space reaches the set maximum value.

[0137] Step 4: Use the local path planner on the obtained rapidly expanding random graph to obtain the local optimal exploration path. The established rapidly expanding random graph has multiple terminal nodes. Based on the idea of ​​graph search, the hybrid A* algorithm is used to calculate all the shortest paths from the root node to the terminal. The exploration information gain function is used to calculate the exploration gain for all shortest paths, and the shortest path with the maximum exploration gain is taken as the local optimal exploration path. The above is an iterative process. Each time the end point of the local optimal exploration path is reached, it will be iterated once until the exploration ends. The exploration information gain function is as follows:

[0138]

[0139] δ i represents the i-th shortest path, w1 and w2 are weight values, represents the path length, Represents the spatial gain of the i-th path, which is calculated by the equal-length ray model. exp represents a straight line path (whose length is equal to δ i Same, direction is determined by δ i The starting point points to δ i end point), Gain() indicates that m i represents, S() represents, Represents the measurement of the current feasible path δ i and path δ exp The evaluation function of the trajectory similarity of the mobile robot is used to avoid paths with large deviations in the exploration direction. This factor can prevent the mobile robot from making intermittent mutations in its exploration direction, especially when the mobile robot enters an intersection area with multiple branches. If this evaluation function is not added, when the mobile robot enters an intersection area with multiple branches, the mobile robot may make small reciprocating movements between the forks in pursuit of higher spatial gains.

[0140] Step 5: According to the planned local optimal exploration path, the speed command is calculated through a pure tracking algorithm to achieve optimal tracking control of the mobile robot on the local optimal exploration path.

[0141] Step 6: During the robot's movement, real-time collision detection is performed to achieve dynamic obstacle avoidance by relying on the passability map output by the offline trajectory library and terrain analysis module.

[0142] Step 7: Repeat steps 1 to 6 above. The robot moves along the path to the continuously updated new target position and updates the environmental map in real time and moves along the path to the continuously updated new target position, finally obtaining the entire environmental map of the area to be explored.

[0143] The present application has the following advantages over the prior art.

[0144] (1) Comprehensively consider the geometric mutual information, topological structure mutual information and semantic mutual information of the environment:

[0145] Disadvantages of existing technologies: Most existing autonomous exploration methods only focus on the geometric mutual information in the environment, resulting in short-sightedness, backtracking and lack of long-term goals.

[0146] Advantages of this application: It introduces a subspace global guidance strategy that integrates semantic mutual information, reduces the risk of short-sightedness and backtracking by considering the geometric information and semantic mutual information of the environment, and improves the long-term goal perception of exploration.

[0147] (2) Using semantic mutual information to improve path planning capabilities in complex environments:

[0148] Disadvantages of existing technologies: Traditional methods are prone to fall into suboptimal paths or have difficulty passing through narrow passages in environments with multiple corridors, multiple branches, and narrow passages.

[0149] Advantages of this application: It proposes a fast-expanding random graph generation strategy that combines semantic mutual information and random sampling, effectively solves the path planning problem in a multi-branch environment, and improves the robot's ability to pass through narrow channels.

[0150] (3) Optimizing the integration strategy of global guidance and local path planning:

[0151] Disadvantages of existing technologies: Existing methods only maintain an overall map, and the computational efficiency decreases as the exploration scope expands. In addition, the RRT tree grows slowly in the later stages of exploration of large-scale scenes based on random sampling, and some corners of the scene cannot be covered, resulting in low exploration efficiency.

[0152] Advantages of this application: It adopts a hierarchical framework of dual-resolution maps, the global guidance layer uses a global sparse octree map for heuristic guidance, and the local exploration layer uses a local dense three-dimensional occupancy map to plan the local optimal exploration path, thereby improving the global coverage and local efficiency of the exploration.

[0153] (4) Introducing the exploration information gain function of trajectory similarity evaluation to improve the intelligence of path selection:

[0154] Disadvantages of existing technologies: Traditional algorithms may produce intermittent reciprocating motion in specific areas, resulting in low efficiency.

[0155] Advantages of this application: The exploration information gain function is introduced to comprehensively consider the path length, spatial gain and trajectory similarity of the path, which reduces the probability of the robot's reciprocating motion in a complex environment and improves the intelligence of path selection.

[0156] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An autonomous exploration method for integrating environmental mutual information in an unknown environment, characterized in that: The autonomous exploration method for integrating environmental mutual information in an unknown environment includes: Acquire dense three-dimensional point cloud data and image data of the environment in real time; the dense three-dimensional point cloud data is obtained by scanning the environment with a laser radar carried by a mobile robot; the image data is obtained by photographing with a camera carried by a mobile robot; Constructing a dense three-dimensional point cloud map containing semantic mutual information according to the dense three-dimensional point cloud data and the image data; Determine a target subspace according to the dense three-dimensional point cloud map; Determine an optimal target subspace sequence and a guide path according to the target subspace; the optimal target subspace sequence is a sequence of target subspaces that the mobile robot will sequentially go to; the guide path is a path connecting each target subspace; Using a fast expanding random tree algorithm, constructing a fast expanding random graph according to the dense three-dimensional point cloud map; Based on the rapidly expanding random graph, determining a local optimal exploration path according to the guided path; The mobile robot moves along the local optimal exploration path until the optimal target subspace sequence in the entire area is explored, and finally obtains an environmental map of the entire area based on all the dense three-dimensional point cloud data and all the image data.

2. The autonomous exploration method for integrating environmental mutual information in an unknown environment according to claim 1, characterized in that: Constructing a dense three-dimensional point cloud map containing semantic information according to the dense three-dimensional point cloud data and the image data, specifically comprising: Extracting semantic information of the environment using a fully convolutional neural network based on the image data; According to the semantic information of the environment, using the fully convolutional neural network, all points in the point cloud of the dense three-dimensional point cloud data are given category labels; Construct a dense 3D point cloud map with semantic information based on point clouds with category labels.

3. The autonomous exploration method for integrating environmental mutual information in an unknown environment according to claim 1, characterized in that: Determining a target subspace according to the dense three-dimensional point cloud map specifically includes: Downsampling the dense three-dimensional point cloud map to obtain a global sparse octree map; The exploration area in the global sparse octree map is evenly divided into a plurality of cubic subspaces; the spatial attributes of the cubic subspaces include exploration states; the exploration states include unexplored, exploring, and explored; The cube subspace in the exploration state is taken as the target subspace.

4. The autonomous exploration method for integrating environmental mutual information in an unknown environment according to claim 1, characterized in that: After determining the local optimal exploration path based on the fast expanding random graph and according to the guide path, the autonomous exploration method for integrating environmental mutual information in an unknown environment further includes: According to the local optimal exploration path, a speed instruction is calculated by a pure tracking algorithm to achieve tracking control of the mobile robot on the local optimal exploration path; According to the offline trajectory library and the passability map, the mobile robot performs real-time collision detection in the local optimal exploration path to achieve dynamic obstacle avoidance; the offline trajectory library is a forward simulated cubic spline curve generated by MATLAB, and the passability map is obtained by superimposing multiple frames of dense three-dimensional point cloud data to analyze the drivable area of ​​the ground around the mobile robot.

5. The autonomous exploration method for integrating environmental mutual information in an unknown environment according to claim 1, characterized in that: A fast expanding random tree algorithm is used to construct a fast expanding random graph according to the dense three-dimensional point cloud map, specifically including: Using an octree to convert the dense three-dimensional point cloud map into a local dense three-dimensional occupancy map; A rapidly expanding random tree algorithm is adopted in combination with random sampling and semantic mutual information to construct a rapidly expanding random graph according to the local dense three-dimensional occupancy map.

6. The autonomous exploration method for integrating environmental mutual information in an unknown environment according to claim 1, characterized in that: Based on the rapidly expanding random graph and according to the guided path, determining a local optimal exploration path specifically includes: Determining a shortest path in the rapidly expanding random graph according to the guided path; The exploration information gain function is used to calculate the exploration gain for all shortest paths, and the shortest path with the maximum exploration gain is taken as the local optimal exploration path.

7. The autonomous exploration method for integrating environmental mutual information in an unknown environment according to claim 6, characterized in that: The exploration information gain function is: Among them, δ i represents the i-th shortest path, δ exp Represents a length and δ i Same, direction is determined by δ i The starting point points to δ i The straight line path of the end point, w1 and w2 are weight values, Represents the metric δ i and path δ exp The evaluation function of trajectory similarity is represents the path length, represents the first node of the i-th shortest path, represents the jth node of the i-th shortest path, Represents the spatial gain of the i-th path. The spatial gain is calculated by the equal-length ray model. Gain() represents each node on the path. Volume gain at, m i Represents the path δ i The number of nodes in the middle, S() represents the current planned path δ i and a pseudo-straight line exploration path of the same length δ exp The cumulative Euclidean distance between .

8. An autonomous exploration system integrating environmental mutual information in an unknown environment, characterized in that: The autonomous exploration system for integrating environmental mutual information in an unknown environment comprises: a hardware platform, a basic layer and an autonomous detection layer; the autonomous detection layer comprises a global guidance layer and a local exploration layer; The hardware platform is used to obtain dense three-dimensional point cloud data and image data of the environment in real time; the dense three-dimensional point cloud data is obtained by scanning the environment with a laser radar carried by a mobile robot; the image data is obtained by shooting with a camera carried by a mobile robot; The base layer is used to construct a dense three-dimensional point cloud map containing semantic mutual information according to the dense three-dimensional point cloud data and the image data; The global guidance layer is used to: determine the target subspace according to the dense three-dimensional point cloud map; Determine an optimal target subspace sequence and a guide path according to the target subspace; the optimal target subspace sequence is a sequence of target subspaces that the mobile robot will sequentially go to; the guide path is a path connecting each target subspace; The local exploration layer is used to: adopt a fast expanding random tree algorithm to construct a fast expanding random graph according to the dense three-dimensional point cloud map; Based on the rapidly expanding random graph, determining a local optimal exploration path according to the guided path; The hardware platform is also used for the mobile robot to move along the local optimal exploration path until the optimal target subspace sequence in the entire area is explored, and finally an environmental map of the entire area is obtained based on all the dense three-dimensional point cloud data and all the image data.

9. The autonomous exploration system for integrating environmental mutual information in an unknown environment according to claim 8, characterized in that: The autonomous exploration system integrating environmental mutual information in an unknown environment further includes a navigation layer; the navigation layer includes: A path tracking module, used to calculate a speed instruction according to the local optimal exploration path through a pure tracking algorithm, so as to achieve tracking control of the mobile robot on the local optimal exploration path; The dynamic obstacle avoidance module is used to perform real-time collision detection in the local optimal exploration path of the mobile robot according to the offline trajectory library and the passability map to achieve dynamic obstacle avoidance; the offline trajectory library is a forward simulated cubic spline curve generated by MATLAB, and the passability map is obtained by analyzing the drivable area of ​​the ground around the mobile robot by superimposing multiple frames of dense three-dimensional point cloud data.

10. The autonomous exploration system for integrating environmental mutual information in an unknown environment according to claim 8, characterized in that: In the aspect of using a fast expanding random tree algorithm to construct a fast expanding random graph according to the dense three-dimensional point cloud map, the local exploration layer includes: A local map conversion module, used to convert the dense three-dimensional point cloud map into a local dense three-dimensional occupancy map using an octree; A fast expanding random graph construction module is used to construct a fast expanding random graph according to the local dense three-dimensional occupancy map by using a fast expanding random tree algorithm combined with random sampling and semantic mutual information.

Images