A method for constructing a topographic map for efficient navigation of an autonomous mobile robot

Abstract

The present application relates to a kind of topography map construction method for facilitating autonomous mobile robot efficient navigation, including topography segmentation network: using depth separable convolution and pyramid pooling module to build high-precision lightweight semantic segmentation network;Detailed division of passable topography: according to the physical geometric property of different topography, more detailed division and annotation are carried out to passable topography, and topography dataset is constructed for the training of model;The construction of topography map: according to the geometric conversion relationship between the original image obtained by sensor and grid map, the mapping relationship between topography image and occupancy grid map is constructed, and topography grid map is created.The present application uses semantic segmentation network to divide the topography in the working environment of mobile robot more carefully, and through pixel scanning, coordinate conversion and grid mapping, topography grid map containing rich information is created, to improve the navigation efficiency of mobile robot.

Description

Technical Field

[0001] The present invention relates to a method for constructing a terrain map that facilitates efficient navigation of an autonomous mobile robot, and belongs to the technical field of robot perception and mapping. Background Art

[0002] Identifying traversable areas is crucial for mobile robot navigation and path planning. Currently, most terrain recognition algorithms only distinguish between traversable and impassable areas. However, an increasing number of mobile robots operate in unstructured or semi-structured environments, which contain a variety of traversable terrains, with significant variations among them, such as concrete, grass, mud, and gravel. As robots traverse different traversable terrains, their safety, travel time, and energy consumption vary.

[0003] When navigating different terrains, humans choose routes with the lowest travel costs based on their goals. Mobile robots can mimic human travel strategies and plan routes based on different mission requirements. For example, they can traverse high-energy-consuming terrain in straight lines to shorten travel time; bypass high-risk terrain such as stagnant water to ensure the robot's own safety; and sacrifice travel time in favor of easily navigable terrain to reduce energy consumption. For unstructured work terrain, constructing a terrain map that contains rich terrain information and travel costs can help mobile robots maximize navigation efficiency while ensuring their own safety. Summary of the Invention

[0004] The purpose of this invention is to address the problem that traditional terrain recognition algorithms have difficulty providing specific traversable terrain information, and traditional navigation algorithms ignore terrain costs (traditional navigation algorithms do not incorporate the different traversal costs of different terrains into path planning calculations), resulting in inaccurate and unreasonable path planning, which in turn causes high-cost or high-risk passage for mobile robots. This invention proposes a method for constructing a terrain map that facilitates efficient navigation for autonomous mobile robots. Specifically, a semantic segmentation network is used to perform a more detailed division of the terrain in the mobile robot's working environment. Through pixel scanning and coordinate conversion, a mapping relationship is established between the occupancy grid map and the terrain semantic information, thereby constructing a terrain grid map containing detailed terrain information.

[0005] To achieve the above object, the technical solution of the present invention is:

[0006] A method for constructing a terrain map that facilitates efficient navigation of an autonomous mobile robot comprises the following steps:

[0007] Step 1: Use depthwise separable convolution to build the encoding path of the terrain segmentation network, use the pyramid pooling module of PspNet to form the decoding path, and build a high-precision and lightweight terrain segmentation network.

[0008] Step 2: Use the mobile robot's onboard camera to collect a sufficient number of landform images. After manual screening and data annotation, a complex landform dataset is constructed. The landform information in the dataset is divided into various landform types, such as cement ground, grassland, mud ground, woodland, gravel ground, water, ditch, etc.

[0009] Step 3: Input the complex landform dataset into the constructed landform segmentation network and adjust the parameter w p Perform training to generate a landform segmentation model.

[0010] Step 4: Use the Gmapping algorithm to fuse the mobile robot's lidar, odometer and other sensor data to construct an occupancy grid map of the mobile robot's working environment.

[0011] Step 5: Use the depth camera carried by the mobile robot to collect the original image A and depth image D of the current terrain environment, input the obtained original image A into the terrain segmentation model, and generate the terrain segmentation image B of the local environment.

[0012] Step 6: Scan pixels and perform coordinate conversion on each frame of terrain segmentation image B. The specific process is:

[0013] S1: Convert the landform segmentation image B into a landform grayscale image C.

[0014] S2: Create a matching list G of landform types and grayscale values, G = {(cement ground: g1), (grassland: g2), (mud ground: g3), (woodland: g4), (gravel ground: g5), (water: g6), (ditch: g7)}, where g k Represents the grayscale value, k∈(1,7).

[0015] S3: Pixel scanning. Based on the ratio of the resolution of the original image A to the resolution of the grid map, the terrain grayscale image C is divided into several terrain blocks of size p×p. The grayscale distribution of all pixels in the (i, j)th terrain block is calculated:

[0016] P ij ={(g1:n1),(g2:n2),(g3:n3),(g4:n4),(g5:n5),(g6:n6),(g7:n7)} (1)

[0017] Among them, n k Indicates the number of pixels corresponding to each grayscale value, k∈(1,7). Calculate P ij The maximum number of pixels in n max :

[0018] n max =max(n k),k∈(1,7) (2)

[0019] According to P ij Find n max The corresponding gray value g max :

[0020]

[0021] Among them, (g max ,n max )∈P ij Then find g according to G max Corresponding landform type terrain ij :

[0022]

[0023] Among them (terrain ij ,g max )∈G. The terrain obtained ij As the overall landform of the current (i, j) landform block, the coordinates (u, v) of the landform pixel point at the center of the current (i, j) landform block are used as the overall pixel coordinates of the landform block.

[0024] S4: Convert the coordinates (u, v) of the terrain pixel point from the two-dimensional pixel coordinate system to the three-dimensional camera coordinate system:

[0025]

[0026] Among them, (u,v,1) T is the homogeneous coordinate of the topographic pixel point (u, v), (X c ,Y c ,Z c ) T is the three-dimensional coordinate of the topographic pixel point coordinate (u, v) in the camera coordinate system, and l is the distance information corresponding to the topographic pixel point coordinate (u, v) in the depth image D. x and f y are the focal lengths of the depth camera in the x and y directions, c x and c y Indicates the offset of the depth camera optical axis in the pixel coordinate system.

[0027] S5: The coordinates of the terrain pixel points (X c ,Y c ,Z c ) T Convert from camera coordinate system to robot coordinate system:

[0028]

[0029] Among them, (X r ,Y r ,Z r ) T is the three-dimensional coordinate of the terrain pixel point in the robot coordinate system. The rotation matrix R and translation matrix T represent the rotation and translation relationship between the camera coordinate system and the robot coordinate system, respectively. They are artificially set according to the geometric position relationship between the depth camera and the robot.

[0030] S6: According to the robot's adaptive Monte Carlo positioning, the coordinates of the terrain pixel points (X r ,Y r ,Z r ) T The 2D world coordinate (X w ,Y w );

[0031] Step 7: Based on the geomorphic semantic information contained in the geomorphic pixel points, the 2D world coordinates (X w ,Y w ) for landform mapping;

[0032] Step 8: Control the mobile robot to move in the working environment and repeat steps 5 to 7 (step 6 S2 does not need to be performed again) until the mobile robot completes the topography mapping of all grids in the map and finally establishes a topography grid map containing topography semantic information.

[0033] In step 2, we used a mobile robot's first-person perspective to collect images of real-world terrain in a real-world work environment. After screening, we manually annotated the dataset using Labelme software. During the data annotation process, we meticulously divided the terrain into seven types: concrete, grass, mud, woodland, gravel, water, and ditch.

[0034] The step seven adds landform semantic information to the occupancy grid map, establishes a mapping relationship between the landform semantic image and the occupancy grid map, and realizes the visualization of the landform grid map.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention uses depthwise separable convolution to construct the encoding path of the terrain segmentation network, which can reduce the number of network parameters and improve the network's terrain segmentation speed, making it more suitable for mobile robots with limited on-board computing resources. The pyramid pooling module is used as the core unit of the decoding path to enhance the network's ability to extract local context information and global information, thereby ensuring the network's segmentation accuracy. A neural network is used to perform detailed terrain segmentation on the mobile robot's working environment, identify different feasible areas, and mark their travel costs, providing rich terrain semantic information for the mobile robot's mapping. The terrain semantic information is coordinate-mapped with the occupancy grid map to construct a terrain grid map, providing more detailed environmental information for the mobile robot's navigation, thereby improving the accuracy and efficiency of the robot's navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 A robot structure diagram of the terrain map construction method for facilitating efficient navigation of an autonomous mobile robot according to the present invention;

[0039] Figure 2 The present invention provides a terrain map construction process based on an occupancy grid of a terrain map construction method for facilitating efficient navigation of an autonomous mobile robot. DETAILED DESCRIPTION

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0041] This implementation uses the Scout Mini robot, Jetson AGX Xavier small workstation, RPLiDARA3 laser radar and RealSense D435 depth camera to build an experimental platform. Figure 1 A method for constructing a terrain map that facilitates efficient navigation of an autonomous mobile robot includes the following specific implementation steps:

[0042] Step 1: Use depthwise separable convolution to build the encoding path of the terrain segmentation network, use the pyramid pooling module of PspNet to form the decoding path, and build a high-precision and lightweight terrain segmentation network.

[0043] Step 2: Use the mobile robot's onboard camera to collect a sufficient number of landform images. After manual screening and data annotation, a complex landform dataset is constructed. The landform information in the dataset is divided into various landform types, such as cement ground, grassland, mud ground, woodland, gravel ground, water, ditch, etc.

[0044] Step 3: Input the complex landform dataset into the constructed landform segmentation network and adjust the parameter w p Perform training to generate a landform segmentation model.

[0045] Step 4: Use the Gmapping algorithm to fuse the mobile robot's lidar, odometer and other sensor data to construct an occupancy grid map of the mobile robot's working environment.

[0046] Step 5: Use the depth camera carried by the mobile robot to collect the original image A and depth image D of the current terrain environment. Resize the original image A to 288×288 and input it into the terrain segmentation model to generate a terrain segmentation image B of the local environment with a size of 320×240.

[0047] Step 6: Scan pixels and perform coordinate conversion on each frame of terrain segmentation image B. The specific process is:

[0048] S1: Convert the landform segmentation image B into a landform grayscale image C.

[0049] S2: Create a matching list G of landform types and grayscale values, G = {(cement ground: 70), (grassland: 150), (mud ground: 200), (woodland: 120), (gravel ground: 188), (water: 30), (ditch: 190)}.

[0050] S3: Pixel scanning. Based on the ratio of the resolution of the original image A to the resolution of the grid map, the terrain grayscale image C is divided into 16×12 terrain blocks of size 20×20. The grayscale distribution of all pixels in the (i, j)th terrain block is calculated:

[0051] P ij ={(70:n1),(150:n2),(200:n3),(120:n4),(188:n5),(30:n6),(190:n7)}(1)

[0052] Among them, i∈(1,16), j∈(1,12), n k Represents the number of pixels corresponding to each grayscale value, k∈(1,7).

[0053] Calculate P ij The maximum number of pixels in n max :

[0054] n max =max(n k ),k∈(1,7) (2)

[0055] According to P ij Find n max The corresponding gray value g max :

[0056]

[0057] Among them, (g max ,n max )∈P ij Then find g according to G max Corresponding landform type terrain ij :

[0058]

[0059] Among them (terrain ij ,g max )∈G. The terrain obtained ij As the overall landform of the current (i, j) landform block.

[0060] The coordinates (u, v) of the geomorphic pixel point at the center of the current (i, j) geomorphic block are used as the overall pixel coordinates of the geomorphic block.

[0061] S4: Convert the coordinates (u, v) of the terrain pixel point from the two-dimensional pixel coordinate system to the three-dimensional camera coordinate system:

[0062]

[0063] Among them, (u,v,1) T is the homogeneous coordinate of the topographic pixel point (u, v), (X c ,Y c ,Z c ) T is the three-dimensional coordinate of the topographic pixel point coordinate (u, v) in the camera coordinate system, and l is the distance information corresponding to the topographic pixel point coordinate (u, v) in the depth image D. x and f y are the focal lengths of the depth camera in the x and y directions, c x and c y Represents the offset of the depth camera optical axis in the pixel coordinate system. They constitute the camera's intrinsic parameter matrix I, which can be obtained through camera calibration:

[0064] S5: The coordinates of the terrain pixel points (X c ,Y c ,Z c ) T Convert from camera coordinate system to robot coordinate system:

[0065]

[0066] Among them, (X r ,Y r ,Z r ) T is the three-dimensional coordinate of the terrain pixel point in the robot coordinate system. The rotation matrix R and translation matrix T represent the rotation and translation relationship between the camera coordinate system and the robot coordinate system, respectively, which are artificially set according to the geometric position relationship between the depth camera and the robot.

[0067] S6: According to the robot's adaptive Monte Carlo positioning, the coordinates of the terrain pixel points (X r ,Y r ,Z r ) T The 2D world coordinate (X w ,Y w ).

[0068] Step 7: Based on the geomorphic semantic information contained in the geomorphic pixel points, the 2D world coordinates (X w ,Y w ) for landform mapping.

[0069] Step 8: Control the mobile robot to move in the working environment and repeat steps 5 to 7 (step 6 S2 does not need to be performed again) until the mobile robot completes the topography mapping of all grids in the map and finally establishes a topography grid map containing topography semantic information.

[0070] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. It is apparent to those skilled in the art that various changes, modifications, substitutions, and variations to these embodiments may be made without departing from the principles and spirit of the present invention, and these changes and modifications still fall within the scope of protection of the present invention.

Claims

1. A method for constructing a terrain map to facilitate efficient navigation of an autonomous mobile robot, characterized by: The following steps are involved: Step 1: Use depthwise separable convolution to build the encoding path of the terrain segmentation network, use the pyramid pooling module of PspNet to form the decoding path, and build a high-precision and lightweight terrain segmentation network; Step 2: Use the mobile robot's onboard camera to collect landform images, and construct a complex landform dataset after screening and data annotation; the landform information in the dataset in step 2 includes cement ground, grass, mud, woodland, gravel ground, water, and ditch; Step 3: Input the complex landform dataset into the constructed landform segmentation network and adjust the parameter w p Conduct training to generate a landform segmentation model; Step 4: Use the Gmapping algorithm to fuse the mobile robot's lidar and odometer sensor data to construct an occupancy grid map of the mobile robot's working environment; Step 5: Use the depth camera carried by the mobile robot to collect the original image A and depth image D of the current terrain environment, input the obtained original image A into the terrain segmentation model, and generate the terrain segmentation image B of the local environment; Step 6: Scan pixels and perform coordinate conversion on each frame of the terrain segmentation image B; The step six specifically includes the following steps: S1: Convert the landform segmentation image B into a landform grayscale image C; S2: Create a matching list G of landform types and grayscale values, G = {(cement ground: g1), (grassland: g2), (mud ground: g3), (woodland: g4), (gravel ground: g5), (water: g6), (ditch: g7)}, where g k Represents the grayscale value, k∈(1,7); S3: Pixel scanning. According to the ratio of the resolution of the original image A to the resolution of the grid map, the terrain grayscale image C is divided into several terrain blocks of size p×p. The grayscale distribution of all pixels in the (i, j)th terrain block is counted: P ij ={(g1:n1),(g2:n2),(g3:n3),(g4:n4),(g5:n5),(g6:n6),(g7:n7)} (1) Where n k Indicates the number of pixels corresponding to each gray value, k∈(1,7), calculate P ij The maximum number of pixels in n max : n max =max(n k ),k∈(1,7) (2) According to P ij Find n max The corresponding gray value g max : Among them, (g max ,n max )∈P ij , find g according to G max Corresponding landform type terrain ij : Among them (terrain ij ,g max )∈G, the terrain obtained ij As the overall landform of the current (i, j) landform block, the coordinates (u, v) of the landform pixel point at the center of the current (i, j) landform block are used as the overall pixel coordinates of the landform block; S4: Convert the coordinates (u, v) of the terrain pixel point from the two-dimensional pixel coordinate system to the three-dimensional camera coordinate system: Among them, (u,v,1) T is the homogeneous coordinate of the topographic pixel point (u, v), (X c ,Y c ,Z c ) T is the three-dimensional coordinate of the topographic pixel point coordinate (u, v) in the camera coordinate system, l is the distance information corresponding to the topographic pixel point coordinate (u, v) in the depth image D, f x and f y are the focal lengths of the depth camera in the x and y directions, c x and c y Indicates the offset of the depth camera optical axis in the pixel coordinate system; S5: The coordinates of the terrain pixel points (X c ,Y c ,Z c ) T Convert from camera coordinate system to robot coordinate system: Among them, (X r ,Y r ,Z r ) T is the three-dimensional coordinate of the terrain pixel point in the robot coordinate system, the rotation matrix R and the translation matrix T represent the rotation and translation relationship between the camera coordinate system and the robot coordinate system respectively; S6: According to the robot's adaptive Monte Carlo positioning, the coordinates of the terrain pixel points (X r ,Y r ,Z r ) T The 2D world coordinate (X w ,Y w ); Step 7: Based on the geomorphic semantic information contained in the geomorphic pixel points, the 2D world coordinates (X w ,Y w ) for landform mapping; Step 8: Control the mobile robot to move in the working environment and repeat steps 5 to 7 until the mobile robot completes the topography mapping of all grids in the map and establishes a topography grid map containing topography semantic information.

2. The method for constructing a terrain map to facilitate efficient navigation of an autonomous mobile robot according to claim 1, characterized in that: In the step 2, the first-person perspective of the mobile robot is used to collect the topographical images in the real working environment.

3. The method for constructing a terrain map to facilitate efficient navigation of an autonomous mobile robot according to claim 1, characterized in that: The step seven establishes a mapping relationship between the landform semantic image and the occupancy grid map by adding landform semantic information to the occupancy grid map.

Images