A multi-resolution multi-level map information structure, a dynamic updating system and a construction method for autonomous exploration

Abstract

The application belongs to the technical field of robot mapping, and discloses a multi-resolution multi-level map information structure, a dynamic updating system and a construction method for autonomous exploration. In the aspect of map information construction, a multi-level map structure is adopted to distinguish different information amount areas in the environment, and a high-efficiency hash mapping data structure is combined to realize accurate and efficient expression of environment voxel information. In the map incremental updating process, a ray projection method is introduced to perceive environment changes, and a variable resolution strategy is combined to focus on processing environment details with high depth change rate, thereby improving the adaptability and detail performance of map updating. The method provides a map construction and updating solution with higher precision and better efficiency for a robot autonomous exploration system, and provides strong support for exploration tasks in a complex unknown environment.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of robot mapping, and particularly relates to a multi-resolution multi-level map information structure, a dynamic updating system and a construction method for autonomous exploration. BACKGROUND

[0002] In autonomous exploration tasks, accurate positioning and mapping (SLAM) technology is one of the core links to realize environment exploration and task execution, and is the basis for task execution. The construction of a map not only needs to complete the reconstruction of spatial information of an unknown environment, but also is an important evaluation basis for autonomous exploration efficiency, task coverage and search path optimization. At the same time, accurate map information can provide necessary input for downstream planning algorithms, such as obstacle distance, gradient distribution and free space topology, etc., which directly affect the safety, smoothness and efficiency of path planning.

[0003] Map construction technology has been researched and developed for decades, and is generally based on sensors such as binocular vision, laser radar or ultrasonic ranging to provide basic environment perception capability. In the robot platform, binocular vision is widely used for environment mapping due to its lightweight, low cost and strong adaptability. The depth map obtained by processing the binocular camera can construct a two-dimensional or three-dimensional map.

[0004] The visual map construction method is mainly divided into deep learning mapping and geometric method mapping. The deep learning mapping method takes semantic map as the research hotspot, and identifies and constructs a semantic map of key objects in the scene through semantic segmentation and target detection technology, which is suitable for three-dimensional map generation in dynamic environment. However, this kind of method depends on high-performance hardware support, and has poor real-time performance, which is not suitable for resource-limited robot on-board platform.

[0005] In contrast, the geometric method has significant advantages in real-time performance and resource utilization efficiency, and is suitable for robot environment mapping. Traditional geometric methods include ray mapping and projection method. The former updates the observation point of the light ray through the sensor light point, and the latter records depth information by projecting the voxel into the depth map. To improve storage and query efficiency, the voxel hashing method optimizes the map storage structure through dynamic growth and efficient insert, delete and modify operations, which is suitable for dynamic exploration tasks. In addition, the octree map compresses the memory by improving the storage structure, the truncated signed distance field (TSDF) constructs the surface of a three-dimensional object by using the distance field representation, and the ESDF provides obstacle distance and gradient information for motion planning.

[0006] Although the existing methods have their own advantages, in autonomous exploration tasks, map construction needs to consider free space, obstacles and object surface information at the same time to ensure flight safety, path optimization and accuracy of target recognition. However, most existing methods are still limited to the processing of single information, which is insufficient in exploring complex scenes, and the map information of known and unknown areas is not fully utilized to guide and improve the efficiency of exploration tasks, which puts higher comprehensive requirements on mapping technology.

[0007] In summary, these methods have their own advantages in three-dimensional mapping, but still face the problem of balancing efficiency, accuracy and applicability, especially in real-time demand in autonomous exploration scenarios of robots, it is difficult to balance environmental details and exploration efficiency. SUMMARY

[0008] The present application aims to provide a multi-resolution multi-level map information structure, a dynamic updating system and a construction method for autonomous exploration, to solve the above technical problems.

[0009] To solve the above technical problems, the specific technical scheme of a multi-resolution multi-level map construction and dynamic updating method for autonomous exploration is as follows:

[0010] A multi-resolution multi-level map information structure for autonomous exploration includes a first layer map, a second layer map, a third layer map and a fourth layer map.

[0011] The first layer map includes unknown space V unknown and known space V known , the unknown space V unknown represents the area outside the sensor perception range that has not been explored in the environment, the known space V known is the area that has been explored by sensor data, V known is gradually expanded through the accumulation of sensor data, forming an environment map represented by voxels;

[0012] The second layer map is based on known space V known , including free space V free , obstacle space V obstacle and remaining space V residue , which are marked by comparing the occupancy probability value of the grid where the three-dimensional coordinates are located with the threshold value, wherein the free space V free is the area where no obstacles exist determined by sensor data, which is the feasible area of the path in the environment, the occupancy probability value is lower than the threshold value, the obstacle space V obstacle is the impassable area, which is determined to exist obstacles by depth information and visual information, the occupancy probability value is higher than the threshold value, and the remaining space V residue is the blind area, the occupancy probability value is equal to the threshold value;

[0013] The third layer map is based on obstacle space V obstacle , including unobserved space V unobserved , observed space V observed and non-exploration space V noneed , which are determined by the distance from the robot to the obstacle surface, the field of view area and the corresponding threshold value, wherein the unobserved space V unobserved is the area with the distance from the robot to the obstacle surface greater than the set threshold value, and within the FOV threshold value range, the observed space V observed is the area with the distance from the robot to the obstacle surface less than or equal to the set threshold value, and within the FOV threshold value range, and the non-exploration space V noneed is the case that the robot's perception FOV threshold value range is outside, i.e. the robot's roll and pitch angles need to be adjusted additionally;

[0014] The fourth layer map is based on unobserved space V unobserved , including target space V target and non-target space V untarget , which are determined by comparing the variance value of the depth map change with the threshold value, wherein the target space V target is the area with non-smooth depth change, and the change value is greater than or equal to the threshold value, and the non-target space V untarget is the area with smooth depth change, and the change value is less than the threshold value.

[0015] The application also discloses a map dynamic updating system of a map information structure, comprising the following modules:

[0016] A sensor data processing module is used to read the depth map of the binocular camera and the pose information of the unmanned aerial vehicle, wherein the pose information is obtained by calculating the three-dimensional position and attitude of the binocular camera in the world coordinate system, combining the rotation matrix and translation matrix between the camera and the robot, and the conversion relationship between the robot and the world coordinate system, to obtain accurate positioning information;

[0017] A multi-resolution processing module is used to generate a variance map of depth change, and the depth change information is extracted by calculating the variance value of all pixel points in the neighborhood of the pixel point in the uv plane disparity in the depth map;

[0018] An obstacle surface nearest distance processing module is used to calculate the nearest distance from the robot to the obstacle surface, and the shortest distance from the robot to the obstacle surface is represented by comparing the depth information of all points in the FOV field of view under the current state with the original stored information to obtain the smaller value, and the distance information of the obstacle is updated in real time;

[0019] Ray projection update module: for updating the voxel occupancy state based on the sensor data processing module information, calculating the ray path from the camera position to the corresponding depth point through the depth value of each pixel point in the extracted camera coordinate system, and updating the occupancy state of each voxel by gradually traversing the voxel along the ray, updating the occupancy and non-occupancy values of each voxel, and dynamically constructing the voxel map;

[0020] Occupancy information processing module: for judging the voxel attribute, comparing the occupancy value and the non-occupancy value of each voxel, if the occupancy value is greater than the non-occupancy value, calculating and assigning the occupancy probability; otherwise, assigning the non-occupancy probability; then, comparing the calculated occupancy probability with the preset threshold value, when the occupancy probability exceeds the set threshold value, marking the voxel as an obstacle, otherwise marking it as a free space, comparing the values with the unknown region threshold value, and the smaller value is the unknown region boundary;

[0021] Euclidean distance symbol field map update module: using the method of split-axis wavefront propagation, calculating and dynamically updating the shortest distance by axis, limiting the update range to a local area, and only updating and calculating the affected voxel area, reducing the calculation overhead of three-dimensional global traversal.

[0022] Further, the camera position information conversion relationship of the sensor data processing module is:

[0023] P W = R u2w ·(R c2u · P c + T c2u ) + T u2w

[0024] Where R W is the three-dimensional coordinates of the camera in the world coordinate system, R u2w is the rotation matrix of the robot IMU coordinate system mapping the world coordinate system, T u2w is the translation matrix of the robot IMU coordinate system mapping the world coordinate system, R c2u is the rotation matrix of the camera mapping the robot IMU coordinate system, T c2u is the translation matrix of the camera mapping the robot IMU coordinate system, P c is the point in the camera coordinate system.

[0025] Further, the multi-resolution processing module uses the dual-camera carried by the unmanned aerial vehicle platform to obtain depth map data, each pixel point d(u,v) in the depth map represents the depth value under the camera view angle, (u,v) is the two-dimensional coordinates of the pixel point in the depth map; for each pixel point d(u,v) in the depth map, extract the depth values of its eight-connected neighborhood pixel points in the uv plane, denoted as set N(u,v):

[0026] N(u,v) = {d(u+i,v+j) | i,j e {-1,0,1}}

[0027]

[0028]

[0029]

[0030]

[0031] wherein, σ 2 (u,v) is the depth variance value of the pixel point (u,v), μ(u,v) is the mean value of the depth value of the neighborhood, the depth change intensity of the corresponding position of the pixel point is obtained according to the depth variance σ 2 (u,v), compared with the depth change threshold δ, for the part higher than δ, the connected domain R is marked, for R, the depth information d is used for further segmentation, and the points within a certain depth difference are divided into the same new connected domain RR k , the number of voxels in the region is N k , the average value of the depth of the voxels in the region is μ k , the depth change variance in the new region is calculated , compared with the threshold θ, V target is obtained untarget ;

[0032] V target The effect of the region on resolution adjustment is as follows: firstly, according to the state of the voxel, if it is in an occupied state, a unique resolution block index is allocated to each voxel, and the size of the new index array is calculated according to the resolution change; secondly, the local coordinates of the voxels and the subdivision grid positions are calculated, which are mapped to the grid with higher resolution through the subdivision factor, and the grid index after subdivision is updated according to the position of the voxel in the subdivision grid, so as to improve the accuracy of the target region and ensure efficient storage and access under different resolutions.

[0033] Further, the obstacle surface nearest distance processing module receives the depth map data D(x,y) in the current FOV of the robot, each point (x,y) represents the depth value of the corresponding pixel in the depth map, and the stored depth information D stored (x,y) related to the obstacle, the minimum value is taken for each point, and the shortest distance D updated (x,y) is updated, according to the perception range threshold τ set by the robot, the region is divided,

[0034] D updated (x,y) = min(D(x,y),D stored (x,y))

[0035] D updated (x,y)>τ, V unobserved

[0036] D updated (x,y)≤τ, V observed

[0037] For the area outside the FOV, according to the perception ability of the robot, it is marked as V noneed , indicating that the pose of the robot needs to be adjusted additionally.

[0038] Further, the light projection updating module specifically comprises:

[0039] Firstly, the light projection algorithm generates a light vector from the camera position to the depth point by calculating the depth value of each pixel point in the camera coordinate system, the starting point of the light vector is the camera position c=(c x , c y , c z ), and the end point is the point corresponding to the depth value e=(e x , e y , e z ), and the occupancy state of the voxel passing through is updated step by step along the ray r direction in the three-dimensional space, and the light projection updating module generates a unique projection serial number for each light to ensure that each voxel in the field of view is updated only once:

[0040] r=c-e=(c x -e x , c y -e y , c z -e z )

[0041] Secondly, in the process of calculating the voxel intersection point, the environment is discretized into a voxel network, each voxel has a size of Δx×Δy×Δz, the voxel index is (i,j,k), the voxel center is (x i , y i , z k ), the initial voxel index is (i0,j0,k0), and the direction of the light vector is used to initialize the variables and in each axis direction for controlling the stepping mode of the light when passing through the space voxel in each direction, by judging the traversal order of the light in each axis direction, the intersection point of the light and the space voxel is tracked, and according to the traversal time The module determines which coordinate axis the ray vector first crosses the boundary of, and then updates the coordinates of the voxels in turn. The ray vector iterates through the voxels it passes through in a loop until it reaches the destination. During this process, each voxel that passes through updates its occupancy status according to the ray's crossing situation. If the ray passes through the voxel, it is marked as occupied or free.

[0042]

[0043]

[0044] Similarly;

[0045] The ray projection update module processes data in parallel along the axis in three-dimensional space to ensure that the voxel information within the ray coverage area is updated.

[0046] Furthermore, the occupancy information processing module uses a queue data structure to cyclically execute and exit the next voxel information until the queue is empty, and then compares the occupancy value C of each voxel. occ And non-occupied value C unocc If C occ Greater than C unocc Then calculate the occupancy probability P of the voxel. occ And assign a value; otherwise, assign the non-occupancy probability P. free The calculated occupancy probability is compared with a preset threshold γ. When the occupancy probability exceeds the set threshold, the voxel is marked as V. obstacle If the occupancy probability is lower than the set threshold, it is marked as V. free For cases where the occupancy probability equals the threshold, the voxel is labeled as V. residue Furthermore, the occupancy information processing module compares the voxel with an unknown space threshold. When the occupancy probability is less than a set threshold β, the voxel is marked as V. unknown As sensor data accumulates, V known Gradually expand to form an updated environmental map, providing the autonomous exploration system with real-time and reliable information on obstacles and feasible paths;

[0047]

[0048] P free =1-P occ

[0049] P>γ, V obstacle

[0050] β <P<γ,V free

[0051] P = γ, V residue

[0052] P≤β, V unknown .

[0053] Further, the ESDF map updating module decomposes the updating process into propagation calculation along x, y, z axes respectively, first calculates the distance field updating on each axis, in each axis direction, calculates the distance from each voxel to the nearest obstacle surface, and updates the distance by wave front propagation, for each voxel, propagates step by step in the axis direction, from the obstacle surface, until the shortest distance is updated.

[0054] The application further discloses a multi-resolution multi-level map construction method of the map dynamic updating system, including the following steps:

[0055] Step 1: constructing an ESDF map module: through second layer map information, based on V free and V obstacle space construction, first, the sensor data processing module is applied to obtain input, then the ray projection updating module is used to update the occupancy probability and voxel information, finally, based on the information, the ESDF map updating module is used to obtain the updated ESDF map;

[0056] Step 2: constructing a to-be-explored map module: through first and fourth layer map information, based on V unknown , V target and V untarget space construction, first, the sensor data processing module is applied to obtain input, then the ray projection updating module is used to update the occupancy probability and voxel information, the voxel attribute is processed through the occupancy information processing module, and the obstacle, free space and to-be-explored boundary are set;

[0057] Step 3: constructing a surface distance map module: through third layer map information, based on V unobserved and V observed space construction, the sensor data processing module and the obstacle surface nearest distance processing module are called to obtain, and only smaller values are updated;

[0058] Step 4: constructing a depth change map module: through depth information and visual information construction, the sensor data processing module and the multi-resolution processing module are called to obtain.

[0059] Further, the storage of the fourth layer map information is based on a voxel hash method, the voxel hash method includes voxels and voxel blocks, the voxel block is a fixed size three-dimensional cube composed of multiple voxels, the index of the voxel block is mapped to a memory address through a hash function, so that the map storage can be dynamically expanded, the map size is not pre-set, and in the processing process, the index of the voxel block is calculated first, and then the index of the voxel is calculated, and the hash function is represented as:

[0060] value = hash(elem) + const + bit_caculator

[0061] Wherein, hash(elem) is the hash value of the current matrix element, bit_caculator is the bit displacement and mixing operation, the unique hash value value is combined with the cumulative result and the random constant const for reducing hash collision, and is compatible with any storage order.

[0062] The multi-resolution multi-level map information structure, dynamic updating system and construction method for autonomous exploration have the following advantages:

[0063] 1. Multi-level map information expression is provided, environment information is expressed in layers from global unknown and known space to local free space, obstacle space, target area, surface distance map, ESDF map, etc., the application provides more suitable map information for autonomous exploration, increases the environmental details of autonomous exploration, and improves the effectiveness of the map.

[0064] 2. Through the repeatability judgment of ray projection numbering and the judgment of the observed space V observed , the repeated processing of information is effectively avoided, the occupation of invalid memory is reduced, and the efficiency and resource utilization of map construction are improved.

[0065] 3. Through dynamic resolution adjustment of the target space V target , more detailed details are provided for the object area of interest, and the original resolution is maintained for the general smooth wall surface and other areas without special requirements, the exploration details are retained in the known map area while optimizing the resource allocation.

[0066] 4. By using the voxel hash method, the dependence on the preset map size is eliminated, the flexibility and universality of the map construction method are improved, and the autonomous exploration demand of various complex environments can be met.

[0067] 5. The application fully utilizes and mines map information, especially known areas, exploration area details and different types of environmental information, and balances the precision and efficiency of exploration map. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 is a map information structure diagram;

[0069] Figure 2 is a map information schematic diagram;

[0070] Figure 3 is an updating method framework diagram of the application;

[0071] Figure 4 is a resolution adjustment schematic diagram;

[0072] Figure 5 is a schematic diagram of the depth change layer effect of the present application;

[0073] Figure 6 is a schematic diagram of the ESDF map layer effect of the present application;

[0074] Figure 7 is a schematic diagram of the surface distance map layer effect of the present application;

[0075] Figure 8 is a schematic diagram of the to-be-explored map layer effect of the present application. DETAILED DESCRIPTION

[0076] In order to better understand the purpose, structure and function of the present application, the present application of a multi-resolution multi-level map information structure for autonomous exploration, a dynamic updating system and a construction method will be further described in detail below in combination with the drawings.

[0077] As shown in Figure 1 Figure 2 The present application of a multi-resolution multi-level map information structure for autonomous exploration includes a first layer map, a second layer map, a third layer map and a fourth layer map.

[0078] The first layer map includes an unknown space V unknown and a known space V known , wherein the unknown space V unknown represents an area outside the sensor perception range that has not yet been explored in the environment, and the known space V known is an area that has been perceived and explored, and through the accumulation of sensor data, V known gradually expands to form an environment map, represented by voxels.

[0079] The second layer map is based on the known space V known , including a free space V free , an obstacle space V obstacle and a remaining space V residue , which are marked by comparing the occupancy probability value of the grid where the three-dimensional coordinates are located with the threshold value, wherein the free space V free is an area determined by the robot through sensor data that no obstacles exist, which is a feasible area of the path in the environment, and the occupancy probability value is lower than the threshold value, the obstacle space V obstacle is an impassable area, which is determined to exist obstacles through depth information and visual information, and the occupancy probability value is higher than the threshold value, and the remaining space V residue is a blind area, and the occupancy probability value is equal to the threshold value.

[0080] The third layer map is based on the obstacle space V obstacle , including an unobserved space Vunobserved , observed space V observed and unneeded exploration space V noneed , three of which are determined by the distance from the robot to the surface of the obstacle, the field of view (FOV) area and the corresponding threshold value, wherein the unobserved space V unobserved is an area where the distance from the robot to the surface of the obstacle is greater than a set threshold value, and is within the FOV threshold value, the observed space V observed is an area where the distance from the robot to the surface of the obstacle is less than or equal to the set threshold value, and is within the FOV threshold value, and the unneeded exploration space V noneed is a case where the robot's perception FOV is outside the threshold value, i.e. the robot's roll and pitch angles need to be adjusted.

[0081] The fourth layer map is based on the unobserved space V unobserved , including target space V target and non-target space V untarget , both of which are determined by comparing the variance value of the depth map with the threshold value, wherein the target space V target is an area where the depth changes are not smooth, and the change value is greater than or equal to the threshold value, and the non-target space V untarget is an area where the depth changes are smooth, and the change value is less than the threshold value.

[0082] As shown in Figure 3 , a multi-resolution multi-level map dynamic updating system for autonomous exploration of the present application comprises the following modules:

[0083] Sensor data processing module: for reading the depth map of the binocular camera and the pose information of the unmanned aerial vehicle, wherein the pose information is obtained by calculating the three-dimensional position and attitude of the binocular camera in the world coordinate system, combined with the rotation matrix and translation matrix between the camera and the robot, and the conversion relationship between the robot and the world coordinate system, to obtain accurate positioning information.

[0084] As shown in Figure 4 , the multi-resolution processing module: for generating a variance map of depth changes, by calculating the variance value of all pixel points in the uv plane disparity neighborhood of the pixel points in the depth map, to extract the depth change information.

[0085] Obstacle surface closest distance processing module: for calculating the closest distance from the robot to the surface of the obstacle. By comparing the depth information of all points in the FOV field of view under the current state with the original stored information, the smaller value represents the shortest distance from the robot to the surface of the obstacle, and the distance information of the obstacle is updated in real time.

[0086] The light projection update module is configured to update the voxel occupancy state based on the sensor data processing module information, calculate the ray path from the camera position to the corresponding depth point through the depth value of each pixel point in the extracted camera coordinate system, update the occupancy state of each voxel by gradually traversing the voxels along the ray, and dynamically construct the voxel map by accumulating the values of occupancy and non-occupancy.

[0087] The occupancy information processing module is configured to judge the voxel attributes. The occupancy value and the non-occupancy value of each voxel are compared. If the occupancy value is greater than the non-occupancy value, the occupancy probability is calculated and assigned. Otherwise, the non-occupancy probability is assigned. Then, the calculated occupancy probability is compared with a preset threshold value. When the occupancy probability exceeds the set threshold value, the voxel is marked as an obstacle. Otherwise, it is marked as a free space. Further, the values are compared with the unknown region threshold value, and the smaller value is the unknown region boundary.

[0088] The Euclidean Signed Distance Field (ESDF) update module is configured to use the method of split-axis wavefront propagation to dynamically update the shortest distance by calculating and propagating axis by axis, and to reduce the calculation overhead of three-dimensional global traversal by limiting the update range to a local area and only updating and calculating the affected voxel area.

[0089] Specifically,

[0090] The conversion relationship of the camera position information of the sensor data processing module is:

[0091] P W = R u2w · (R c2u · P c + T c2u ) + T u2w

[0092] wherein P W is the three-dimensional coordinates of the camera in the world coordinate system, R u2w is the rotation matrix of the robot IMU coordinate system mapping the world coordinate system, T u2w is the translation matrix of the robot IMU coordinate system mapping the world coordinate system, R c2u is the rotation matrix of the camera mapping the robot IMU coordinate system, T c2u is the translation matrix of the camera mapping the robot IMU coordinate system, and P c is the point in the camera coordinate system.

[0093] The multi-resolution processing module uses the binocular camera carried by the unmanned aerial vehicle platform to obtain depth map data. Each pixel point d(u,v) in the depth map represents the depth value under the camera view angle, and (u,v) is the two-dimensional coordinates of the pixel point in the depth map.

[0094] like Figure 5 As shown, for each pixel d(u,v) in the depth map, the depth values ​​of its eight-connected neighboring pixels in the uv plane are extracted and denoted as set N(u,v):

[0095] N(u,v)={d(u+i,v+j)|i,j∈{-1,0,1}}

[0096]

[0097]

[0098]

[0099]

[0100] Where, σ 2 (u,v) represents the depth variance of pixel (u,v), and μ(u,v) represents the mean depth of the neighborhood. The first row and first column, and the last row and last column of the ignored image are calculated. Based on the depth variance σ... 2 (u,v) retrieves the depth change intensity at the corresponding pixel location and compares it with the depth change threshold δ. For portions higher than δ, a connected component R is labeled using the Breadth First Search (BFS) method. R is further segmented using depth information d. Points within a certain depth difference are grouped into the same new connected component RR using the BFS method. k The number of voxels in the region is N k The average voxel depth within the region is μ k Calculate the variance of depth variation within the new region. V is obtained by comparing it with a threshold θ. target With V untarget .

[0101] V target The role of the region in resolution adjustment is as follows: First, based on the state of the voxel, if it is in an occupied state, a unique resolution block index is assigned to each voxel, and the new index array size is calculated according to the resolution change. Second, the local coordinates and subdivision grid positions of the voxel are calculated, and they are mapped to a higher resolution grid through the subdivision factor. The subdivision grid index is updated according to the position of the voxel in the subdivision grid to improve the accuracy of the target region, while ensuring efficient storage and access at different resolutions.

[0102] The obstacle surface nearest distance processing module receives depth map data D(x,y) within the robot's current field of view (FOV). Each point (x,y) represents the depth value of the corresponding pixel in the depth map, as well as the stored depth information D related to the obstacle. stored(x, y), update the shortest distance D after taking the smaller value for each point updated (x, y), according to the perception range threshold τ set by the robot, the region is divided.

[0103] D updated (x, y) = min(D(x, y), D stored (x, y))

[0104] D updated (x, y) > τ, V unobserved

[0105] D updated (x, y) ≤ τ, V observed

[0106] For the region outside the FOV, according to the perception ability of the robot, it is marked as V noneed , indicating that the pose of the robot needs to be adjusted, such as roll and pitch angles, to expand the field of view coverage.

[0107] The ray projection updating module is used to update the voxel occupancy state based on the information of the sensor data processing module, specifically including:

[0108] First, the ray casting algorithm generates a ray vector from the camera position to the depth point by calculating the depth value of each pixel point in the camera coordinate system. The starting point of the ray vector is the camera position c = (c x , c y , c z ), and the end point is the point corresponding to the depth value e = (e x , e y , e z ), and the occupancy state of the voxel passing through is updated step by step along the ray r direction in three-dimensional space. In order to avoid repeated calculation, the module generates a unique projection serial number for each ray to ensure that each voxel in the field of view is updated only once.

[0109] r = c - e = (c x - e x , c y - e y , c z - e z )

[0110] Secondly, in the process of calculating the intersection of voxels, the environment is discretized into a voxel network, each voxel size is Δx × Δy × Δz, the voxel index is (i, j, k), the voxel center is (x i , y j , z k ), the initial voxel index is (i0, j0, k0), and the direction of the ray vector is used to initialize the variables and The step mode of controlling the light ray to pass through the space voxel in each direction. By judging the passing order of the light ray in each axis direction, tracking the intersection of the light ray and the space voxel, and updating the occupancy state of the voxel according to the time of passing in each axis The module judges which coordinate axis direction the light vector first passes through, and then updates the coordinates of the voxel in turn. The light vector traverses the voxel through a loop until the end point is reached. In the process, each passed voxel updates its occupancy state according to the passing condition of the light ray, and marks it as occupied or free if it passes through the voxel.

[0111]

[0112]

[0113] Similarly

[0114] To ensure the integrity and efficiency of the light ray update operation, the light ray projection update module processes in parallel in the three-dimensional space according to the axis direction, ensuring that the voxel information in the coverage range of the ray is updated.

[0115] The occupancy information processing module is used to judge and update the occupancy attribute of the voxel. Specifically, the module executes and pushes out the next voxel information in a loop through a queue data structure until the queue is empty, compares the occupancy value C occ with the non-occupancy value C unocc , if C occ is greater than C unocc , the occupancy probability P occ of the voxel is calculated and assigned; otherwise, the non-occupancy probability P free is assigned. The calculated occupancy probability is compared with the preset threshold value γ, when the occupancy probability exceeds the set threshold value, the voxel is marked as V obstacle ; if the occupancy probability is lower than the set threshold value, it is marked as V free . For the case where the occupancy probability is equal to the threshold value, the voxel is marked as V residue . Further, the module compares the voxel with the unknown space threshold value, when the occupancy probability is less than the set threshold value β, the voxel is marked as V unknown . Through this method, V known gradually expands to form an updated environment map, providing real-time and reliable obstacle and feasible path information for the autonomous exploration system.

[0116]

[0117] P free = 1-P occ

[0118] P>γ, V obstacle

[0119] β<P<γ, V free

[0120] P=γ, V residue

[0121] P≤β, V unknown

[0122] As Figure 6 shown, the ESDF update module decomposes the update process into propagation calculations along the x, y, z axes respectively, first calculating the update of the distance field on each axis. This method avoids the high computational overhead of three-dimensional global traversal by processing axis by axis. Specifically, in each axis direction, the distance of each voxel to the nearest obstacle surface is calculated, and the distance is updated by wavefront propagation. For each voxel, the propagation is performed step by step in the axis direction, starting from the obstacle surface until the shortest distance is updated.

[0123] The autonomous exploration-oriented multi-resolution multi-level map construction method of the application comprises the following steps:

[0124] Step 1: Constructing the Euclidean distance symbolic field map module: through the second layer map information, based on V free and V obstacle space construction, first applying the sensor data processing module to obtain input, then using the ray projection update module to update the occupancy probability and voxel information, and finally based on the information, using the Euclidean distance symbolic field map update module to obtain the updated ESDF map.

[0125] Step 2: Constructing the to-be-explored map module: through the first, fourth layer map information, based on V unknown , V target and V untarget space construction, first applying the sensor data processing module to obtain input, then using the ray projection update module to update the occupancy probability and voxel information, processing the voxel attributes through the occupancy information processing module, and setting the obstacle, free space and to-be-explored boundary.

[0126] Step 3: Constructing the surface distance map module: as Figure 7 shown, through the third layer map information, based on V unobserved and V observed space construction, calling the sensor data processing module and the obstacle surface nearest distance processing module to obtain, and only updating smaller values.

[0127] Step 4: Constructing the depth change map module: through the depth information and visual information construction, calling the sensor data processing module and the multi-resolution processing module to obtain.

[0128] Further, the storage of the four-layer map information is based on a voxel hash method, which optimizes the map storage structure through dynamic growth and efficient add, delete, modify and query operations.

[0129] Further, the voxel hash method includes voxels and voxel blocks, the voxel block is a fixed-size three-dimensional cube composed of multiple voxels, the index of which is mapped to a memory address through a hash function, so that the map storage can be dynamically expanded, the map size is not pre-set, and in the processing process, the index of the voxel block is calculated first, and then the index of the voxel is calculated, so as to reduce the time occupied by the frequent use of the find function of unordered_map type, and reduce the time complexity. The hash function is represented as:

[0130] value = hash(elem) + const + bit_caculator

[0131] Wherein, hash(elem) is the hash value of the current matrix element, bit_caculator is a bit shift and mixing operation, combined with the cumulative result and a random constant const used to reduce hash collision, the unique hash value value can be compatible with any storage order.

[0132] It can be understood that the present application is described through some embodiments, and those skilled in the art know that various changes or equivalent replacements can be made to these features and embodiments without departing from the spirit and scope of the present application. In addition, under the guidance of the present application, these features and embodiments can be modified to adapt to specific conditions and materials without departing from the spirit and scope of the present application. Therefore, the present application is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of the present application are within the scope of the present application.

Claims

1. A map dynamic updating system for autonomous exploration-oriented multi-resolution multi-level map information structure, the map information structure comprising a first layer map, a second layer map, a third layer map and a fourth layer map; The first layer map includes unknown space V unknown and known space V known , the unknown space V unknown represents the area outside the sensor perception range which has not been explored in the environment, the known space V known is the area which has been perceived and explored, and V known is gradually expanded by the accumulation of sensor data to form an environment map represented by voxels; The second layer map is based on a known space V known , including free space V free , obstacle space V obstacle and remaining space V residue , which are marked by comparing the occupancy probability value of the grid where the three-dimensional coordinates are located with the threshold value, wherein the free space V free is an area where no obstacles exist determined by the sensor data of the robot, which is a feasible area of the path in the environment, and the occupancy probability value is lower than the threshold value, the obstacle space V obstacle is an area that cannot be passed, which is determined to exist obstacles by depth information and visual information, and the occupancy probability value is higher than the threshold value, and the remaining space V residue is a blind area of the field of view, and the occupancy probability value is equal to the threshold value; The third layer map is based on obstacle space V obstacle , including unobserved space V unobserved , observed space V observed and no need to explore space V noneed , which are determined by the distance from the robot to the obstacle surface, the field of view area and the corresponding threshold value, wherein the unobserved space V unobserved is the area with the distance from the robot to the obstacle surface greater than the set threshold value and within the FOV threshold value range, the observed space V observed is the area with the distance from the robot to the obstacle surface less than or equal to the set threshold value and within the FOV threshold value range, and the no need to explore space V noneed is the case that the robot's perception FOV threshold value range is outside, i.e. the robot's roll and pitch angles need to be adjusted additionally; The fourth layer map is based on unobserved space V unobserved , including target space V target and non-target space V untarget , both are judged by comparing the variance value of depth map change with the threshold value, wherein the target space V target is the area with non-smooth depth change, and the change value is greater than or equal to the threshold value, and the non-target space V untarget is the area with smooth depth change, and the change value is less than the threshold value; characterized in that The map dynamic updating system comprises the following modules: A sensor data processing module for reading the depth map of a binocular camera and the pose information of a UAV, wherein the pose information is obtained by calculating the three-dimensional position and attitude of the binocular camera in the world coordinate system, combining the rotation matrix and translation matrix between the camera and the robot, and the conversion relationship between the robot and the world coordinate system to obtain accurate positioning information; A multi-resolution processing module for generating a variance map of depth changes, which extracts depth change information by calculating the variance value of all pixel points in the neighborhood of the pixel point in the depth map in the uv plane disparity; An obstacle surface nearest distance processing module for calculating the nearest distance from the robot to the obstacle surface, which compares the depth information of all points in the FOV field under the current state with the original stored information to take the smaller value, representing the shortest distance from the robot to the obstacle surface, and updates the distance information of the obstacle in real time; A ray projection updating module for updating the voxel occupancy state based on the sensor data processing module information, which calculates the ray path from the camera position to the corresponding depth point by extracting the depth value of each pixel point in the camera coordinate system, and updates the occupancy state of each voxel by gradually traversing the voxels along the ray path, and dynamically constructs the voxel map by accumulating the values of occupancy and non-occupancy; An occupancy information processing module for judging the voxel attributes, which compares the occupancy value and the non-occupancy value of each voxel, and if the occupancy value is greater than the non-occupancy value, calculates and assigns the occupancy probability; otherwise, assigns the non-occupancy probability; then, compares the calculated occupancy probability with the preset threshold value, and when the occupancy probability exceeds the set threshold value, marks the voxel as an obstacle, otherwise marks it as a free space, and compares the values with the unknown region threshold value, and the smaller value is the unknown region boundary; An Euclidean distance signed field map updating module which uses the method of split-axis wavefront propagation to calculate and dynamically update the shortest distance by limiting the update range to a local area and only updating the affected voxel area, thereby reducing the computational overhead of three-dimensional global traversal.

2. The map dynamic updating system according to claim 1, characterized in that, The conversion relationship of the camera position information of the sensor data processing module is: ， wherein is a three-dimensional coordinate of the camera in the world coordinate system, is a rotation matrix of the robot IMU coordinate system mapping the world coordinate system, is a translation matrix of the robot IMU coordinate system mapping the world coordinate system, is a rotation matrix of the camera mapping the robot IMU coordinate system, is a translation matrix of the camera mapping the robot IMU coordinate system, is a point in the camera coordinate system.

3. The map dynamic updating system according to claim 1, characterized in that, The multi-resolution processing module uses the binocular camera carried by the UAV platform to obtain depth map data, and each pixel point d(u,v) in the depth map represents the depth value under the camera view angle, and (u,v) is the two-dimensional coordinate of the pixel point in the depth map; for each pixel point d(u,v) in the depth map, the depth values of its eight-connected neighborhood pixel points in the uv plane are extracted, denoted as set N(u,v): ， wherein, is the depth variance value of the (u, v) pixel point, is the mean value of the neighborhood depth value, according to the depth variance obtain the depth change intensity of the corresponding position of the pixel point, and compare the depth change intensity with a depth change threshold , and for the part higher than , perform connected domain R marking, and for R, further segment using depth information d, and points within a certain depth difference are divided into the same new connected domain , the number of voxels in the region is , the average value of the voxel depth in the region is , calculate the depth change variance in the new region , and compare the depth change variance with a threshold , and obtain and ; The role of the region for resolution adjustment is: firstly, according to the state of the voxel, if it is in the occupied state, a unique resolution block index is assigned to each voxel, and the new index array size is calculated according to the resolution change; secondly, the local coordinates of the voxel and the subdivision grid position are calculated, which are mapped to the grid with higher resolution through the subdivision factor, and the grid index after subdivision is updated according to the position of the voxel in the subdivision grid, so as to improve the accuracy of the target region, while ensuring efficient storage and access under different resolutions.

4. The map dynamic updating system according to claim 1, characterized in that, The obstacle surface nearest distance processing module receives the depth map data within the current FOV of the robot Each point (x, y) represents the depth value of the corresponding pixel in the depth map, and the stored depth information related to the obstacle Update the shortest distance after taking the smaller value for each point According to the perception range threshold set by the robot Carry out regional division, ， For the regions outside the FOV, according to the sensing ability of the robot, mark as V noneed , indicating that the pose of the robot needs to be adjusted additionally.

5. The map dynamic updating system according to claim 1, characterized in that, The ray projection updating module specifically comprises: Firstly, the light projection algorithm generates a light vector from the camera position to the depth point by calculating the depth value of each pixel point in the camera coordinate system, the starting point of the light vector is the camera position , the end point is the point corresponding to the depth value , and the occupancy state of the voxel passed is updated step by step along the ray r direction in three-dimensional space, the light projection update module generates a unique projection serial number for each light to ensure that each voxel in the field of view is updated only once: ， Secondly, in the process of calculating the intersection of voxels, the environment is discretized into a network of voxels, each voxel has a size of , a voxel index of , a voxel center of , an initial voxel index of , and the direction of the ray vector is used to initialize the variables , and in each axis direction, which are used to control the stepping mode of the ray when it passes through the space voxels in each direction. By judging the traversal order of the ray in each axis direction, the intersection of the ray and the space voxel is tracked. According to the traversal time of each axis , , , the module determines which coordinate axis direction the ray vector first traverses, and then updates the coordinates of the voxel in turn. The ray vector traverses the voxels step by step through a loop until it reaches the end point. In this process, each traversed voxel updates its occupancy state according to the traversal of the ray. If the voxel is traversed, it is marked as occupied or free. ； The ray projection updating module processes in parallel in the axial direction in three-dimensional space to ensure that the voxel information in the coverage range of the ray is updated.

6. The map dynamic updating system according to claim 1, characterized in that, The occupancy information processing module executes in a loop and pushes out the next voxel information through a queue data structure until the queue is empty, compares the occupancy value of each voxel with the non-occupancy value , if is greater than , calculates the occupancy probability of the voxel and assigns the value; Conversely, the value is the non-occupancy probability The calculated occupancy probability is compared with a preset threshold When the occupancy probability exceeds the set threshold, the voxel is marked as V obstacle If the occupancy probability is lower than the set threshold, it is marked as V free If the occupancy probability is equal to the threshold, the voxel is marked as V residue Further, the occupancy information processing module compares the voxel with an unknown space threshold, and when the occupancy probability is less than the set threshold , the voxel is marked as V unknown With the accumulation of sensor data, V known gradually expands to form an updated environment map, providing real-time and reliable obstacle and feasible path information for the autonomous exploration system; 。 7. The map dynamic updating system according to claim 1, characterized in that, The Euclidean distance symbol field map updating module decomposes the updating process into propagation calculation along x, y and z axes respectively, first calculates the distance field updating on each axis, calculates the distance from each voxel to the nearest obstacle surface in each axis direction, and updates the distance by wave front propagation, for each voxel, propagates step by step in the axis direction, starts from the obstacle surface, and updates the shortest distance.

8. A method of constructing a multi-resolution multi-level map of a dynamic map updating system according to any one of claims 2 to 7, characterized in that, The method comprises the following steps: Step 1: Constructing the Euclidean Distance Signed Field Map module: Based on the second layer map information, the V free and V obstacle Space construction, first, the sensor data processing module is applied to obtain input, then the ray projection update module is used to update the occupancy probability and voxel information, and finally, based on the information, the Euclidean Distance Signed Field Map update module is used to obtain the updated ESDF map; Step 2: Constructing the module to be explored: through the first, fourth layer map information, based on V unknown , V target and V untarget space construction, first apply the sensor data processing module to obtain input, then use the ray projection update module to update the occupancy probability and voxel information, process the voxel attributes through the occupancy information processing module, and set the obstacles, free space and boundaries to be explored; Step 3: Constructing the surface distance map module: through the third layer map information, based on V unobserved and V observed Space construction, call sensor data processing module and obstacle surface nearest distance processing module to obtain, only update smaller values; Step 4: Constructing the depth change map module: constructing by depth information and visual information, calling the sensor data processing module and the multi-resolution processing module to obtain.

9. The multi-resolution multi-level map construction method of claim 8, wherein, The storage of the four-layer map information is based on a voxel hashing method, the voxel hashing method comprises voxels and voxel blocks, the voxel block is a fixed-size three-dimensional cube composed of multiple voxels, the index of the voxel block is mapped to a memory address through a hash function, so that the map storage can be dynamically expanded, the map size is not pre-set, and in the processing process, the index of the voxel block is calculated first, and then the index of the voxel is calculated, the hash function is represented as: ， Wherein, hash(elem) is the hash value of the current matrix element, bit_caculator is a bit shift and mixing operation, combined with the accumulated result and a random constant const used to reduce hash conflicts, a unique hash value value, compatible with any storage order.

Images