A lightweight map representation method for UAV based on depth information

Abstract

The present invention discloses a lightweight UAV map representation method based on depth information, which separates the fine part and the coarse part of each frame of dense point cloud data, and represents them with sparse point sets in rich texture areas and sparse surface element sets in low texture areas, thereby retaining the environmental details in the high-resolution depth map while reducing redundant depth information. The relative relationship between a series of sparse maps and the camera poses of each frame is stored, and a complete query interface is provided for the map, so as to realize the lightweight map representation of dense environments, and set the criteria for updating historical data of the map.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicles and map representation, and in particular to a method for a rotary-wing unmanned aerial vehicle equipped with a direct or indirect depth sensor to construct a lightweight map representation directly based on depth information. Background Art

[0002] When a drone is performing a mission in a complex environment, it generally uses depth sensors to obtain the relative position information of obstacles, and then builds a raster map to further plan an executable trajectory. However, maintaining such a fused map online requires a large amount of memory and processing performance overhead, and in the case of poor positioning, maintaining the consistency of the map also requires a lot of computing power to ensure the quality of planning. Therefore, directly using depth information to construct a lightweight map representation method can greatly improve the efficiency of drones, which are computationally sensitive vehicles, to fly safely in non-fixed environments.

[0003] Existing technologies usually project depth information onto a three-dimensional point cloud, build a kd-tree (k dimensional tree), and provide a nearest neighbor query interface to characterize the map. However, due to the limited detection range of the depth sensor, the depth information of a single observation is relatively small. Existing technologies usually directly maintain a queue, build a separate kd-tree for each frame of depth sensor data, and store the relative relationship between the camera position and posture between frames. When querying the nearest neighbor, start with the most recent frame of data and first determine whether it is within the sensor's perception range. If so, the query result is obtained. Otherwise, continue to query the historical data until the query is found.

[0004] The methods of the prior art have the following defects:

[0005] 1. It takes up a lot of memory, even more than the fused rasterized map. Since a large amount of historical sensor data is stored, more memory is needed, which brings additional burden to the limited computing and storage resources on the drone;

[0006] 2. Incompleteness of query results: When the query point is close to the edge of the depth sensor's sensing range, the query result is likely to be wrong.

[0007] In summary, existing map representation algorithms on UAV platforms generally have problems such as requiring large memory and processing performance overhead. The incompleteness of query results will also lead to defects in the UAV's obstacle perception function, causing great problems for the safe flight of UAVs, which are vehicles sensitive to computing resources, in non-fixed environments. Summary of the invention

[0008] In order to solve the above problems, the present invention provides a lightweight map representation method for UAV based on depth information. The method maintains a queue to store the relative relationship between a series of sparse maps and camera positions and postures, and provides a complete nearest neighbor query interface to characterize the map, thereby realizing sparse map representation of dense environments.

[0009] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0010] A method for representing a lightweight map of an unmanned aerial vehicle based on depth information, the method comprising the following steps:

[0011] Step 1: continuously obtain a depth map through a sensor, and use edge detection or high-frequency information extraction methods on the depth map to separate pixels in the texture-rich area;

[0012] Step 2: Using the projection relationship of the sensor, the depth information of the extracted pixels is restored into a three-dimensional point cloud to form a sparse point cloud map of the texture-rich area;

[0013] Step 3, when querying the safe space of a spatial point, if the queried spatial point is not in the sparse point cloud map of the rich texture area, a sparse surface is generated for this spatial point during the query process and the sparse point cloud map is updated;

[0014] Step 4, storing the sparse point cloud maps and sensor poses corresponding to a series of depth maps into the map queue and pose queue respectively;

[0015] Step 5: Establish a global map based on the data in the map queue and the pose queue, and provide a query interface for the sparse point cloud map to obtain a sparse map representation of the dense environment.

[0016] Furthermore, in step 1, the edge detection or high-frequency information extraction method is a Canny edge detection method, and the sensor is a lidar, a depth camera, a stereo camera or a monocular camera.

[0017] Furthermore, the specific method of step 2 is:

[0018] Step 2.1, converting the pixel points in the extracted depth map into three-dimensional point information according to the projection formula of the sensor;

[0019] Step 2.2, collect the three-dimensional points to obtain a sparse point cloud map of the rich texture area.

[0020] Furthermore, the specific method of step 3 is:

[0021] Step 3.1, when the safe space of any spatial point A is to be queried, first query its nearest neighbor distance in the sparse point cloud map of the rich texture area, denoted as L1;

[0022] Step 3.2, project point A onto the imaging plane of the sensor to obtain the corresponding pixel point B. If the depth of point B has been used to construct a sparse point cloud map of a texture-rich area, directly return the nearest distance L1 and complete step 3; otherwise, perform approximate calculation of the surface element of the environment where point B is located, and then execute steps 3.3-3.5; the specific method of surface element approximation calculation is:

[0023] Take point B as the center of the initial surface element, L1 as the surface element radius, and calculate the normal vector of the pixel area, i.e. the surface element, based on the depth information of the pixels within the radius of point B;

[0024] According to the normal vector information, the center position of the face element and the surface element area are updated, and then the normal vector of the face element area is updated until the change of the center position of the face element is less than the preset value;

[0025] Step 3.3, based on the geometric information of the facet, the index of the pixels covered by the projection to the facet is established, and the facet replaces these pixels, compressing the redundant pixel depth information into 6 parameters consisting of position and normal direction, so as to realize the lightweight representation and update of the map;

[0026] Step 3.4, based on the geometric information of the surface element and point A, calculate the shortest distance from point A to the surface element, denoted as L2;

[0027] Step 3.5, return the smaller value of L1 and L2 as the query result, and determine whether the query result is complete through the query interface;

[0028] In subsequent query operations, if the pixel where point B is located has been constructed with a surfel, the surfel in this area will not be constructed again.

[0029] Furthermore, the specific method of step 4 is: when the difference between the sensor pose corresponding to the current depth map frame and the latest sensor pose in the pose queue exceeds a preset value, or when the difference between the timestamp of the current depth map frame and the timestamp of the latest data in each queue exceeds a preset value, the sparse point cloud map and sensor pose corresponding to the current depth map frame are stored in the map queue and the pose queue, respectively.

[0030] Furthermore, the specific method of step 5 is:

[0031] Step 5.1, obtaining the relative relationship between the sensor poses according to the pose queue, and establishing a global map by using the relative relationship between the sparse point cloud map in the map queue and the sensor poses;

[0032] Step 5.2, when querying the distance between any point A in the space and all obstacles in each sparse point cloud map, for each sparse point cloud map, the distance between point A and the sensor perception boundary corresponding to the sparse point cloud map is also calculated;

[0033] Step 5.3: If the minimum obstacle distance obtained by the query is less than the corresponding sensor perception boundary distance, it means that the query result is complete and the query result is obtained; otherwise, it means that the query result is incomplete;

[0034] In step 5.4, when the query result is incomplete, the relationship between the sensor poses is used to transform the coordinates of point A to the sensor coordinate system corresponding to the previous sensor pose, and the query and calculation process is repeated until the nearest obstacle information of point A is obtained.

[0035] A method for representing a lightweight map of an unmanned aerial vehicle based on depth information comprises the following steps:

[0036] Use edge detection or high-frequency information extraction methods on the depth map to separate the pixels in the texture-rich area;

[0037] Using the camera projection relationship, the depth information of the extracted pixels is restored to a three-dimensional point cloud to form a sparse point cloud map of the texture-rich area;

[0038] The remaining pixels in the depth map generally have relatively flat depth variations, which are used in the nearest neighbor distance query process to process them, generate sparse surfaces and update the map;

[0039] The relative relationship between a series of sparse maps and the camera pose of each frame is stored, and a complete query interface is provided for the map to achieve sparse map representation of dense environments.

[0040] Furthermore, the depth information of the extracted pixels is restored to a three-dimensional point cloud to form a sparse point cloud map of the texture-rich area, including:

[0041] According to the camera projection formula, the pixels extracted from the depth map are converted into three-dimensional point information;

[0042] The obtained point set is processed to obtain a sparse point cloud map of the texture-rich area;

[0043] When used in the nearest neighbor distance query process, it is processed to generate a sparse surface and update the map, including:

[0044] When the safe space of any spatial point A is to be queried, first query its nearest distance in the point set of the rich texture area, denoted as L1;

[0045] Project point A onto the camera plane to obtain the corresponding pixel point B. If the depth of point B has been used to construct a point set in the rich texture area, directly return its nearest distance; otherwise, perform an approximate calculation of the surface element of the environment where point B is located;

[0046] Take point B as the center of the initial surface element, take L1 as the surface element radius, and calculate the normal vector of the pixel area, i.e. the surface element, based on the depth information of the pixels within the radius of point B;

[0047] According to the normal vector information, the center position of the face element and the surface element area are updated, and then the normal vector of the face element area is updated until the change of the center position of the face element is less than the preset value;

[0048] According to the geometric information of the facet, the index of the pixels covered by the projection is established, and the facet replaces these pixels. The large amount of redundant pixel depth information is compressed into 6 parameters consisting of position and normal direction, realizing the lightweight representation and update of the map.

[0049] According to the geometric information of the face element and point A, the shortest distance from point A to the face element is calculated, which is recorded as L2;

[0050] Return the smaller value of L1 and L2 as the query result, and use the query interface to determine whether the query result is complete;

[0051] In subsequent query operations, if the pixel where point B is located has been constructed, the surface element in this area will not be constructed again;

[0052] The relative relationship between a series of sparse maps and the camera poses of each frame is stored, and a complete query interface is provided for the map, including:

[0053] The sparse map of each frame and the relative relationship between the camera poses of each frame are stored to build a global map;

[0054] When querying the distance between any point A in the space and an obstacle in the map, the distance to the sensor perception boundary of the frame from the query point is calculated at the same time;

[0055] If the query result of the nearest obstacle distance is less than the distance to the boundary, it means that the query result is complete and the query result is obtained; otherwise, it means that the query result is incomplete;

[0056] When the query result is incomplete, the position relationship between cameras is used to transform the coordinates of point A to the coordinate system of the previous frame camera, and the query and calculation process is repeated until the nearest obstacle information of point A is obtained.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] 1. The present invention separates the fine part and the coarse part of the dense point cloud data of each frame, and represents them with a sparse point set in the rich texture area and a sparse surface element set in the low texture area, thereby retaining the environmental details in the high-resolution depth map and reducing redundant depth information.

[0059] 2. The present invention stores a series of sparse maps and the relative relationship between the camera positions of each frame, and provides a complete query interface for the map, realizes lightweight map representation of dense environments, and sets criteria for updating historical data of the map.

[0060] 3. The present invention realizes the sparse representation of dense environment, which can greatly reduce the amount of point cloud data, and the data information can better represent the environmental information. Whether it is memory usage or query efficiency, there is a significant performance improvement.

[0061] 4. The present invention provides a complete query interface. In complex environments, the nearest neighbor query interface provided by the prior art can easily cause the trajectory to collide with obstacles, which is extremely dangerous for drones. The method of the present invention provides a complete query interface. Within the allowable range of sensor error, the trajectory generated by the drone can avoid obstacles, which is safer for drones.

[0062] 5. The method of the present invention is accurate and fast. The present invention reduces the overall data volume and improves the speed of nearest neighbor query by setting two criteria for updating historical data. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0064] Figure 1 Flowchart for sparse map representation of dense environments.

[0065] Figure 2 Schematic diagram of the incomplete results when querying historical data for existing methods.

[0066] Figure 3 Schematic diagram of the transformation of depth map into sparse point cloud in texture-rich area and sparse surface elements in low-texture area. Specific implementation methods

[0067] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0069] A method for representing a lightweight map of an unmanned aerial vehicle based on depth information, the method comprising the following steps:

[0070] Step 1: continuously obtain a depth map through a sensor, and use edge detection or high-frequency information extraction methods on the depth map to separate pixels in the texture-rich area;

[0071] Step 2: Using the projection relationship of the sensor, the depth information of the extracted pixels is restored into a three-dimensional point cloud to form a sparse point cloud map of the texture-rich area;

[0072] Step 3, when querying the safe space of a spatial point, if the queried spatial point is not in the sparse point cloud map of the rich texture area, a sparse surface is generated for this spatial point during the query process and the sparse point cloud map is updated;

[0073] Step 4, storing the sparse point cloud maps and sensor poses corresponding to a series of depth maps into the map queue and pose queue respectively;

[0074] Step 5: Establish a global map based on the data in the map queue and the pose queue, and provide a query interface for the sparse point cloud map to obtain a sparse map representation of the dense environment.

[0075] In step 1, the edge detection or high-frequency information extraction method is a Canny edge detection method, and the sensor is a lidar, a depth camera, a stereo camera or a monocular camera.

[0076] The specific method of step 2 is:

[0077] Step 2.1, converting the pixel points in the extracted depth map into three-dimensional point information according to the projection formula of the sensor;

[0078] Step 2.2, collect the three-dimensional points to obtain a sparse point cloud map of the rich texture area.

[0079] The specific method of step 3 is:

[0080] Step 3.1, when the safe space of any spatial point A is to be queried, first query its nearest neighbor distance in the sparse point cloud map of the rich texture area, denoted as L1;

[0081] Step 3.2, project point A onto the imaging plane of the sensor to obtain the corresponding pixel point B. If the depth of point B has been used to construct a sparse point cloud map of a texture-rich area, directly return the nearest distance L1 and complete step 3; otherwise, perform approximate calculation of the surface element of the environment where point B is located, and then execute steps 3.3-3.5; the specific method of surface element approximation calculation is:

[0082] Take point B as the center of the initial surface element, L1 as the surface element radius, and calculate the normal vector of the pixel area, i.e. the surface element, based on the depth information of the pixels within the radius of point B;

[0083] According to the normal vector information, the center position of the face element and the surface element area are updated, and then the normal vector of the face element area is updated until the change of the center position of the face element is less than the preset value;

[0084] Step 3.3, based on the geometric information of the facet, the index of the pixels covered by the projection to the facet is established, and the facet replaces these pixels, compressing the redundant pixel depth information into 6 parameters consisting of position and normal direction, so as to realize the lightweight representation and update of the map;

[0085] Step 3.4, based on the geometric information of the surface element and point A, calculate the shortest distance from point A to the surface element, denoted as L2;

[0086] Step 3.5, return the smaller value of L1 and L2 as the query result, and determine whether the query result is complete through the query interface;

[0087] In subsequent query operations, if the pixel where point B is located has been constructed with a surfel, the surfel in this area will not be constructed again.

[0088] The specific method of step 4 is: when the difference between the sensor pose corresponding to the current depth map frame and the latest sensor pose in the pose queue exceeds a preset value, or when the difference between the timestamp of the current depth map frame and the timestamp of the latest data in each queue exceeds a preset value, the sparse point cloud map and sensor pose corresponding to the current depth map frame are stored in the map queue and the pose queue respectively.

[0089] The specific method of step 5 is:

[0090] Step 5.1, obtaining the relative relationship between the sensor poses according to the pose queue, and establishing a global map by using the relative relationship between the sparse point cloud map in the map queue and the sensor poses;

[0091] Step 5.2, when querying the distance between any point A in the space and all obstacles in each sparse point cloud map, for each sparse point cloud map, the distance between point A and the sensor perception boundary corresponding to the sparse point cloud map is also calculated;

[0092] Step 5.3: If the minimum obstacle distance obtained by the query is less than the corresponding sensor perception boundary distance, it means that the query result is complete and the query result is obtained; otherwise, it means that the query result is incomplete;

[0093] In step 5.4, when the query result is incomplete, the relationship between the sensor poses is used to transform the coordinates of point A to the sensor coordinate system corresponding to the previous sensor pose, and the query and calculation process is repeated until the nearest obstacle information of point A is obtained.

[0094] A lightweight map representation method for UAVs based on depth information, such as Figure 1 As shown, including:

[0095] S1: Use edge detection or high-frequency information extraction methods (such as Canny edge detection method) on the depth map to separate the pixels in the texture-rich area;

[0096] S2: Using the camera projection relationship, the depth information of the extracted pixels is restored to a three-dimensional point cloud to form a sparse point cloud map of the rich texture area;

[0097] S3: The remaining pixels in the depth map generally have relatively flat depth variations, which are used in the nearest neighbor distance query process to process them, generate sparse surfaces and update the map;

[0098] S4: Stores a series of sparse maps and the relative relationship between the camera poses of each frame, and provides a complete query interface for the map to achieve sparse map representation of dense environments.

[0099] In addition, this method also sets the criteria for map update history data, including:

[0100] The pose update criterion is that when the pose of the camera in the current frame differs from the pose of the latest frame in the queue by more than a certain threshold, the historical data is updated and added to the queue;

[0101] The time difference update criterion is that when the difference between the timestamp of the current frame camera and the historical timestamp of the latest frame in the queue exceeds a certain threshold, the historical data is updated and added to the queue.

[0102] In step S1, the source of the depth map includes but is not limited to a laser radar, a depth camera, a stereo camera, a monocular camera, etc. In addition, any other edge extraction and high frequency extraction means can be an alternative to Canny edge detection.

[0103] Step S2 includes:

[0104] S21: converting the pixel points extracted from the depth map into three-dimensional point information according to the camera projection formula;

[0105] S22: Process the obtained point set to obtain a sparse point cloud map of the rich texture area.

[0106] Step S3 includes:

[0107] S31: When the safe space of any spatial point A is to be queried, first query its nearest distance in the point set of the rich texture area, which is recorded as L1;

[0108] S32: Project point A onto the camera plane to obtain the corresponding pixel point B. If the depth of point B has been used to construct the point set of the rich texture area, directly return its nearest distance. Otherwise, perform an approximate calculation of the surface element of the environment where point B is located, such as Figure 3 As shown, the specific method is:

[0109] S321: taking point B as the center of the initial surface element, taking L1 as the surface element radius, and calculating the normal vector of the pixel area, i.e., the surface element, according to the depth information of the pixels within the radius of point B;

[0110] S322: updating the center position of the facet and the surface facet region according to the normal vector information, and then updating the normal vector of the facet region until the change in the center position of the facet is less than a preset value;

[0111] S33: Establish the index of the pixels covered by the projection according to the geometric information of the facet, and complete the replacement of these pixels by the facet, compress a large amount of redundant pixel depth information into 6 parameters consisting of position and normal direction, and realize the lightweight representation and update of the map;

[0112] S34: Calculate the shortest distance from point A to the surface element according to the geometric information of the surface element and point A, which is recorded as L2;

[0113] S35: Return the smaller value of L1 and L2 as the query result, and determine whether the query result is complete through the query interface.

[0114] In subsequent query operations, if the pixel where point B is located has been constructed with a surfel, the surfel in this area will not be constructed again.

[0115] like Figure 2 As shown, the results of the existing method when querying historical data may be incomplete. Therefore, step S4 of this method adopts the following method:

[0116] S41: storing the relative relationship between the sparse map of each frame and the camera pose of each frame to establish a global map;

[0117] S42: when querying the distance between any point A in the space and an obstacle in the map, simultaneously calculating the distance from the query point to the sensor perception boundary of the frame;

[0118] S43: If the query result of the nearest obstacle distance is less than the distance to the boundary, it means that the query result is complete and the query result is obtained; otherwise, it means that the query result is incomplete;

[0119] S44: When the query result is incomplete, the coordinates of point A are converted to the coordinate system of the camera of the previous frame by using the posture relationship between the cameras, and the query and calculation process are repeated until the nearest obstacle information of point A is obtained.

[0120] In summary, the present invention separates the fine part and the coarse part of the dense point cloud data of each frame, and represents them with sparse point sets in rich texture areas and sparse surface element sets in low texture areas, which retains the environmental details in the high-resolution depth map while reducing redundant depth information. The relative relationship between a series of sparse maps and the camera poses of each frame is stored, and a complete query interface is provided for the map, so as to realize the lightweight map representation of dense environments and set the criteria for updating historical data of the map.

Claims

1. A method for representing lightweight maps of unmanned aerial vehicles based on depth information, characterized in that: The following steps are involved: Step 1: continuously obtain a depth map through a sensor, and use edge detection or high-frequency information extraction methods on the depth map to separate pixels in the texture-rich area; Step 2: Using the projection relationship of the sensor, the depth information of the extracted pixels is restored into a three-dimensional point cloud to form a sparse point cloud map of the texture-rich area; Step 3, when querying the safe space of a spatial point, if the queried spatial point is not in the sparse point cloud map of the rich texture area, a sparse surface is generated for this spatial point during the query process and the sparse point cloud map is updated; Step 4, storing the sparse point cloud maps and sensor poses corresponding to a series of depth maps into the map queue and pose queue respectively; Step 5: Establish a global map based on the data in the map queue and the pose queue, and provide a query interface for the sparse point cloud map to obtain a sparse map representation of the dense environment.

2. The method for representing a lightweight map of an unmanned aerial vehicle based on depth information according to claim 1, characterized in that: In step 1, the edge detection or high-frequency information extraction method is a Canny edge detection method, and the sensor is a lidar, a depth camera, a stereo camera or a monocular camera.

3. The method for representing a lightweight map of an unmanned aerial vehicle based on depth information as claimed in claim 1, characterized in that: The specific method of step 2 is: Step 2.1, converting the pixel points in the extracted depth map into three-dimensional point information according to the projection formula of the sensor; Step 2.2, collect the three-dimensional points to obtain a sparse point cloud map of the rich texture area.

4. The method for representing a lightweight map of an unmanned aerial vehicle based on depth information according to claim 1, characterized in that: The specific method of step 3 is: Step 3.1, when the safe space of any spatial point A is to be queried, first query its nearest neighbor distance in the sparse point cloud map of the rich texture area, denoted as L1; Step 3.2, project point A onto the imaging plane of the sensor to obtain the corresponding pixel point B. If the depth of point B has been used to construct a sparse point cloud map of a texture-rich area, directly return the nearest distance L1 and complete step 3; otherwise, perform approximate calculation of the surface element of the environment where point B is located, and then execute steps 3.3-3.5; the specific method of surface element approximation calculation is: Take point B as the center of the initial surface element, take L1 as the surface element radius, and calculate the normal vector of the pixel area, i.e. the surface element, based on the depth information of the pixels within the radius of point B; According to the normal vector information, the center position of the face element and the surface element area are updated, and then the normal vector of the face element area is updated until the change of the center position of the face element is less than the preset value; Step 3.3, based on the geometric information of the facet, the index of the pixels covered by the projection to the facet is established, and the facet replaces these pixels, compressing the redundant pixel depth information into 6 parameters consisting of position and normal direction, so as to realize the lightweight representation and update of the map; Step 3.4, based on the geometric information of the surface element and point A, calculate the shortest distance from point A to the surface element, denoted as L2; Step 3.5, return the smaller value of L1 and L2 as the query result, and determine whether the query result is complete through the query interface; In subsequent query operations, if the pixel where point B is located has been constructed with a surfel, the surfel in this area will not be constructed again.

5. The method for representing a lightweight map of an unmanned aerial vehicle based on depth information as claimed in claim 1, characterized in that: The specific method of step 4 is: when the difference between the sensor pose corresponding to the current depth map frame and the latest sensor pose in the pose queue exceeds a preset value, or when the difference between the timestamp of the current depth map frame and the timestamp of the latest data in each queue exceeds a preset value, the sparse point cloud map and sensor pose corresponding to the current depth map frame are stored in the map queue and the pose queue respectively.

6. The method for representing a lightweight map of an unmanned aerial vehicle based on depth information according to claim 1, characterized in that: The specific method of step 5 is: Step 5.1, obtaining the relative relationship between the sensor poses according to the pose queue, and establishing a global map by using the relative relationship between the sparse point cloud map in the map queue and the sensor poses; Step 5.2, when querying the distance between any point A in the space and all obstacles in each sparse point cloud map, for each sparse point cloud map, the distance between point A and the sensor perception boundary corresponding to the sparse point cloud map is also calculated; Step 5.3: If the minimum obstacle distance obtained by the query is less than the corresponding sensor perception boundary distance, it means that the query result is complete and the query result is obtained; otherwise, it means that the query result is incomplete; In step 5.4, when the query result is incomplete, the relationship between the sensor poses is used to transform the coordinates of point A to the sensor coordinate system corresponding to the previous sensor pose, and the query and calculation process is repeated until the nearest obstacle information of point A is obtained.

7. A method for representing lightweight maps of unmanned aerial vehicles based on depth information, characterized in that: The following steps are involved: Use edge detection or high-frequency information extraction methods on the depth map to separate the pixels in the texture-rich area; Using the camera projection relationship, the depth information of the extracted pixels is restored to a three-dimensional point cloud to form a sparse point cloud map of the texture-rich area; The remaining pixels in the depth map generally have relatively flat depth variations, which are used in the nearest neighbor distance query process to process them, generate sparse surfaces and update the map; The relative relationship between a series of sparse maps and the camera pose of each frame is stored, and a complete query interface is provided for the map to achieve sparse map representation of dense environments.

8. The method for representing a lightweight map of an unmanned aerial vehicle based on depth information as claimed in claim 7, characterized in that: The depth information of the extracted pixels is restored to a 3D point cloud to form a sparse point cloud map of the texture-rich area, including: According to the camera projection formula, the pixels extracted from the depth map are converted into three-dimensional point information; The obtained point set is processed to obtain a sparse point cloud map of the texture-rich area; When used in the nearest neighbor distance query process, it is processed to generate a sparse surface and update the map, including: When the safe space of any spatial point A is to be queried, first query its nearest distance in the point set of the rich texture area, denoted as L1; Project point A onto the camera plane to obtain the corresponding pixel point B. If the depth of point B has been used to construct a point set in the rich texture area, directly return its nearest distance; otherwise, perform an approximate calculation of the surface element of the environment where point B is located; Take point B as the center of the initial surface element, take L1 as the surface element radius, and calculate the normal vector of the pixel area, i.e. the surface element, based on the depth information of the pixels within the radius of point B; According to the normal vector information, the center position of the face element and the surface element area are updated, and then the normal vector of the face element area is updated until the change of the center position of the face element is less than the preset value; According to the geometric information of the facet, the index of the pixels covered by the projection is established, and the facet replaces these pixels. The large amount of redundant pixel depth information is compressed into 6 parameters consisting of position and normal direction, realizing the lightweight representation and update of the map. According to the geometric information of the surface element and point A, the shortest distance from point A to the surface element is calculated, which is recorded as L2; Return the smaller value of L1 and L2 as the query result, and use the query interface to determine whether the query result is complete; In subsequent query operations, if the pixel where point B is located has been constructed, the surface element in this area will not be constructed again; The relative relationship between a series of sparse maps and the camera poses of each frame is stored, and a complete query interface is provided for the map, including: The sparse map of each frame and the relative relationship between the camera poses of each frame are stored to build a global map; When querying the distance between any point A in the space and an obstacle in the map, the distance to the sensor perception boundary of the frame from the query point is calculated at the same time; If the query result of the nearest obstacle distance is less than the distance to the boundary, it means that the query result is complete and the query result is obtained; otherwise, it means that the query result is incomplete; When the query result is incomplete, the position relationship between cameras is used to transform the coordinates of point A to the coordinate system of the previous frame camera, and the query and calculation process is repeated until the nearest obstacle information of point A is obtained.

Images