Three-dimensional environment modeling method and device, storage medium and program product

Abstract

The invention provides a three-dimensional environment modeling method and device, a storage medium and a program product, and relates to the technical field of three-dimensional environment modeling. The method comprises the following steps: acquiring three-dimensional point cloud data of a flight environment of an unmanned aerial vehicle; based on the three-dimensional point cloud data, region division is carried out on the flight environment to obtain three-dimensional point cloud data corresponding to each region in a plurality of regions, the plurality of regions comprise a first region and a second region, the first region comprises a point cloud sparse region and a space structure complex region, and the second region comprises a point cloud dense region and a space structure simple region; creating an octree map corresponding to the first area according to the three-dimensional point cloud data corresponding to the first area; creating a grid map corresponding to the second area according to the three-dimensional point cloud data corresponding to the second area; and splicing the octree map and the grid map to obtain a three-dimensional environment model corresponding to the flight environment. According to the invention, the three-dimensional environment model can be efficiently and accurately established, and the autonomous navigation and efficient inspection effects of the unmanned aerial vehicle are ensured.

Description

Technical Field

[0001] This application relates to the technical field of three-dimensional environment modeling, and in particular, to a three-dimensional environment modeling method, apparatus, storage medium, and program product. Background Art

[0002] In an unmanned aerial vehicle (UAV) inspection system, three-dimensional environment modeling is one of the key technologies for the UAV to achieve autonomous navigation and efficient inspection tasks.

[0003] When implementing three-dimensional environment modeling in related technologies, there is a contradiction between resolution and storage efficiency. If high-precision modeling of the three-dimensional environment is performed, the three-dimensional environment model will occupy a large amount of storage space, and at the same time, it requires high computing resources. It is difficult for lightweight UAVs with limited payloads to carry storage devices and computing devices that meet the requirements; conversely, if the modeling accuracy is too low, it will lead to the loss of environmental information, affecting the navigation accuracy and safety of the UAV.

[0004] Therefore, there is an urgent need for a three-dimensional environment modeling solution that can both ensure meeting the navigation accuracy requirements and take into account storage efficiency and computing resources to ensure that the UAV can achieve autonomous navigation and efficient inspection. Summary of the Invention

[0005] Embodiments of this application provide a three-dimensional environment modeling method, apparatus, storage medium, and program product, aiming to establish a three-dimensional environment model that can both ensure the navigation accuracy requirements of the UAV and save storage resources and computing resources, and ensure the autonomous navigation and efficient inspection effects of the UAV.

[0006] In a first aspect, embodiments of this application provide a three-dimensional environment modeling method, including:

[0007] Obtain three-dimensional point cloud data of the flight environment of the UAV;

[0008] Based on the three-dimensional point cloud data, divide the flight environment into regions to obtain the three-dimensional point cloud data corresponding to each region in multiple regions. The multiple regions include a first region and a second region. The first region includes a point cloud sparse region and a complex spatial structure region, and the second region includes a point cloud dense region and a simple spatial structure region;

[0009] Create an octree map corresponding to the first region according to the three-dimensional point cloud data corresponding to the first region;

[0010] Create a grid map corresponding to the second region according to the three-dimensional point cloud data corresponding to the second region;

[0011] Stitch the octree map and the grid map to obtain a three-dimensional environment model corresponding to the flight environment.

[0012] In a possible implementation, creating the octree map corresponding to the first region based on the three-dimensional point cloud data corresponding to the first region includes:

[0013] Set multiple segmentation points within the first region, and use the segmentation points to perform axial three-dimensional space division on the first region to obtain multiple branch nodes;

[0014] Determine the number of point cloud data included in the branch nodes;

[0015] Among the multiple branch nodes, if the difference in the number of point cloud data included in adjacent branch nodes is less than a preset difference threshold, then merge the adjacent branch nodes to form a new branch node;

[0016] If there is a target branch node with a density of point cloud data greater than a preset density threshold, then divide the target branch node to form multiple new branch nodes;

[0017] For each branch node, determine whether the three-dimensional point cloud data corresponding to the branch node represents multiple objects;

[0018] Divide the branch node representing multiple objects to obtain leaf nodes corresponding to the branch node, and the three-dimensional point cloud data corresponding to the leaf nodes represents a single object;

[0019] Encode the branch nodes and leaf nodes to determine the position information of the objects in the first region, and form an octree map corresponding to the first region.

[0020] In a possible implementation, determining the number of point cloud data included in the branch nodes includes:

[0021] Eliminate the noise points deviating from the main body point cloud in the three-dimensional point cloud data corresponding to the first region through statistical filtering to obtain the first filtered data;

[0022] Use radius filtering to eliminate the points with a correlation less than the correlation threshold with the main body point cloud in the first filtered data to obtain the second filtered data;

[0023] Remove the duplicate points in the second filtered data through voxel filtering to obtain the third filtered data;

[0024] Based on the third filtered data, determine the number of point cloud data included in the branch nodes.

[0025] In a possible implementation, before dividing the branch node representing multiple objects, it further includes:

[0026] Determine that the number of point cloud data in the branch node representing multiple objects is greater than or equal to the number threshold.

[0027] In a possible implementation, it further includes:

[0028] Overlay the octree map with a previously created octree map to obtain an updated octree map;

[0029] Store the updated octree map in a storage space.

[0030] In a possible implementation, the creating the grid map corresponding to the second area according to the three-dimensional point cloud data corresponding to the second area includes:

[0031] Divide the three-dimensional space corresponding to the second area at intervals according to a preset grid map resolution to obtain a three-dimensional grid space composed of multiple grid cells;

[0032] Project the three-dimensional point cloud data corresponding to the second area in the three-dimensional grid space, and mark the occupancy status of each grid cell according to the projection result, where the occupancy status indicates whether there is an object in the grid cell;

[0033] Set the occupancy status of the grid cells around the grid cells with objects to having objects according to a preset expansion coefficient to obtain the grid map corresponding to the second area.

[0034] In a possible implementation, the projecting the three-dimensional point cloud data corresponding to the second area in the three-dimensional grid space includes:

[0035] Obtain the height value of the three-dimensional point cloud data corresponding to the second area;

[0036] Filter out the three-dimensional point cloud data with a height value less than a height threshold to obtain fourth filtered data;

[0037] Project the fourth filtered data in the three-dimensional grid space.

[0038] In a possible implementation, the splicing the octree map and the grid map to obtain the three-dimensional environment model corresponding to the flight environment includes:

[0039] Align the octree map and the grid map spatially to ensure that the map data at the same position is consistent;

[0040] Perform map splicing on the aligned maps to obtain the three-dimensional environment model corresponding to the flight environment.

[0041] In a second aspect, an embodiment of the present application provides a three-dimensional environment modeling device, including:

[0042] An acquisition module that acquires three-dimensional point cloud data of the flight environment of a drone;

[0043] A region division module divides the flight environment based on the 3D point cloud data to obtain the 3D point cloud data corresponding to each of multiple regions. The multiple regions include a first region and a second region. The first region includes a sparse point cloud region and a region with a complex spatial structure. The second region includes a dense point cloud region and a region with a simple spatial structure;

[0044] An octree map creation module creates an octree map corresponding to the first region according to the 3D point cloud data corresponding to the first region;

[0045] A grid map creation module creates a grid map corresponding to the second region according to the 3D point cloud data corresponding to the second region;

[0046] A splicing module splices the octree map and the grid map to obtain a 3D environment model corresponding to the flight environment.

[0047] In a third aspect, an embodiment of the present application provides a 3D environment modeling device, including: a memory, a processor;

[0048] The memory stores computer execution instructions;

[0049] The processor executes the computer execution instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementation manners of the first aspect.

[0050] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementation manners of the first aspect.

[0051] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the above first aspect and / or various possible implementation manners of the first aspect.

[0052] The three-dimensional environment modeling method, device, storage medium, and program product provided by the embodiments of the present application collect three-dimensional point cloud data of the UAV flight environment, divide the flight environment into regions, and use different modeling methods for different regions. In regions with sparse point clouds and complex spatial structures, an octree map is created. In regions with dense point clouds and simple spatial structures, a grid map is used for modeling. The octree maps and grid maps corresponding to each region are stitched together to obtain a three-dimensional environment model corresponding to the UAV flight environment. That is, it utilizes the advantages of the octree map, which has high storage efficiency in sparse environments and can efficiently represent complex structures in three-dimensional space, and combines the characteristics of the grid map, which has high storage efficiency and simple calculations in dense environments, to establish a three-dimensional environment model that can not only meet the UAV navigation accuracy requirements but also save storage resources and computing resources, ensuring the UAV's autonomous navigation and efficient inspection effects. Description of the Drawings

[0053] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0054] Figure 1 Flow schematic of the three-dimensional environment modeling method provided by the embodiments of the present application Figure 1 ;

[0055] Figure 2 Flow schematic of the three-dimensional environment modeling method provided by the embodiments of the present application Figure 2 ;

[0056] Figure 3 Schematic diagram of the three-dimensional space voxelization process provided by the embodiments of the present application;

[0057] Figure 4 Schematic diagram of the octree structure provided by the embodiments of the present application;

[0058] Figure 5 Flow schematic of the three-dimensional environment modeling method provided by the embodiments of the present application Figure 3 ;

[0059] Figure 6 Schematic diagram of the grid map provided by the embodiments of the present application;

[0060] Figure 7 Schematic diagram of the structure of the three-dimensional environment modeling device provided by the present application;

[0061] Figure 8 Schematic diagram of the structure of the three-dimensional environment modeling device provided by the present application.

[0062] Through the above-mentioned drawings, specific embodiments of the present application have been shown, and will be described in more detail hereinafter. These drawings and the written description are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by reference to specific embodiments. Detailed Description of the Embodiments

[0063] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0064] Currently, drones are widely used in inspection tasks in various scenarios. For example, in the inspection task of a substation, using a drone for inspection is fast and has a wide field of view, which can greatly improve the efficiency of substation inspection. In the existing drone inspection system, 3D environmental modeling is one of the key technologies for drones to achieve autonomous navigation and efficient inspection tasks.

[0065] The related technologies face multiple challenges in realizing 3D environmental modeling. First, although the traditional grid map has a simple structure and is easy to implement, there is an inherent contradiction between its resolution and storage efficiency when representing a complex 3D environment. The higher the resolution of the grid map, the larger the storage space required, and at the same time, the computational complexity of path planning also increases. On the contrary, if the resolution is too low, it will lead to the loss of environmental information, affecting the navigation accuracy and safety of the drone. Second, although the 3D modeling method based on point cloud can provide rich environmental information, the point cloud data is usually large in scale, not only occupying a large amount of storage space, but also posing extremely high requirements for computing resources when directly applied to drone navigation. Especially for small multi-rotor drones, due to their limited payload, it is difficult to carry high-performance computing devices, so it is particularly difficult to directly process point cloud data. The octree map has a high storage efficiency when dealing with sparse environments, but may lead to an increase in storage overhead in dense environments.

[0066] In summary, there is a contradiction between the modeling accuracy and storage efficiency in the above 3D environmental modeling methods, and a 3D environmental modeling method that can not only meet the requirements of drone navigation accuracy but also reasonably utilize storage resources cannot be provided to ensure that the drone realizes autonomous navigation and efficient inspection.

[0067] In response to this, the present application provides a three-dimensional environment modeling method. By collecting three-dimensional point cloud data of the UAV flight environment and dividing the three-dimensional point cloud data into regions, in regions with sparse point clouds and complex spatial structures, taking full advantage of the advantages of the octree map, which has a high storage efficiency in sparse environments and can efficiently represent complex structures in three-dimensional space, the octree map is used for modeling. In regions with dense point clouds and simple spatial structures, taking full advantage of the characteristics of the grid map, which has a high storage efficiency and simple calculation in dense environments, the grid map is used for modeling. Finally, the octree maps and grid maps corresponding to each region are stitched together to obtain a three-dimensional environment model corresponding to the UAV flight environment, achieving the establishment of a three-dimensional environment model that can not only meet the UAV navigation accuracy requirements but also save storage resources and computing resources, ensuring the UAV autonomous navigation and efficient inspection effects.

[0068] The following uses specific embodiments to elaborate in detail on the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems. These several specific embodiments below can be combined with each other, and for the same or similar concepts or processes, they may not be repeated in some embodiments. The following will describe the embodiments of the present application in conjunction with the accompanying drawings.

[0069] Figure 1 Flow schematic of the three-dimensional environment modeling method provided by the present application Figure 1 , as Figure 1 shown, the method includes:

[0070] S101. Obtain three-dimensional point cloud data of the UAV flight environment.

[0071] Specifically, sensors such as lidar and cameras are used to obtain three-dimensional point cloud data of the UAV flight environment. In addition, the pose information of the UAV, including position information and attitude information, can also be obtained through sensors installed on the UAV as auxiliary information for three-dimensional environment modeling. The UAV flight environment is the flight area corresponding to the UAV's inspection task, such as a substation.

[0072] In some embodiments, key frames related to the UAV flight environment can be obtained from the front-end tracking thread of the ORB-SLAM2 platform, and the key frames are converted into three-dimensional point cloud data.

[0073] S102. Based on the three-dimensional point cloud data, divide the flight environment into regions to obtain three-dimensional point cloud data corresponding to each region in multiple regions. The multiple regions include a first region and a second region. The first region includes a point cloud sparse region and a complex spatial structure region, and the second region includes a point cloud dense region and a simple spatial structure region.

[0074] It can be understood that for regions with sparse point clouds, a larger modeling scale can be used, which can reduce the waste of storage resources; for regions with complex spatial structures, a smaller modeling scale is required to ensure that as much spatial information as possible is provided to ensure the safety of the automatic cruise of the drone. Therefore, for regions with sparse point clouds or complex spatial structures, a modeling method with flexible scale change is needed.

[0075] For regions with dense point clouds, the information carried by the point cloud data may be redundant. Therefore, the spatial information represented by the point cloud data should be extracted as much as possible to reduce the waste of storage resources; for regions with simple spatial structures, from the perspective of computational efficiency and storage efficiency, a simple method should be used for representation. Therefore, for regions with dense point clouds and simple spatial structures, a modeling method that can efficiently extract the information of point cloud data and is simple and intuitive is needed.

[0076] The embodiments of the present application reasonably divide the three-dimensional point cloud data so as to adopt different modeling methods based on the characteristics of the point cloud data in different regions subsequently.

[0077] S103. Create an octree map corresponding to the first region according to the three-dimensional point cloud data corresponding to the first region.

[0078] The octree map is based on the octree data structure and is an efficient three-dimensional space representation method that can achieve multi-scale modeling. The octree map recursively divides the three-dimensional space into smaller voxels and stores them in a tree structure. There are three types of nodes in the octree map: root node, branch node, and leaf node. The root node represents the entire three-dimensional space, usually a cube, which defines the initial spatial boundary. The root node recursively divides the space into several sub-cubes, and each sub-cube corresponds to a branch node. This recursive process continues until certain conditions are met, such as reaching the maximum depth or spatial resolution. The nodes at the end of the octree map are called leaf nodes.

[0079] Using the octree map to model the first region can represent it with larger voxels or less node information in the octree map in regions with sparse point clouds. At the same time, for regions with complex structures, smaller voxels can be used, and as much node information as possible can be used to accurately represent it, taking into account both storage efficiency and accuracy.

[0080] S104. Create a grid map corresponding to the second region according to the three-dimensional point cloud data corresponding to the second region.

[0081] Among them, the principle of the grid map is mainly to equally divide the environmental space into several grids, and judge whether there are obstacles in each grid through specific methods, and then draw the map, which is an efficient and intuitive modeling method.

[0082] Modeling the second area using a grid map can fully extract the information carried by the point cloud in the dense area of the point cloud and represent it in the simplest and most intuitive way. For areas with simple structures, using a grid map can achieve high computational efficiency.

[0083] S105. Stitch the octree map and the grid map to obtain the three-dimensional environment model corresponding to the flight environment.

[0084] The three-dimensional environment modeling method provided by the embodiments of the present application collects three-dimensional point cloud data corresponding to the flight environment of the unmanned aerial vehicle, divides the flight environment according to the regional characteristics in the flight environment, and uses different modeling methods for different regions. For sparse point cloud regions and regions with complex spatial structures, an octree map is used for modeling. For dense point cloud regions and regions with simple spatial structures, a grid map is used for modeling. Finally, the octree map and the grid map are stitched to form a hybrid map. Through the above technical means, the spatial efficiency of the octree map and the intuitiveness of the grid map are combined, which can provide an accurate environmental representation in three-dimensional space, maintain a low computational complexity at the same time, improve the flexibility of three-dimensional space modeling, and optimize the data storage and processing efficiency.

[0085] In a possible implementation manner, creating the octree map corresponding to the first area according to the three-dimensional point cloud data corresponding to the first area includes the following steps:

[0086] S201. Set a plurality of segmentation points in the first area, and use the segmentation points to perform axial three-dimensional space division on the first area to obtain a plurality of branch nodes.

[0087] The basic idea of the octree map is to voxelize the three-dimensional space, recursively divide the three-dimensional space into smaller voxels, and store them in a tree structure. The process of voxelizing the first area is as Figure 3 shown. The first area is voxelized into a cubic area as the root node. Segmentation points are set on this cube for axial three-dimensional space division, which is equivalent to dividing the cube obtained by voxelizing the first area into smaller cubes as branch nodes. Further, according to certain characteristics, the cube corresponding to the branch node is divided to obtain the leaf nodes at the end of the octree. This division process can be represented by the Figure 4 octree structure shown.

[0088] S202. Determine the number of point cloud data contained in the branch nodes.

[0089] S203. Among the multiple branch nodes, if the difference in the number of point cloud data contained in adjacent branch nodes is less than a preset difference threshold, then merge the adjacent branch nodes to form a new branch node.

[0090] It can be understood that for branch nodes with approximately the same number of point clouds, their point cloud density is the same and can be represented at the same modeling scale. Therefore, before dividing the leaf nodes, first merge those with similar density to reduce the number of branch nodes.

[0091] S204. If there is a target branch node whose density of the contained point cloud data is greater than a preset density threshold, then divide the target branch node to form multiple new branch nodes.

[0092] Among them, the density of the point cloud data refers to the ratio of the number of point cloud data in the branch node to the volume of the three-dimensional space corresponding to the branch node. If the density is greater than the preset density threshold, it means that the structure of this area is complex and the spatial information carried by the point cloud data is more. It should be represented at a more precise scale. Therefore, it is necessary to further divide the nodes based on the current branch node to ensure the accuracy of spatial representation.

[0093] So far, the construction of the adaptive octree according to the point cloud data density has been realized. Next, based on the idea of the minimum bounding box, on the basis of the branch nodes obtained by the construction of the adaptive octree, layer-by-layer division is carried out to obtain leaf nodes.

[0094] S205. For each branch node, determine whether the three-dimensional point cloud data corresponding to the branch node represents multiple objects.

[0095] Among them, an object refers to devices, buildings, obstacles, etc. existing in the first area, and multiple objects can be distinguished according to information such as contours, point cloud aggregation degree, and point cloud density.

[0096] S206. Divide the branch node representing multiple objects to obtain the leaf nodes of the corresponding branch node, and the three-dimensional point cloud data corresponding to the leaf nodes represents a single object.

[0097] In some embodiments, based on the idea of the minimum bounding box, first obtain the maximum coordinates and minimum coordinates of the smallest point cloud data corresponding to the object in each dimension, and form a cube surrounding the object based on the maximum coordinates and minimum coordinates in each dimension, that is, the minimum bounding box. A leaf node stores the three-dimensional point cloud data that can represent a minimum bounding box.

[0098] To avoid problems such as waste of storage resources and difficulty in retrieval caused by data redundancy, when storing the point cloud data corresponding to the leaf nodes of the octree, each leaf node retains a preset number of point cloud data. Exemplarily, each leaf node retains 1000 point cloud data points.

[0099] S207. Encode the branch nodes and leaf nodes to determine the position information of the objects in the first region, and form an octree map corresponding to the first region.

[0100] In one implementation, the memory allocation is automatically adjusted according to the amount of octree map data updated in real time, further optimizing the speed of map generation and update, ensuring efficient use of resources, and overall improving the generation efficiency of the octree map and the practicality of the map.

[0101] The three-dimensional environment modeling method provided by the embodiments of the present application adopts an octree construction method that combines adaptive octree construction and minimum bounding box octree construction. First, according to the density of the point cloud data, the branch nodes of the octree are adaptively divided. On the basis of the branch nodes, the idea of the minimum bounding box is adopted to divide the leaf nodes that only contain a single object. This method can intelligently partition according to the density of the point cloud data, adopt refined partitioning for the dense area to ensure high spatial resolution and capture fine features; while for the sparse area, large voxels are used for partitioning to reduce memory occupancy and accelerate modeling. By calculating the minimum bounding box of the point cloud data and gradually subdividing it until the leaf node conditions are met, the refined construction of the octree model is realized, and unnecessary spatial partitioning is avoided.

[0102] In one implementation, a tree-based representation of the three-dimensional environment space is used based on the octree integration framework, providing maximum flexibility in terms of mapping area and resolution, while allowing effective probability updates for the occupied space and free space, and keeping the memory consumption to a minimum. The occupied space is obtained from the endpoints of a distance sensor (such as a laser rangefinder), while the free space corresponds to the observation area between the sensor and the endpoints. A compression method is introduced to reduce the memory requirements by locally combining coherent mapping volumes in the mapped free areas and occupied spaces, ensuring the compactness of the resulting model. An extension of the mapping method is developed to construct an octree map hierarchy using the hierarchical dependencies in the environment. This extension maintains a set of sub-maps in the tree structure, where each node represents a subspace of the environment.

[0103] In one possible implementation, the determining the number of point cloud data included in the branch node includes:

[0104] Remove the noise points deviating from the main point cloud in the three-dimensional point cloud data corresponding to the first region through statistical filtering to obtain first filtered data; remove the points with a correlation less than the correlation threshold with the main point cloud in the first filtered data using radius filtering to obtain second filtered data; remove the duplicate points in the second filtered data through voxel filtering to obtain third filtered data; determine the number of point cloud data included in the branch nodes based on the third filtered data.

[0105] Among them, in the statistical filtering process, first calculate the average distance from each point to its neighboring points, and set a suitable threshold range according to these distance distributions. Then, identify all points exceeding this threshold range as noise points deviating from the main point cloud and remove them. The point cloud data after statistical filtering will be smoother and neater, which is beneficial to subsequent analysis and applications.

[0106] Radius filtering is a point cloud data filtering method based on the local neighborhood. Its core idea is that for each point in the three-dimensional point cloud data, the algorithm will check the positional relationship with other points within its neighborhood, that is, a spherical region centered on this point with a set radius as the range. If the distance between the points in the neighborhood and the center point is less than or equal to the set radius threshold, these points are regarded as valid points; conversely, if the distance is greater than the radius threshold, these points are regarded as points with a correlation less than the correlation threshold with the main point cloud and will be removed.

[0107] Voxel filtering includes the following steps: First, divide the point cloud data space into multiple cubic voxel grids according to the voxel size set by the user; traverse the input three-dimensional point cloud data and assign each point to the corresponding voxel grid; for each voxel, calculate a representative point based on the three-dimensional point cloud data in it. A common method is to calculate the average value (centroid) of all points within the voxel, and use this average value as the coordinate of the center point of the voxel. In this way, each voxel is represented by a center point for the three-dimensional point cloud data inside it; gather the representative points in each voxel to form the downsampled three-dimensional point cloud data. These representative points retain the main features and structure of the original point cloud, but the quantity is greatly reduced, thereby reducing the computational amount of subsequent processing. Through voxel filtering, the density of the three-dimensional point cloud data can be effectively reduced, the computational amount can be reduced, and at the same time, the main features and structure of the point cloud are retained.

[0108] The three-dimensional environment modeling method provided by the embodiments of this application performs filtering processing on the data before building the octree map to obtain better visualization effects and further compress the number of point clouds, making the octree map more accurate and also reducing the computational amount required to build the octree map.

[0109] In a possible implementation manner, before splitting the branch nodes representing multiple objects, it further includes:

[0110] Determine that the number of point cloud data within a branch node representing multiple objects is greater than or equal to a threshold number.

[0111] It can be understood that when the number of point cloud data within the branch node is less than the threshold number, it indicates that the branch node has reached the preset optimal resolution, and it can be used as the leaf node at the end of the octree, stopping further division of the branch node to prevent invalid segmentation.

[0112] The 3D environment modeling method provided by the embodiments of the present application, by setting a fixed threshold number of point cloud data, prevents invalid segmentation and waste of storage resources while ensuring the resolution of the octree map.

[0113] In a possible implementation manner, the 3D environment modeling method further includes:

[0114] Overlay the octree map with a previously created octree map to obtain an updated octree map; store the updated octree map in a storage space.

[0115] Each time after creating an octree map through steps S201 to S207 in the above embodiments, first read the previously created octree map from the storage space, overlay the new octree map with the previously created octree map, automatically adjust the structure of the tree according to the position of the newly added node, place the newly added points in the correct position, obtain an updated octree map, and store the updated octree map in the storage space. Repeat this process over time until an ideal octree map is obtained.

[0116] The 3D environment modeling method provided by the embodiments of the present application, based on the idea of real-time octree conversion, overlays the new octree map obtained from point cloud data with a previously created octree map, continuously updates the node information in the octree map, and improves the accuracy of the octree map.

[0117] In a possible implementation manner, creating the grid map corresponding to the second region according to the three-dimensional point cloud data corresponding to the second region includes the following steps:

[0118] S501. According to a preset grid map resolution, divide the three-dimensional space corresponding to the second region at intervals to obtain a three-dimensional grid space composed of multiple grid cells.

[0119] Specifically, according to the value range of the valid point cloud data, construct of the three-dimensional grid space; 、 、 The values of are obtained according to the following formula

[0120]

[0121] In the formula, x, y, and z are the coordinates of the point cloud in three-dimensional space; N represents the number of points contained in the valid point cloud; R is the grid map resolution; ceil(·) represents the ceiling function.

[0122] S502. Project the three-dimensional point cloud data corresponding to the second region into the three-dimensional grid space, and mark the occupancy status of each grid cell according to the projection result, where the occupancy status indicates whether there is an object in the grid cell.

[0123] It can be understood that when projecting the three-dimensional point cloud data into the three-dimensional grid space, the grid cells where the projection falls can be considered to have objects and should be marked as occupied. Set the occupancy status p. When the grid cell is occupied, set p to 1. That is, the occupancy status of the grid cell can be represented by the following formula

[0124]

[0125] In the formula, is the grid coordinate obtained by projecting the three-dimensional point cloud data from the original spatial coordinates , into the three-dimensional grid space. Specifically, is obtained by the following formula

[0126]

[0127] S503. According to the preset expansion coefficient, set the occupancy status of the grid cells around the grid cells with objects as having objects to obtain the grid map corresponding to the second region.

[0128] Setting the expansion coefficient is to set a safety margin between the passable area of the UAV and the obstacles to ensure that the UAV maintains a safe distance from the obstacles during flight. Considering the size of the UAV itself and the safety distance, expand the obstacles in the environment, such as substation equipment, in the grid map. Introduce the expansion parameter E to represent the number of circles of obstacle expansion. Specifically, the expansion coefficient is obtained by the following formula

[0129]

[0130] In the formula, r represents the sum of the radius of the UAV and the minimum safety distance, and R is the grid map resolution

[0131] The schematic diagram of the grid map obtained after the expansion process is as Figure 6 shown, where the black cells are the occupied cells determined according to the point cloud, and the gray cells are the additional occupied cells due to obstacle expansion (E = 1).

[0132] The 3D environment modeling method provided by the embodiments of the present application adopts a grid map modeling method for areas with dense point clouds and simple spatial structures. First, the space is divided into a 3D grid space, then the point cloud data is projected into the grid space, and the occupancy status of the projected grids is marked, which can efficiently extract the information contained in the dense point cloud data and visually represent the distribution of obstacles in the map. In addition, an expansion coefficient is set to fully consider the radius and safety distance of the UAV, and the grid cells occupied by the obstacles in the map are expanded to ensure the safety of the UAV during flight.

[0133] In a possible implementation manner, the projecting the 3D point cloud data corresponding to the second region into the 3D grid space includes:

[0134] Obtaining the height values of the 3D point cloud data corresponding to the second region; filtering out the 3D point cloud data with height values less than the height threshold to obtain the fourth filtered data; projecting the fourth filtered data into the 3D grid space.

[0135] The ground point cloud data in the 3D point cloud data often occupies a large amount of space, and its distribution is relatively uniform and lacks features. After filtering out the ground point cloud, it is possible to focus more on the target objects of interest, thereby improving the accuracy and detail performance of the 3D model.

[0136] It can be understood that the 3D point cloud data with height values less than the height threshold in the 3D point cloud data can be regarded as ground point cloud data, and the ground point cloud data is filtered out.

[0137] The 3D environment modeling method provided by the embodiments of the present application uses the fourth filtered data after filtering out the ground point cloud data for grid map modeling, which can improve the establishment efficiency of the grid map and the accuracy of the grid map.

[0138] In a possible implementation manner, the splicing the octree map and the grid map to obtain the 3D environment model corresponding to the flight environment includes:

[0139] Performing spatial alignment on the octree map and the grid map to ensure that the map data at the same position is consistent; performing map splicing on the aligned maps to obtain the 3D environment model corresponding to the flight environment.

[0140] For the edge positions of the grid map and the octree map, there may be the same object existing in both the grid map and the octree map. According to the point cloud data characteristics, contours, and other characteristics of the object, the edges of the maps are spatially aligned to integrate the information of the grid map and the octree map. The aligned maps can be stitched together to obtain a three-dimensional environment model corresponding to the UAV flight environment. This model combines the advantages of the grid map and the octree map and can efficiently provide navigation information for the UAV.

[0141] The three-dimensional environment modeling method provided by the embodiments of the present application fully considers the consistency of the grid map and the octree map at the map edges, spatially aligns the octree map and the grid map to ensure the consistency of data at the same position. The aligned maps are stitched together to obtain a three-dimensional environment model corresponding to the UAV flight environment, which combines the spatial efficiency of the octree map and the intuitiveness of the grid map, can provide an accurate environmental representation in three-dimensional space, while maintaining a low computational complexity, and can provide higher adaptability and flexibility under different environments and task requirements. By selecting suitable map types in different regions, the data storage and processing efficiency can be optimized.

[0142] Figure 7 It is a schematic structural diagram of a three-dimensional environment modeling device provided by the present application, as Figure 7 shown, the three-dimensional environment modeling device 70 provided in this embodiment includes:

[0143] An acquisition module 701, which acquires three-dimensional point cloud data of the flight environment of the UAV.

[0144] A region division module 702, based on the three-dimensional point cloud data, divides the flight environment into regions to obtain the three-dimensional point cloud data corresponding to each region in multiple regions. The multiple regions include a first region and a second region. The first region includes a region with sparse point clouds and a region with complex spatial structures, and the second region includes a region with dense point clouds and a region with simple spatial structures.

[0145] An octree map creation module 703, according to the three-dimensional point cloud data corresponding to the first region, creates an octree map corresponding to the first region.

[0146] A grid map creation module 704, according to the three-dimensional point cloud data corresponding to the second region, creates a grid map corresponding to the second region.

[0147] A stitching module 705, which stitches the octree map and the grid map to obtain a three-dimensional environment model corresponding to the flight environment.

[0148] In a possible implementation manner, the octree map creation module 703 is specifically used for:

[0149] Set multiple segmentation points within the first region, and use the segmentation points to divide the first region in the axial three-dimensional space to obtain multiple branch nodes.

[0150] Determine the number of point cloud data contained in the branch nodes.

[0151] Among the multiple branch nodes, if the difference in the number of point cloud data contained in adjacent branch nodes is less than a preset difference threshold, then merge the adjacent branch nodes to form a new branch node.

[0152] If there is a target branch node whose density of the contained point cloud data is greater than a preset density threshold, then divide the target branch node to form multiple new branch nodes.

[0153] For each branch node, determine whether the three-dimensional point cloud data corresponding to the branch node represents multiple objects.

[0154] Divide the branch node representing multiple objects to obtain the leaf nodes of the corresponding branch node, and the three-dimensional point cloud data corresponding to the leaf nodes represents a single object.

[0155] Encode the branch nodes and leaf nodes to determine the position information of the objects in the first region, and form an octree map corresponding to the first region.

[0156] In a possible implementation manner, the octree map creation module 703 is further configured to:

[0157] Remove the noise points deviating from the main body point cloud in the three-dimensional point cloud data corresponding to the first region through statistical filtering to obtain the first filtered data.

[0158] Use radius filtering to remove the points in the first filtered data with a correlation less than the correlation threshold with the main body point cloud to obtain the second filtered data.

[0159] Remove the duplicate points in the second filtered data through voxel filtering to obtain the third filtered data;

[0160] Based on the third filtered data, determine the number of point cloud data contained in the branch nodes.

[0161] In a possible implementation manner, the octree map creation module 703 is further configured to:

[0162] Determine that the number of point cloud data in the branch node representing multiple objects is greater than or equal to the number threshold.

[0163] In a possible implementation manner, the three-dimensional environment modeling device 70 further includes an octree overlay module 706, configured to:

[0164] Overlay the octree map with the previously created octree map to obtain an updated octree map.

[0165] Store the updated octree map in the storage space.

[0166] In a possible implementation, the grid map creation module 704 is further configured to:

[0167] According to the preset grid map resolution, divide the three-dimensional space corresponding to the second region at intervals to obtain a three-dimensional grid space composed of multiple grid cells.

[0168] Project the three-dimensional point cloud data corresponding to the second region into the three-dimensional grid space, and mark the occupancy status of each grid cell according to the projection result, where the occupancy status indicates whether there is an object in the grid cell.

[0169] According to the preset expansion coefficient, set the occupancy status of the grid cells around the grid cells with objects as having objects to obtain the grid map corresponding to the second region.

[0170] In a possible implementation, the grid map creation module 704 is further configured to:

[0171] Obtain the height values of the three-dimensional point cloud data corresponding to the second region.

[0172] Filter out the three-dimensional point cloud data with height values less than the height threshold to obtain the fourth filtered data;

[0173] Project the fourth filtered data into the three-dimensional grid space.

[0174] In a possible implementation, the splicing module 705 is specifically configured to:

[0175] Spatially align the octree map and the grid map to ensure that the map data at the same position is consistent.

[0176] Splice the aligned maps to obtain the three-dimensional environment model corresponding to the flight environment.

[0177] The three-dimensional environment modeling device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effects are similar, which will not be elaborated here in this embodiment.

[0178] Figure 8 This is a schematic structural diagram of the three-dimensional environment modeling device provided in this application. As Figure 8As shown in the figure, the electronic device 80 provided in this embodiment includes: at least one processor 801 and a memory 802. Optionally, the device 80 further includes a communication component 803. Among them, the processor 801, the memory 802, and the communication component 803 are connected through a bus 804.

[0179] In the specific implementation process, at least one processor 801 executes the computer-executable instructions stored in the memory 802, so that at least one processor 801 executes the above-mentioned method.

[0180] For the specific implementation process of the processor 801, reference may be made to the above method embodiment, and its implementation principle and technical effect are similar, so they will not be elaborated here in this embodiment.

[0181] In the above embodiment, it should be understood that the processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the invention may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0182] The memory may include a high-speed random access memory (RAM), and may also include a non-volatile memory (NVM), such as at least one disk memory.

[0183] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0184] This application also provides a computer program product, including a computer program, which implements the above-mentioned method when executed by a processor.

[0185] The present application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above method.

[0186] The above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0187] An exemplary readable storage medium is coupled to the processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0188] The division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0189] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0190] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0191] If a function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc., all kinds of media that can store program codes.

[0192] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When this program is executed, it executes the steps including the above method embodiments; and the aforementioned storage medium includes: ROMs, RAMs, magnetic disks, or optical discs, etc., all kinds of media that can store program codes.

[0193] Finally, it should be noted that: After considering the specification and practicing the invention disclosed herein, those skilled in the art will easily think of other implementation manners of the present invention. The present invention is intended to cover any variations, uses, or adaptive changes of the present invention. These variations, uses, or adaptive changes follow the general principles of the present invention and include the common general knowledge or conventional technical means in the technical field of the present invention that are not disclosed in the present invention. It is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A three-dimensional environment modeling method, characterized in that: include: Obtain three-dimensional point cloud data of the UAV's flight environment; Based on the three-dimensional point cloud data, the flight environment is divided into regions to obtain three-dimensional point cloud data corresponding to each of a plurality of regions, wherein the plurality of regions include a first region and a second region, the first region includes a sparse point cloud region and a complex spatial structure region, and the second region includes a dense point cloud region and a simple spatial structure region; Creating an octree map corresponding to the first area according to the three-dimensional point cloud data corresponding to the first area; Creating a grid map corresponding to the second area according to the three-dimensional point cloud data corresponding to the second area; The octree map and the grid map are spliced ​​to obtain a three-dimensional environment model corresponding to the flight environment.

2. The method according to claim 1, characterized in that The step of creating an octree map corresponding to the first area according to the three-dimensional point cloud data corresponding to the first area includes: Setting a plurality of segmentation points in the first region, and dividing the first region into three-dimensional axial space using the segmentation points to obtain a plurality of branch nodes; Determine the number of point cloud data contained in the branch node; Among the plurality of branch nodes, if the difference in the number of point cloud data included in adjacent branch nodes is less than a preset difference threshold, the adjacent branch nodes are merged to form a new branch node; If there is a target branch node including point cloud data with a density greater than a preset density threshold, the target branch node is segmented to form a plurality of new branch nodes; For each branch node, determining whether the three-dimensional point cloud data corresponding to the branch node represents multiple objects; Segmenting the branch nodes representing multiple objects to obtain leaf nodes corresponding to the branch nodes, wherein the three-dimensional point cloud data corresponding to the leaf nodes represents a single object; The branch nodes and the leaf nodes are encoded to determine the location information of the objects in the first area, and to form an octree map corresponding to the first area.

3. The method according to claim 2, characterized in that The step of determining the number of point cloud data included in the branch node includes: By statistical filtering, noise points deviating from the main point cloud in the three-dimensional point cloud data corresponding to the first area are eliminated to obtain first filtered data; Use radius filtering to remove points in the first filtered data whose correlation with the main point cloud is less than a correlation threshold, to obtain second filtered data; Removing duplicate points in the second filtered data by voxel filtering to obtain third filtered data; Based on the third filtered data, the number of point cloud data included in the branch node is determined.

4. The method according to claim 2, characterized in that: Before splitting the branch nodes representing multiple objects, the following steps are also included: It is determined that the number of point cloud data in the branch node representing the multiple objects is greater than or equal to a number threshold.

5. The method according to any one of claims 1 to 4, characterized in that: Also includes: Overlaying the octree map with the previously created octree map to obtain an updated octree map; The updated octree map is stored in a storage space.

6. The method according to any one of claims 1 to 4, characterized in that: The step of creating a grid map corresponding to the second area according to the three-dimensional point cloud data corresponding to the second area includes: According to a preset grid map resolution, the three-dimensional space corresponding to the second area is divided into intervals to obtain a three-dimensional grid space composed of a plurality of grid units; Projecting the three-dimensional point cloud data corresponding to the second area in the three-dimensional grid space, and marking the occupancy state of each of the grid cells according to the projection result, wherein the occupancy state indicates whether there is an object in the grid cell; According to a preset expansion coefficient, the occupancy state of the grid cells around the grid cell where the object exists is set to that the object exists, and a grid map corresponding to the second area is obtained.

7. The method according to claim 6, characterized in that The projecting the three-dimensional point cloud data corresponding to the second area in the three-dimensional grid space includes: Obtaining a height value of the three-dimensional point cloud data corresponding to the second area; Filter out the three-dimensional point cloud data whose height value is less than the height threshold to obtain fourth filtered data; The fourth filtered data is projected in the three-dimensional grid space.

8. The method according to any one of claims 1 to 4, characterized in that: The step of splicing the octree map and the grid map to obtain a three-dimensional environment model corresponding to the flight environment includes: Spatially aligning the octree map and the grid map to ensure that map data at the same location are consistent; The aligned maps are stitched together to obtain a three-dimensional environment model corresponding to the flight environment.

9. A three-dimensional environment modeling device, characterized in that: include: An acquisition module is used to obtain the three-dimensional point cloud data of the UAV's flight environment; A region division module, based on the three-dimensional point cloud data, divides the flight environment into regions to obtain three-dimensional point cloud data corresponding to each of a plurality of regions, wherein the plurality of regions include a first region and a second region, the first region includes a sparse point cloud region and a complex spatial structure region, and the second region includes a dense point cloud region and a simple spatial structure region; An octree map creation module, which creates an octree map corresponding to the first area according to the three-dimensional point cloud data corresponding to the first area; A grid map creation module, which creates a grid map corresponding to the second area according to the three-dimensional point cloud data corresponding to the second area; The splicing module splices the octree map and the grid map to obtain a three-dimensional environment model corresponding to the flight environment.

10. A three-dimensional environment modeling device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 8 when being executed.

Images