Scene completion method and system based on aerial point cloud prior guidance

Abstract

The present application discloses a scene completion method and system based on aerial point cloud prior guidance, the method comprising: obtaining prior point cloud data, processing the prior point cloud data to obtain a traversable node graph and the boundary of the area to be supplemented; obtaining a global planning path according to the traversable node graph and the boundary of the area to be supplemented; performing point cloud supplementary sampling according to the global planning path to obtain supplementary sampling data, fusing the supplementary sampling data with the prior point cloud data to obtain target point cloud data. The present application improves data acquisition efficiency while ensuring acquisition integrity.

Description

Technical Field

[0001] The present application relates to the field of data acquisition technology, and in particular to a scene completion method and system based on aerial point cloud prior guidance. Background Art

[0002] To solve the problems of missing ground data and incomplete data collection when aerial drones and manned aircraft obtain point cloud information through fixed routes. At present, data supplementation is generally required through manual collection. Although the existing methods based on manual supplementation have certain advantages in flexibility and operability, they have a series of problems such as low collection efficiency, poor data quality and consistency, and high cost of manual supplementation, which limit its application potential in high-precision data supplementation in large-scale urban scenes. The autonomous exploration system based on ground vehicles or drones can achieve accurate positioning and path planning through real-time data fusion, and can provide high-precision, high-efficiency and automated three-dimensional environmental data acquisition and processing. It can stably and efficiently explore indoor and outdoor environments autonomously, reduce dependence on manual operations, and improve the degree of automation of data collection and processing. However, in the process of autonomous point cloud supplementation, the existing autonomous exploration system has many problems such as low exploration efficiency, incomplete coverage, incomplete environmental understanding and perception, and insufficient data integration capabilities due to the lack of prior guidance. The above challenges limit the effectiveness and reliability of autonomous exploration systems in large-scale three-dimensional urban modeling.

[0003] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0004] The main purpose of this application is to provide a scene completion method and system based on aerial point cloud prior guidance, aiming to solve the problem in the prior art that when drones obtain point cloud information, ground area data collection is incomplete, and manual collection is required to supplement point cloud data, resulting in low data collection efficiency.

[0005] A first aspect of an embodiment of the present application provides a scene completion method based on aerial point cloud prior guidance, and the scene completion method based on aerial point cloud prior guidance comprises the following steps: acquiring prior point cloud data, processing the prior point cloud data to obtain a traversable node graph and a boundary of an area to be supplemented; obtaining a global planned path based on the traversable node graph and the boundary of the area to be supplemented; performing point cloud supplementary sampling according to the global planned path to obtain supplementary sampling data, and fusing the supplementary sampling data with the prior point cloud data to obtain target point cloud data.

[0006] Optionally, in one embodiment of the present application, the processing of the prior point cloud data to obtain a traversable node graph and a boundary of the area to be re-measured specifically includes: segmenting the prior point cloud data to obtain a non-ground point cloud; clustering the non-ground point cloud to obtain a plurality of building point clouds; performing integrity detection and boundary generation on the plurality of building point clouds to obtain an inclosed slice cluster and a two-dimensional point set; and generating the boundary of the area to be re-measured and a traversable node graph based on the slice cluster and the two-dimensional point set.

[0007] Optionally, in one embodiment of the present application, the integrity check and boundary generation of the multiple building point clouds to obtain an unclosed slice cluster and a two-dimensional point set specifically includes: for each of the building point clouds, slicing the building point cloud to obtain multiple slices; clustering each of the slices to obtain multiple block clusters corresponding to the slices; performing closure check on each of the block clusters, and treating the unclosed block clusters as unclosed slice clusters; and projecting the multiple building point clouds onto a two-dimensional plane to obtain a two-dimensional point set.

[0008] Optionally, in one embodiment of the present application, the generation of the boundary of the area to be supplemented and the passable node graph based on the slice cluster and the two-dimensional point set specifically includes: for each non-closed slice cluster, calculating the difference set between the slice cluster and the two-dimensional point set, and calculating the skeleton line corresponding to the two-dimensional point set; generating a desired completed boundary based on the difference set points and the skeleton line in the difference set; extending the desired completed boundary to obtain an extended point set, and connecting the points at both ends of the difference set with the extended point set to obtain a complete completed boundary; obtaining a total completed boundary of the building based on the completed boundaries corresponding to all the slice clusters; and obtaining the boundary of the area to be supplemented and the passable node graph based on the total completed boundary.

[0009] Optionally, in one embodiment of the present application, obtaining a global planning path based on the traversable node graph and the boundary of the area to be re-measured specifically includes: constructing a graph structure based on the traversable node graph; calculating the shortest path between the boundaries of the area to be re-measured based on the graph structure; and optimizing the shortest path to obtain a global planning path.

[0010] Optionally, in one embodiment of the present application, the graph structure includes a node set and an edge set; the calculating the shortest path between the boundaries of the area to be re-measured based on the graph structure specifically includes: determining the shortest distance between node pairs based on the node set and the edge set; constructing a distance matrix based on the shortest distance; and calculating the access order of accessing the boundaries of the area to be re-measured from the current position based on the distance matrix to obtain the shortest path.

[0011] Optionally, in one embodiment of the present application, the point cloud supplementary sampling is performed according to the global planning path to obtain supplementary sampling data, and the supplementary sampling data and the prior point cloud data are merged to obtain target point cloud data, which specifically includes: collecting real-time environmental information, identifying a supplementary measurement area according to the real-time environmental information and the global planning path, and collecting point cloud data of the supplementary measurement area to obtain supplementary sampling data; unifying the supplementary sampling data and the prior point cloud data into the same coordinate system, respectively extracting corresponding feature points from the supplementary sampling data and the prior point cloud data, and after aligning the supplementary sampling data and the prior point cloud data according to the feature points, performing coordinate transformation on the supplementary sampling data, and fusing the transformed supplementary sampling data with the prior point cloud data to obtain target point cloud data.

[0012] The second aspect of the embodiment of the present application further provides a scene completion system based on aerial point cloud prior guidance, wherein the scene completion system based on aerial point cloud prior guidance includes:

[0013] A missing area identification and division module is used to obtain a priori point cloud data, process the a priori point cloud data, and obtain a passable node diagram and the boundary of the area to be supplemented;

[0014] A route planning module, used to obtain a global planning path according to the traversable node graph and the boundary of the area to be re-measured;

[0015] The missing scene acquisition and fusion module is used to perform point cloud supplementary acquisition according to the global planning path to obtain supplementary acquisition data, and fuse the supplementary acquisition data with the prior point cloud data to obtain target point cloud data.

[0016] The third aspect of an embodiment of the present application also provides a terminal, wherein the terminal includes: a memory, a processor, and a scene completion program based on aerial point cloud prior guidance stored in the memory and executable on the processor, wherein the scene completion program based on aerial point cloud prior guidance implements the steps of the scene completion method based on aerial point cloud prior guidance as described above when executed by the processor.

[0017] The fourth aspect of an embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a scene completion program based on aerial point cloud prior guidance, and when the scene completion program based on aerial point cloud prior guidance is executed by a processor, the steps of the scene completion method based on aerial point cloud prior guidance as described above are implemented.

[0018] Beneficial effects: The present application provides a scene completion method and system based on aerial point cloud prior guidance. The method extracts the boundaries of the area to be explored and the traversable nodes from the prior point cloud, thereby constructing the shortest global path to access all the areas to be supplemented, thereby extracting effective information from the prior point cloud, and thus improving data collection efficiency while ensuring collection integrity. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of a preferred embodiment of the scene completion method based on aerial point cloud prior guidance of the present application;

[0021] Figure 2 It is a flowchart of specific implementation steps of the entire execution process in a preferred embodiment of the scene completion method based on aerial point cloud prior guidance of the present application;

[0022] Figure 3 It is a schematic diagram of the experimental steps and effects of a preferred embodiment of the scene completion method based on aerial point cloud prior guidance of the present application;

[0023] Figure 4 It is a structural diagram of a preferred embodiment of the scene completion system based on aerial point cloud prior guidance of the present application;

[0024] Figure 5 This is a structural diagram of a preferred embodiment of the terminal of this application.

[0025] Description of reference numerals:

[0026] 100. Missing area recognition and division module; 200. Route planning module; 300. Missing scene collection and fusion module.

[0027] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and effect of the present application clearer and more specific, the technical scheme in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. The described embodiments are only possible technical implementations of the present application, not all possible implementations. Based on the embodiments in the present application, those skilled in the art can completely combine the embodiments of the present application to obtain other embodiments without creative work, and these embodiments are also within the scope of protection of the present application.

[0029] In response to the challenging problems of existing data supplementation solutions, this application realizes the information extraction of prior data through a method of constructing a ground traversable topological map based on prior data obtained by drones and a method of detecting building integrity and generating supplementary boundaries. Then, combined with the vehicle-mounted platform mapping trajectory planning and scene completion method based on prior information, the ability and efficiency of supplementary data for large-scale urban ground missing data can be improved, providing complete data support for the refined mapping of urban three-dimensional scenes and the construction of real-scene models.

[0030] Among related technologies, aerial laser scanning is an important means to achieve the acquisition of large-scale urban 3D geometric structures. However, in areas with dense buildings or obstructed by vegetation, there are problems such as incomplete data collection in the ground area and missing hollow structures, which makes it difficult to meet the needs of high-precision 3D reconstruction of buildings.

[0031] The following describes the scene completion method and system based on aerial point cloud prior guidance of the embodiment of the present application with reference to the accompanying drawings. In view of the problem that in the above-mentioned related technologies, when drones acquire point cloud information, the ground area data collection is incomplete, and manual collection is required to supplement the point cloud data, resulting in low data collection efficiency, the present application provides a scene completion method based on aerial point cloud prior guidance. In this method, by extracting the boundaries of the area to be explored and the traversable nodes from the prior point cloud, it is possible to construct the shortest global path to access all areas to be supplemented, and realize the extraction of effective information in the prior point cloud, thereby improving the data collection efficiency while ensuring the integrity of the collection. As a result, the technical problem that in the related technologies, when drones acquire point cloud information, the ground area data collection is incomplete, and manual collection is required to supplement the point cloud data, resulting in low data collection efficiency is solved.

[0032] This application adopts an autonomous three-dimensional point cloud acquisition method for buildings with collaborative air-ground and cross-perspective collaboration to provide more complete data support for high-precision three-dimensional reconstruction of buildings. Specifically, it fully integrates the point cloud information collected from the air and the instance segmentation results to achieve complete point cloud detection and planning of the completion area; samples the node graph through which vehicles can pass based on prior information, and plans the global path in combination with the completion area; and combines the regional autonomous exploration system to achieve missing data completion with prior guidance and complete data fusion.

[0033] The purpose of this application is to automatically extract the target information to be re-surveyed and the passable information in the point cloud collected from the air, and fully integrate it with the vehicle-borne autonomous exploration system, so as to overcome a series of problems encountered in manual re-survey, such as low collection efficiency, poor data quality and consistency, and high cost of manual re-survey, and form a vehicle-borne platform mapping trajectory planning and scene completion solution based on prior information for large-scale urban areas, so as to achieve efficient re-collection of missing data in large-scale scenes.

[0034] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0035] The scene completion method based on aerial point cloud prior guidance described in the preferred embodiment of the present application is as follows: Figure 1 As shown, the scene completion method based on aerial point cloud prior guidance includes the following steps:

[0036] In step S101, a priori point cloud data is obtained, and the a priori point cloud data is processed to obtain a traversable node graph and a boundary of an area to be re-measured.

[0037] In a possible implementation, the prior point cloud data is segmented to obtain ground point cloud and non-ground point cloud; the non-ground point cloud is clustered to obtain multiple building point clouds; integrity detection and boundary generation are performed on the multiple building point clouds to obtain non-closed slice clusters and two-dimensional point sets; based on the slice clusters and the two-dimensional point sets, the boundaries of the area to be re-measured and the passable node graph are generated.

[0038] Specifically, in the process of automatic identification and division of missing areas based on aerial point cloud priors, the prior point cloud data collected by the drone is input; the ground point cloud and non-ground point cloud are segmented by using tools, the non-ground point cloud is clustered, and the building point cloud that meets the conditions is retained. Each building point cloud is sliced ​​and clustered again, and each slice is tested for closure to generate an exploration boundary, project the building point cloud to a two-dimensional plane, generate a skeleton line, and determine the completion boundary based on the skeleton line and the central axis point, merge the completion boundaries of all buildings, and determine the area to be supplemented and the passable nodes; output the boundary of the area to be supplemented and the passable node map.

[0039] It should be noted that in order to extract effective information from the prior point cloud to guide the vehicle-mounted system to complete the task autonomously, this application proposes a method that combines the construction of a traversable node graph with the integrity detection of the building point cloud, that is, to extract the boundaries of the area to be explored and the traversable nodes from the prior point cloud, so as to build the shortest global path to access all areas to be re-surveyed, and to extract effective information from the prior point cloud. In view of a series of problems in manual re-survey, such as low collection efficiency, poor data quality and consistency, and high cost of manual re-survey, the application of ground autonomous exploration system in the field of data re-survey is innovatively introduced, and combined with the point cloud obtained a priori, the ground vehicle-mounted autonomous exploration system is guided to perform automated data re-survey, thereby overcoming the problems of lack of prior guidance, low exploration efficiency, incomplete coverage, incomplete environmental understanding and perception, and insufficient data integration capabilities in existing point cloud completion.

[0040] In a possible implementation, for each of the building point clouds, the building point cloud is sliced ​​to obtain a plurality of slices; each of the slices is clustered to obtain a plurality of block clusters corresponding to the slice; a closure check is performed on each of the block clusters, and the inclosed block clusters are regarded as inclosed slice clusters; and the plurality of building point clouds are projected onto a two-dimensional plane to obtain a two-dimensional point set.

[0041] In a possible implementation, for each of the non-closed slice clusters, the difference set between the slice cluster and the two-dimensional point set is calculated, and the skeleton line corresponding to the two-dimensional point set is calculated; based on the difference set points in the difference set and the skeleton line, the desired completion boundary is generated; the desired completion boundary is extended to obtain an extended point set, and the points at both ends of the difference set are connected to the extended point set to obtain a complete completion boundary; based on the completion boundaries corresponding to all the slice clusters, the total completion boundary of the building is obtained; based on the total completion boundary, the boundary of the area to be supplemented and the passable node graph are obtained.

[0042] For details, see Figure 2 In the process of segmentation and clustering of the prior point cloud, point cloud segmentation: first, use tools such as csf (Cloth Simulation Filtering) cloth simulation algorithm or global mapper (multi-functional map drawing software) to segment the prior point cloud collected by the drone into ground point cloud and non-ground point cloud; non-ground point cloud clustering: cluster the non-ground point cloud to identify independent building point clouds. During the clustering process, retain the building point clouds that meet the point cloud quantity conditions to ensure the accuracy of subsequent analysis.

[0043] In the process of building point cloud integrity detection and boundary generation, slicing processing: for each building point cloud Slice processing, through Discrete cutting of the axis divides the point cloud into multiple slices ; Slice clustering: for each slice Perform clustering to find out the different blocks (clusters) corresponding to the height layer of the building ); Closure detection: for each cluster Perform closure detection and determine whether it is closed by calculating the maximum Euclidean distance between adjacent points of the contour points; Projection and skeleton line extraction: convert the building point cloud Projecting onto a two-dimensional plane gives , and calculate its contour and skeleton line, and extract the central axis of the skeleton line. It can be understood that slicing is the key step to simplify the three-dimensional point cloud into two-dimensional processing, clustering helps to identify the specific structure of the building; closure detection is to identify the boundary area that needs to be completed, and projection and skeleton line extraction are to determine the direction and path of completion.

[0044] In the process of generating and merging the completion boundary, determine the completion boundary: according to the unclosed slice cluster With projection The difference of , as well as the relationship between the difference points and the skeleton lines, generate the desired completion boundary; boundary extension and connection: extend the completion boundary a certain distance along a specific direction and connect the difference points The points at both ends and the expanded point set form a complete completion boundary ; Merge and delete: Combine the clustering results of all slices to get the total completed boundary of the building ,According to the overlapping of the supplementary measurement areas of the buildings to be explored, some of the supplementary measurement areas are merged and deleted to realize the extraction of traversable nodes in the prior point cloud and the determination of the supplementary measurement areas.

[0045] In the embodiment of the present application, the missing area is automatically identified and divided based on the prior point cloud collected by the aerial point cloud: the technology of extracting the supplementary measurement boundary based on the prior point cloud collected by the drone. Figure 3 In the upper part, the CSF cloth simulation algorithm is first used or the global mapper is used to assist in segmenting the prior point cloud into ground point cloud and non-ground point cloud. Then the non-ground point cloud is clustered and the building point cloud that meets the point cloud quantity conditions is retained, so as to perform integrity detection and explore boundary generation for each building's point cloud.

[0046] Specifically, for each building point cloud Processing, its coordinates are , where each point , , , Respectively represent the Points axis, Axis and Axis coordinates, first slice the building point cloud. The slicing process can be done by The axis is represented by discrete cutting. Assume that the height interval of each slice is , you can define the slice as ,in, Indicates slices, containing all Points within the height range, Indicates Slices in The starting height on the axis. Then for each slice Perform clustering, set For Slices The set of clusters in , where each cluster ,represent The The goal of clustering is to find out the different blocks of the slice corresponding to the height layer of the building for further specific analysis. The clustering result can be expressed as: ,in It is a slice The number of clusters in . Each cluster It consists of a set of points, namely , each cluster needs to meet the condition of the minimum number of point clouds, that is, satisfy the following formula (1):

[0047] ; (1)

[0048] in, is a threshold value, which indicates the minimum number of point clouds required in each cluster, that is, a valid cluster The number of points in should be greater than or equal to . Perform a closure check on the clustering results of all tangents. First, The point cloud in the image is extracted by contour extraction. , and retain the sorting rules. According to the geometric characteristics, the maximum Euclidean distance of adjacent points of a closed linear point cloud should be less than the threshold, that is, if the following distance formulas (2) and (3) are satisfied, the point cloud is considered to be open. If it is not closed, it will be used for the generation of subsequent exploration boundaries.

[0049] ; (2)

[0050] ; (3)

[0051] in, Is a distance threshold used to determine whether a linear point cloud is closed. If the maximum Euclidean distance between any adjacent points in the linear point cloud is less than this threshold, the linear point cloud is considered closed. Discretization of the projection boundary yields a two-dimensional point set , , , Represents the points in the point set after projection onto the two-dimensional plane, that is, the three-dimensional building point cloud The point set obtained after projection onto the two-dimensional plane Each point in the point set; find the outline of the point set and save its sorting and generate the skeleton line of the point set, extract the central axis of the skeleton line and sample it to obtain the central axis skeleton point set , , , , represents the medial axis skeleton point extracted from the skeleton line of the two-dimensional point set, that is, The skeleton point set obtained after contour extraction and skeleton line calculation Each point in , Represents the set of medial axis skeleton points Middle Points Axis and Axis coordinates, i.e., skeleton points The two-dimensional coordinates of .

[0052] Take all unclosed tangent clusters , find the unclosed tangent cluster and projection The difference of That is the expected completion boundary, for the difference set Each point in , calculate it and the skeleton point set Euclidean distance of each point, find the nearest skeleton point for each point , and then connect each difference point and the nearest skeleton point into a line segment LineSegment , if the line segment is If the distance of any point in the and its line segments.

[0053] Then along Direction extension line segment LineSegment A certain distance , get the set of points , as shown in formula (4):

[0054] , (4)

[0055] Connect Difference The points at both ends By gathering all the points, we can get the expected completion boundary of this slice cluster. , the clustering of all slices is combined to obtain the total completed boundary of the building ,Finally, according to the overlap of the supplementary measurement areas of all the buildings to be explored, some of the supplementary measurement areas are merged and deleted, so as to realize the extraction of traversable nodes in the prior point cloud and the determination of the supplementary measurement areas.

[0056] In step S102, a global planning path is obtained according to the traversable node graph and the boundary of the area to be re-measured.

[0057] In a possible implementation, a graph structure is constructed according to the traversable node graph; the shortest path between the boundaries of the area to be re-measured is calculated according to the graph structure; and the shortest path is optimized to obtain a global planning path.

[0058] In a possible implementation, the graph structure includes a node set and an edge set. According to the node set and the edge set, the shortest distance between node pairs is determined; according to the shortest distance, a distance matrix is ​​constructed; according to the distance matrix, the access sequence of accessing the boundary of the area to be re-measured from the current position is calculated to obtain the shortest path.

[0059] Specifically, the ground point cloud is initially processed: the acquired ground point cloud (ground_pc) is uniformly sampled to reduce the amount of data and highlight key nodes; non-ground point clouds (collision_pc) are identified as potential collision point clouds, and are used to eliminate ground point clouds that are close to the collision point clouds on the projection plane; the retained ground nodes form a node set V, which is stored in the variable nodes_. Build a graph structure: According to the distance relationship between the nodes in the node set V, build an edge set E and store it in the variable graph_; traverse all edge connection relationships and sample on each edge to generate detection points; check the overlap between the detection points and the collision point cloud, eliminate edge connections with collisions, and ensure the feasibility and safety of the path. Read the re-survey area file and determine the nodes: read the files of all areas to be re-surveyed; ensure that each re-surveyed area contains at least one ground node; for each re-surveyed area, define a node set , indicating the All nodes in the region, Indicates the number of retest areas. Calculate the shortest distance between retest areas: Use algorithm (a graph search algorithm that is used to find the shortest path from a starting point to an end point in a graph or grid) Find all sets in graph_ and The shortest distance between all pairs of nodes ; Save the shortest distance in the variable Shortest_Node_Path_Diagram; Based on the shortest distance, construct the distance matrix D, where Indicates the supplementary test area And the supplementary test area The shortest internal node distance between. Global path planning: using The algorithm, based on the distance matrix D, calculates the order in which the robot visits all the boundaries of the re-measured areas from the current position; it merges the shortest paths between all nodes to form a global planning path. and is a node set, and these two intervals are the intervals in the node set V in the graph structure; traverse All points in the interval, then traverse For all points in the interval, find the distance between two points, save the shortest distance, and then form a distance matrix between all intervals.

[0060] It can be understood that the present application realizes the global route planning of the vehicle-mounted mapping platform for scene completion of missing building areas through preliminary processing of ground point clouds, construction of graph structure, reading of supplementary survey area files, calculation of the shortest distance between supplementary survey areas and planning of global paths.

[0061] It should be noted that in the process of generating the global planning path, the generated boundary of the area to be re-measured and the passable node graph information are first read; according to the passable node graph, the system constructs a graph structure, including a node set and an edge set; The system uses path planning algorithms such as the GIS and other algorithms to calculate the shortest paths to visit all areas to be re-surveyed on the graph structure. These paths take into account the distance and connection relationship between nodes. The calculated paths are optimized to ensure the feasibility and safety of the paths. At the same time, the system will verify whether the paths cover all areas to be re-surveyed and make adjustments as needed. Finally, the system outputs global planning paths, which will guide the ground-based vehicle-mounted autonomous exploration system to perform automated data re-survey.

[0062] In the embodiment of the present application, in the process of global route planning of the vehicle-mounted mapping platform for scene completion of building missing areas, reference is made to Figure 3 In the flowchart on the lower left, in order to obtain the carrier of global path planning, the system performs preliminary uniform sampling on the acquired ground point cloud (ground_pc). This process aims to streamline the data volume and highlight key nodes. Next, the system identifies the non-ground point cloud (collision_pc) as a potential collision point cloud and uses it to eliminate the ground point cloud that is close to the collision point cloud on the projection plane. Through this process, the retained ground nodes form a node set , and stores it in the variable nodes_.

[0063] Construct edge sets based on the distance relationship between nodes , stored in the variable graph_. Then, the system traverses all edge connections and samples each edge to generate detection points. These detection points will be checked again for coincidence with the collision point cloud, thereby eliminating those edge connections with collisions to ensure the feasibility and safety of the planned path.

[0064] In order to further carry out global planning, the system reads the files of all areas to be re-surveyed and ensures that each re-surveyed area contains at least one ground node. Respectively represent the 1st to the The set of nodes in the region, each set Contains all nodes in the region .

[0065] Subsequently, through Algorithm to find the shortest distance among all pairs of nodes between all sets in graph_ As all borders and border The shortest distance between the two nodes is stored in the variable Shortest_Node_Path_Diagram. The shortest distance between the sets satisfies the following conditions:

[0066] ; (5)

[0067] Among them, the collection With Collection The shortest distance among all pairs of nodes between and Respectively represent and The node set in the supplementary measurement area, Representing a collection The nodes, Representing a collection The nodes, Refers to the distance between any two nodes. The system calculates the shortest internal node distance between the supplementary measurement areas. , determine the distance relationship between each supplementary survey area, and use this to construct the distance matrix D, the formula is as follows:

[0068] ; (6)

[0069] ; (7)

[0070] in, , Respectively represent sets and Any node in , based on this matrix, can be used again The algorithm is used to calculate the order in which the robot visits all boundaries from the current position. The shortest path between all nodes is finally merged to form the global planning path extracted through prior information. This planning process gives the vehicle-mounted autonomous exploration system the ability to conduct efficient and highly automated data collection in a large-scale urban environment based on prior information guidance.

[0071] In step S103, point cloud supplementary sampling is performed according to the global planning path to obtain supplementary sampling data, and the supplementary sampling data is fused with the prior point cloud data to obtain target point cloud data.

[0072] In a possible implementation, real-time environmental information is collected, and a supplementary measurement area is identified based on the real-time environmental information and the global planning path, and point cloud data of the supplementary measurement area is collected to obtain supplementary data; the supplementary data and the prior point cloud data are unified into the same coordinate system, and corresponding feature points are extracted from the supplementary data and the prior point cloud data, respectively, and after the supplementary data and the prior point cloud data are aligned based on the feature points, the coordinate transformation is performed on the supplementary data, and the transformed supplementary data is fused with the prior point cloud data to obtain target point cloud data.

[0073] Specifically, in the process of autonomous exploration and complete acquisition, an autonomous exploration method based on a hierarchical architecture is used to explore in a complex three-dimensional environment; automatic and complete acquisition of point cloud information in a given area is achieved to ensure that every small area (patch) on the surface of the target building is successfully acquired; the definition of successful acquisition includes distance constraints (Formula 8) and angle constraints (Formula 9), that is, the viewpoint Need to meet the surface center point distance and angle requirements. In the process of subscribing to the global path and boundary information, the global path and related boundary information are obtained. In the process of autonomous exploration of the supplementary survey area, the robot (vehicle or terminal) performs segmented navigation based on the shortest global path; in the navigation process, the robot autonomously explores the supplementary survey area, that is, those areas that have not been collected before or have been incompletely collected. In the process of point cloud fusion, the supplementary sampled content is fused with the prior point cloud; the fusion process includes denoising, unifying the coordinate system, aligning GPS (Global Positioning System) / IMU (Inertial Measurement Unit) and other data; extracting feature points and matching them, and using the ICP algorithm (Iterative Closest Point algorithm) for precise alignment; performing coordinate transformation and data fusion to complete data repair and point cloud splicing and fusion.

[0074] In the embodiment of the present application, in the process of autonomous collection and point cloud fusion of local missing scenes guided by global paths, the aim is to utilize the prior information extracted in the above steps, combined with the method of autonomous exploration and fusion of empty space and point cloud in large-scale complex three-dimensional environments, so as to achieve autonomous exploration and completion with prior guidance. Figure 3 In the lower right part, we first use a method based on a layered architecture and capable of autonomous exploration in complex 3D environments, which can achieve automatic and complete acquisition of point cloud information within a given area. Complete acquisition is defined as a small patch of a surface area of ​​the target building. is the center, and its normal vector And point to free space, if the center point of the Surface patch needs to be the viewpoint of the robot's position If covered, the following formula is satisfied:

[0075] ; (8)

[0076] ; (9)

[0077] That is, if the viewpoint and the center point of the surface satisfy the distance constraint and the angle constraint, the surface can be considered to be successfully re-sampled.

[0078] Then, the system is modified to subscribe to the global path and related boundary information published in step 2, so as to achieve segmented navigation based on the shortest global path and autonomous exploration of the supplementary survey area. Finally, the supplementary survey content and the prior point cloud are fused, that is, through denoising, unified coordinate system, alignment of GPS / IMU and other data, extraction of feature points and matching, precise alignment by ICP algorithm, coordinate transformation and data fusion, data repair to complete the splicing and fusion of point clouds, and realize automatic completion of point clouds.

[0079] Next, a scene completion system based on aerial point cloud prior guidance proposed in accordance with an embodiment of the present application is described with reference to the accompanying drawings.

[0080] Figure 4 It is a block diagram of a scene completion system based on aerial point cloud prior guidance according to an embodiment of the present application.

[0081] like Figure 4 As shown, the scene completion system based on aerial point cloud prior guidance includes: a missing area recognition and division module 100, a route planning module 200, and a missing scene collection and fusion module 300.

[0082] Specifically, the missing area identification and division module 100 is used to obtain a priori point cloud data, process the a priori point cloud data, and obtain a passable node graph and the boundary of the area to be supplemented;

[0083] A route planning module 200, configured to obtain a global planning path according to the traversable node graph and the boundary of the area to be re-measured;

[0084] The missing scene acquisition and fusion module 300 is used to perform point cloud acquisition and fusion according to the global planning path and the prior point cloud data to obtain target point cloud data.

[0085] Figure 5 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal (which may also be a robot or a vehicle) may include:

[0086] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0087] When the processor 502 executes the program, the scene completion method based on aerial point cloud priori guidance provided in the above embodiment is implemented.

[0088] Furthermore, the terminal further includes:

[0089] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0090] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0091] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0092] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EIS) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0093] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0094] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0095] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned scene completion method based on aerial point cloud prior guidance is implemented.

[0096] One embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the present application Figure 1 A scene completion method based on aerial point cloud prior guidance is provided in any of the corresponding embodiments.

[0097] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0098] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0099] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0100] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or N wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary, and then storing it in a computer memory.

[0101] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0102] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0103] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0104] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

[0105] It should be understood that the application of the present application is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to this application.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A scene completion method based on aerial point cloud prior guidance, characterized in that: The scene completion method based on aerial point cloud prior guidance includes: Acquire prior point cloud data, process the prior point cloud data, and obtain a traversable node graph and a boundary of an area to be re-measured; Obtaining a global planning path according to the traversable node graph and the boundary of the area to be re-measured; Performing point cloud supplementary sampling according to the global planning path to obtain supplementary sampling data, and fusing the supplementary sampling data with the prior point cloud data to obtain target point cloud data; The processing of the prior point cloud data to obtain a traversable node graph and a boundary of the area to be re-measured specifically includes: Segmenting the prior point cloud data to obtain a non-ground point cloud; Clustering the non-ground point cloud to obtain multiple building point clouds; Performing integrity detection and boundary generation on the plurality of building point clouds to obtain an inclosed slice cluster and a two-dimensional point set; Generate a boundary of a region to be re-measured and a passable node graph according to the slice cluster and the two-dimensional point set; The step of generating the boundary of the area to be re-measured and a passable node graph according to the slice cluster and the two-dimensional point set specifically includes: For each of the inclosed slice clusters, calculating a difference set between the slice cluster and the two-dimensional point set, and calculating a skeleton line corresponding to the two-dimensional point set; Generating a desired completion boundary according to the difference points in the difference set and the skeleton line; Extending the desired completion boundary to obtain an extended point set, and connecting the points at both ends of the difference set with the extended point set to obtain a complete completion boundary; Obtaining a total completed boundary of the building according to the completed boundaries corresponding to all the slice clusters; According to the total completed boundary, the boundary of the area to be supplemented and the passable node graph are obtained.

2. The scene completion method based on aerial point cloud prior guidance according to claim 1 is characterized in that: The integrity detection and boundary generation of the plurality of building point clouds to obtain an unclosed slice cluster and a two-dimensional point set specifically includes: For each of the building point clouds, slicing the building point cloud to obtain a plurality of slices; Clustering each of the slices to obtain a plurality of block clusters corresponding to the slices; Performing a closure check on each of the block clusters, and treating the inclosed block clusters as inclosed slice clusters; Project the plurality of building point clouds onto a two-dimensional plane to obtain a two-dimensional point set.

3. The scene completion method based on aerial point cloud prior guidance according to claim 1 is characterized in that: The obtaining of a global planning path according to the traversable node graph and the boundary of the area to be re-measured specifically includes: Constructing a graph structure according to the traversable node graph; Calculating the shortest path between the boundaries of the area to be re-measured according to the graph structure; The shortest path is optimized to obtain a global planning path.

4. The scene completion method based on aerial point cloud prior guidance according to claim 3 is characterized in that: The graph structure includes a node set and an edge set; The calculating the shortest path between the boundaries of the to-be-surveyed area according to the graph structure specifically includes: Determine the shortest distance between node pairs according to the node set and the edge set; Constructing a distance matrix according to the shortest distance; According to the distance matrix, the access sequence of accessing the boundary of the area to be re-measured from the current position is calculated to obtain the shortest path.

5. The scene completion method based on aerial point cloud prior guidance according to claim 1, characterized in that: The method of performing supplementary sampling of point clouds according to the global planning path to obtain supplementary sampling data, and fusing the supplementary sampling data with the prior point cloud data to obtain target point cloud data specifically includes: Collecting real-time environmental information, identifying a supplementary measurement area according to the real-time environmental information and the global planning path, and collecting point cloud data of the supplementary measurement area to obtain supplementary measurement data; The supplementary data and the prior point cloud data are unified into the same coordinate system, the corresponding feature points are extracted from the supplementary data and the prior point cloud data respectively, and the supplementary data and the prior point cloud data are aligned according to the feature points, and then the coordinates of the supplementary data are transformed, and the transformed supplementary data are fused with the prior point cloud data to obtain the target point cloud data.

6. A scene completion system based on aerial point cloud prior guidance, characterized in that: The scene completion system based on aerial point cloud prior guidance includes: A missing area identification and division module is used to obtain a priori point cloud data, process the a priori point cloud data, and obtain a passable node diagram and the boundary of the area to be supplemented; The missing area identification and division module is also used to: segment the prior point cloud data to obtain a non-ground point cloud; cluster the non-ground point cloud to obtain a plurality of building point clouds; perform integrity detection and boundary generation on the plurality of building point clouds to obtain an unclosed slice cluster and a two-dimensional point set; generate the boundary of the area to be supplemented and a passable node graph based on the slice cluster and the two-dimensional point set; The missing area identification and division module is also used for: for each of the inclosed slice clusters, calculating the difference set between the slice cluster and the two-dimensional point set, and calculating the skeleton line corresponding to the two-dimensional point set; generating a desired completion boundary according to the difference set points and the skeleton line in the difference set; extending the desired completion boundary to obtain an extended point set, and connecting the points at both ends of the difference set with the extended point set to obtain a complete completion boundary; obtaining a total completion boundary of the building according to the completion boundaries corresponding to all the slice clusters; obtaining a boundary of the area to be supplemented and a passable node graph according to the total completion boundary; A route planning module, used to obtain a global planning path according to the traversable node graph and the boundary of the area to be re-measured; The missing scene acquisition and fusion module is used to perform point cloud supplementary acquisition according to the global planning path to obtain supplementary acquisition data, and fuse the supplementary acquisition data with the prior point cloud data to obtain target point cloud data.

7. A terminal, characterized in that: The terminal includes: a memory, a processor, and a scene completion program based on aerial point cloud prior guidance stored in the memory and executable on the processor. When the scene completion program based on aerial point cloud prior guidance is executed by the processor, the steps of the scene completion method based on aerial point cloud prior guidance as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a scene completion program based on aerial point cloud prior guidance, and when the scene completion program based on aerial point cloud prior guidance is executed by a processor, the steps of the scene completion method based on aerial point cloud prior guidance as described in any one of claims 1 to 5 are implemented.

Images