A system and method for optimizing point clouds of regular objects in large urban scenes

Abstract

This invention provides a point cloud optimization system and method for regular objects in large urban scenes, relating to the field of point cloud data processing technology. The method includes: processing multiple oblique photographic images containing the regular object to be optimized to obtain a corresponding 3D point cloud; selecting 3D points representing the regular object to be optimized to obtain a first optimized point cloud, and reconstructing a triangular mesh model; smoothing the 3D points in the triangular mesh model to obtain a second optimized point cloud, and processing the topological relationships between the 3D points in the second optimized point cloud; reconstructing a white-film mesh of the regular object to be optimized based on the topological relationships; obtaining baked textures of the regular object to be optimized based on each oblique photographic image; and baking the white-film mesh based on the baked textures to obtain a 3D mesh model of the regular object to be optimized. The beneficial effects are that it reduces the point cloud data volume, improves processing speed, removes noise in the point cloud, and effectively improves the model accuracy of the 3D mesh model.

Description

Technical Field

[0001] This invention relates to the field of point cloud data processing technology, and in particular to a system and method for optimizing point clouds of regular objects in large urban scenes. Background Technology

[0002] 3D reconstruction of buildings has become a research hotspot in the field of spatial information technology in recent years. Constructing complete 3D models of buildings is an important prerequisite and foundation for the construction and development of many virtual platforms such as "digital cities" and "digital earth." Meanwhile, with the innovative development of Earth observation technology, building point cloud data can be easily acquired through various methods, laying the foundation for research on building point cloud reconstruction.

[0003] However, the complex structure of urban scenes, including various components such as buildings, vehicles, trees, billboards, and streetlights, results in a large number of noise or impurities in the point cloud data of buildings in urban scenes. Consequently, the reconstruction models obtained by existing point cloud reconstruction technologies are not accurate enough to meet the application needs of existing urban scenes. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a point cloud optimization system for regular objects in large urban scenes, comprising:

[0005] The image processing module is used to acquire multiple oblique photographic images containing objects of rules to be optimized in a large city scene, and to process each oblique photographic image to obtain the corresponding three-dimensional point cloud.

[0006] The first optimization module, connected to the image processing module, is used to filter out each three-dimensional point representing the regular object to be optimized from the three-dimensional point cloud to obtain the first optimized point cloud, and to reconstruct the triangular mesh model of the regular object to be optimized based on the first optimized point cloud.

[0007] The second optimization module, connected to the first optimization module, is used to smooth the three-dimensional points in the triangular mesh model to obtain a second optimized point cloud, and to process and obtain the topological relationship between each three-dimensional point in the second optimized point cloud.

[0008] The baking processing module, which is connected to the image processing module and the second optimization module, is used to reconstruct the white film mesh of the regular object to be optimized according to the topological relationship, obtain the baked texture of the regular object to be optimized according to each of the oblique photogrammetric images, and bake the white film mesh according to the baked texture to obtain the three-dimensional mesh model of the regular object to be optimized, which is used as the point cloud optimization result of the regular object to be optimized.

[0009] Preferably, the first optimization module includes:

[0010] The elimination unit is used to cluster the three-dimensional point cloud to obtain multiple clusters, process them to obtain the feature class group to which each cluster belongs, and then eliminate the three-dimensional points contained in each cluster corresponding to the feature class group that does not belong to the object to be optimized from the three-dimensional point cloud.

[0011] The downsampling unit, connected to the elimination unit, is used to construct vectors between each three-dimensional point in the cluster and a preset reference point for each retained cluster, calculate the distance between the vectors associated with each pair of three-dimensional points, extract the three-dimensional points whose distance is less than a preset value as redundant points, and then use the centroid of each redundant point to replace each redundant point in order to downsample the cluster.

[0012] A filtering unit, connected to the downsampling unit, is used to count the total number of each three-dimensional point contained in each cluster after downsampling and the radius of the circumscribed sphere of each three-dimensional point. When the total number is less than a preset number and the radius is less than a preset radius, the cluster is deleted to obtain the first optimized point cloud.

[0013] The reconstruction unit, connected to the filtering unit, is used to reconstruct the triangular mesh model of the regular object to be optimized based on the first optimized point cloud.

[0014] Preferably, the rejection unit includes:

[0015] A sub-unit is constructed to build a Kd tree based on the 3D point cloud and configure a corresponding point cloud index for each node in the Kd tree.

[0016] The clustering subunit, connected to the construction subunit, is used to perform nearest neighbor search clustering on the Kd tree in ascending order of search radius within a preset range of the number of clustering points and within a preset search radius to obtain multiple clusters associated with each search radius;

[0017] The identification subunit, connected to the clustering subunit, is used to arrange each search radius and the number of each associated cluster in ascending order of the search radius to form a sequence, and input the sequence into a pre-trained identification model to obtain the feature group to which each cluster belongs;

[0018] The elimination subunit, connected to the recognition subunit, is used to eliminate, based on the recognized feature groups and the point cloud index of each node, each three-dimensional point contained in each cluster corresponding to the feature group that does not belong to the object to be optimized in the three-dimensional point cloud.

[0019] Preferably, the number of preset cluster points ranges from 100 to 25,000.

[0020] Preferably, the reconstruction unit uses a point cloud greedy triangulation algorithm to reconstruct the first optimized point cloud to obtain the triangular mesh model of the regular object to be optimized.

[0021] Preferably, the second optimization module includes:

[0022] Coplanar point filtering unit, used to delete all coplanar points in the triangular mesh model;

[0023] A smoothing unit, connected to the coplanar point filtering unit, is used to perform bilinear interpolation on the non-edge points retained in the triangular mesh model after deleting each of the coplanar points, so as to obtain the second optimized point cloud;

[0024] A topology generation unit, connected to the smoothing unit, is used to process the topology relationships between the three-dimensional points in the second optimized point cloud.

[0025] Preferably, the coplanar point filtering unit includes:

[0026] The first processing subunit is used to perform a neighborhood search for each vertex of each triangular mesh in the triangular mesh model to obtain other vertices within a preset neighborhood range as neighborhood points, and then calculate the line-to-line angle between each line formed between the vertex and each of the neighborhood points.

[0027] The judgment subunit, connected to the first processing subunit, is used to treat the vertex as the coplanar point and delete it when any of the line angles corresponding to the vertex is not within the first preset range, and to generate a judgment signal when all the line angles corresponding to the vertex are within the first preset range.

[0028] The second processing subunit, connected to the judgment subunit, is used to perform a neighborhood search on each of the neighboring points associated with the vertex according to the judgment signal to obtain the successor neighboring points, and then calculate the line-to-line angle between each of the straight lines formed between each of the neighboring points and the corresponding successor neighboring points.

[0029] The third processing subunit, which is connected to the first processing subunit and the second processing subunit respectively, is used to calculate the variance of each line angle corresponding to all the neighboring points and all the successor neighboring points associated with the vertex, and delete the vertex as the coplanar point when the variance is within a second preset range.

[0030] Preferably, the smoothing unit includes:

[0031] The first marking subunit is used to mark each vertex of the associated line angle within a third preset range as the non-edge point;

[0032] An interpolation processing subunit, connected to the first marking subunit, is used to perform grid convergence on each of the line angles associated with each of the non-edge points to obtain multiple convergence values, and then perform bilinear interpolation processing on each of the non-edge points according to each convergence value to obtain the second optimized point cloud.

[0033] Preferably, the topology generation unit includes:

[0034] The second marking subunit is used to mark each vertex of the associated line angle within a fourth preset range as an edge point;

[0035] The projection subunit is used to project each three-dimensional point in the second optimized point cloud onto the same two-dimensional plane through normals to form a projection point cloud.

[0036] The regression processing subunit is connected to the second labeling subunit and the projection subunit respectively. It is used to perform linear regression correction on each projection point in the projection point cloud that corresponds to each edge point and belongs to the same cluster, so as to straighten the edges formed by each edge point and obtain the corrected point cloud.

[0037] The topology connection subunit, which connects to the regression processing subunit, is used to triangulate the corrected point cloud in the two-dimensional plane to obtain the topological relationship of each three-dimensional point in the second optimized point cloud.

[0038] This invention also provides a method for optimizing point clouds of regular objects in large urban scenes, applied to the aforementioned optimization system for point clouds of regular objects in large urban scenes. The method includes:

[0039] Step S1: Obtain multiple oblique photographic images containing objects to be optimized in a large city scene, and process each oblique photographic image to obtain the corresponding three-dimensional point cloud.

[0040] Step S2: Select the three-dimensional points representing the regular object to be optimized from the three-dimensional point cloud to obtain the first optimized point cloud, and reconstruct the triangular mesh model of the regular object to be optimized based on the first optimized point cloud.

[0041] Step S3: Smooth the three-dimensional points in the triangular mesh model to obtain the second optimized point cloud, and process the topological relationship between each three-dimensional point in the second optimized point cloud;

[0042] Step S4: Reconstruct the white film mesh of the regular object to be optimized based on the topological relationship, obtain the baked texture of the regular object to be optimized based on each of the oblique photographic images, and bake the white film mesh based on the baked texture to obtain the three-dimensional mesh model of the regular object to be optimized, which is used as the point cloud optimization result of the regular object to be optimized.

[0043] The above technical solution has the following advantages or beneficial effects: After clustering the 3D point cloud reconstructed based on oblique photogrammetry, interference outliers outside the regular objects to be optimized are identified and removed. Then, the point cloud after removing interference outliers is downsampled and filtered in sequence, which reduces the point cloud data volume, improves the processing speed, removes noise in the point cloud and the remaining outliers after removal. Furthermore, through two greedy triangulations of the point cloud, coplanar point filtering and surface smoothing are achieved, further reducing the amount of point cloud data and effectively improving the model accuracy of the 3D mesh model obtained by point cloud optimization. Attached Figure Description

[0044] Figure 1 A schematic diagram of the structure of a point cloud optimization system for regular objects in a large urban scene, as a preferred embodiment of the present invention;

[0045] Figure 2 This is a flowchart illustrating a preferred embodiment of the present invention for optimizing point clouds of regular objects in a large urban setting. Detailed Implementation

[0046] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The present invention is not limited to this embodiment; other embodiments that conform to the spirit of the present invention may also fall within the scope of the present invention.

[0047] In a preferred embodiment of the present invention, based on the above-mentioned problems existing in the prior art, a point cloud optimization system for regular objects in large urban scenes is provided, such as... Figure 1 As shown, it includes:

[0048] Image processing module 1 is used to acquire multiple oblique photographic images containing regular objects to be optimized in a large city scene, and to process each oblique photographic image to obtain the corresponding three-dimensional point cloud.

[0049] The first optimization module 2 is connected to the image processing module 1. It is used to filter out each three-dimensional point representing the regular object to be optimized from the three-dimensional point cloud to obtain the first optimized point cloud, and to reconstruct the triangular mesh model of the regular object to be optimized based on the first optimized point cloud.

[0050] The second optimization module 3, connected to the first optimization module 2, is used to smooth the three-dimensional points in the triangular mesh model to obtain the second optimized point cloud, and to process the topological relationship between each three-dimensional point in the second optimized point cloud.

[0051] Baking module 4 is connected to image processing module 1 and second optimization module 3 respectively. It is used to reconstruct the white film mesh of the regular object to be optimized according to the topological relationship, obtain the baked texture of the regular object to be optimized according to each oblique photogrammetric image, and bake the white film mesh according to the baked texture to obtain the three-dimensional mesh model of the regular object to be optimized, which is used as the point cloud optimization result of the regular object to be optimized.

[0052] Specifically, in this embodiment, the regular object to be optimized in the aforementioned large city scene is preferably an object with hard edges, i.e., a non-curved surface object, including but not limited to buildings. The aforementioned oblique photogrammetry images are preferably obtained by a drone equipped with high-resolution area array digital cameras at different viewpoints. These high-resolution area array digital cameras preferably include a downward-viewing camera, a front-viewing camera, a rear-viewing camera, a left-viewing camera, and a right-viewing camera. Then, based on multiple oblique photogrammetry images obtained from each viewpoint camera, a 3D point cloud is reconstructed to obtain the corresponding 3D point cloud. Taking a building as an example of the regular object to be optimized, it can be understood that in a large city scene, in addition to the regular object to be optimized, each oblique photogrammetry image is highly likely to also contain other urban components, including but not limited to trees, billboards, streetlights, etc. Therefore, if a 3D network model is directly generated based on the reconstructed 3D point cloud, it will seriously affect the accuracy of the 3D mesh model.

[0053] To address the aforementioned technical issues, the 3D point cloud is first optimized using the first optimization module 2 to remove outlier points other than the regular objects to be optimized, i.e., removing 3D points representing, but not limited to, trees, billboards, and streetlights. Based on this, the first optimization module 2 includes:

[0054] The elimination unit 21 is used to cluster the 3D point cloud to obtain multiple clusters, process them to obtain the feature class group to which each cluster belongs, and then eliminate the 3D points contained in each cluster corresponding to the feature class group of the object to be optimized from the 3D point cloud.

[0055] The downsampling unit 22 is connected to the elimination unit 21. It is used to construct the vector between each three-dimensional point in the cluster and the preset reference point for each retained cluster, calculate the distance between the vectors associated with each pair of three-dimensional points, and extract the three-dimensional points with a distance less than the preset value as redundant points. Then, the centroid of each redundant point is used to replace each redundant point to downsample the cluster.

[0056] The filtering unit 23 is connected to the downsampling unit 22. It is used to count the total number of three-dimensional points contained in each cluster and the radius of the circumscribed sphere of each three-dimensional point for each cluster after downsampling. When the total number is less than a preset number and the radius is less than a preset radius, the cluster is deleted to obtain the first optimized point cloud.

[0057] The reconstruction unit 24 is connected to the filtering unit 23 and is used to reconstruct the triangular mesh model of the regular object to be optimized based on the first optimized point cloud.

[0058] Specifically, to remove interfering outliers other than the objects to be optimized, it is first necessary to identify the interfering outliers. In this embodiment, point cloud clustering is performed by the removal unit 21, and the clusters that do not belong to the objects to be optimized are identified based on the feature groups of each cluster obtained from the clustering; these are the interfering outliers. More specifically, the removal unit 21 includes:

[0059] Construct subunit 211 to build a Kd tree based on the 3D point cloud and configure the corresponding point cloud index for each node in the Kd tree;

[0060] Clustering subunit 212, connected to construction subunit 211, is used to perform nearest neighbor search clustering on the Kd tree in ascending order of search radius within a preset range of the number of clustering points and within a preset search radius to obtain multiple clusters associated with each search radius;

[0061] The identification subunit 213 is connected to the clustering subunit 212. It is used to arrange each search radius and the number of each associated cluster in order of increasing search radius to form a sequence, and input the sequence into the pre-trained identification model to obtain the feature group to which each cluster belongs.

[0062] The elimination subunit 214 is connected to the recognition subunit 213, which is used to eliminate the three-dimensional points contained in each cluster corresponding to the feature group that does not belong to the object to be optimized in the three-dimensional point cloud based on the recognized feature group and the point cloud index of each node.

[0063] Specifically, in this embodiment, the point cloud is sampled from the object surface, and the normal of the object surface is the point cloud normal. Therefore, it is preferable to calculate the normal direction of the 3D point cloud using the least squares method to initially confirm the object surface contained in the 3D point cloud. Then, each 3D point in the 3D point cloud is saved in the structure of (X, Y, Z, R, G, B). Traversing from the outermost edge of the 3D point cloud, a structured Kd tree is constructed. Here, (X, Y, Z) represents the spatial coordinates of the 3D point, and (R, G, B) represents the image color information of the 3D point.

[0064] Each node of the constructed Kd-tree corresponds to a 3D point in the 3D point cloud. Preferably, each node in the Kd-tree is configured with a corresponding point cloud index to represent the position information of each node. For example, the point cloud index of the root node can be configured as (0), the point cloud indices of the two child root nodes connected to the root node can be configured as (0,0) and (0,1) respectively, the point cloud indices of the two nodes connected to the child root node (0,0) can be configured as (0,0,0) and (0,0,1) respectively, the point cloud indices of the two nodes connected to the child root node (0,1) can be configured as (0,1,0) and (0,1,1) respectively, and so on. However, this does not limit the technical solution.

[0065] Furthermore, based on the constructed Kd-tree, point cloud clustering can be performed using nearest neighbor search clustering. Specifically, clustering can be performed according to a preset search radius range and a preset range of the number of cluster points. In this embodiment, the minimum and maximum number of points required for a cluster are limited by the range of the number of cluster points. The preset range of the number of cluster points is preferably 100 to 25,000; in other words, a cluster requires a minimum of 100 points and a maximum of 25,000 points.

[0066] In nearest neighbor search clustering, it is preferable to first select the lower limit of the search radius range as the initial search radius to start a nearest neighbor search clustering, obtaining the corresponding initial clusters. Each 3D point contained in each initial cluster can be represented by a point cloud index vector cil(0), which contains all the point cloud indices of each 3D point in each initial cluster. Then, the initial search radius is expanded to obtain a first search radius, and then a nearest neighbor search clustering is performed based on the first search radius to obtain the corresponding first clusters. Each 3D point contained in each first cluster can be represented by a point cloud index vector cil(1), which contains all the point cloud indices of each 3D point in each first cluster. Then, the first search radius is expanded to obtain a second search radius, and then a nearest neighbor search clustering is performed based on the second search radius to obtain the corresponding second clusters. Each 3D point contained in each second cluster can be represented by a point cloud index vector cil(2), which contains all the point cloud indices of each 3D point in each second cluster. This process continues until the upper limit of the preset search radius is reached. It is understood that the smaller the search radius, the more the actual objects will be divided into multiple clusters; conversely, if the search radius is too large, multiple objects will be divided into a single cluster. Preferably, for each node in the Kd-tree, the farthest distance between the 3D points on the same object surface as the 3D point corresponding to that node is selected as the upper limit of the preset search radius.

[0067] After multiple nearest-neighbor clustering operations, we can successively obtain the initial search radius and its associated initial clusters, the first search radius and its associated first clusters, the second search radius and its associated second clusters, and so on. Considering the shapes of various outliers, including but not limited to trees, power lines, towers, billboards, streetlights, and buildings, for buildings, the number of clusters changes in a stepwise manner as the search radius gradually increases. For trees, which include the canopy and trunk, when the search radius is small, each 3D point within its defined area represents the canopy. Due to the relatively large area of ​​the canopy, the number of clusters initially remains one as the search radius gradually increases until it covers the trunk, at which point the number of clusters suddenly increases to two. As the search radius further increases, the number of clusters remains two until it covers the trunk, and then continues to increase until the search radius covers the ground. Taking trees as an example, to remove trees, it is necessary to identify which clusters represent trees. The above analysis shows that the characteristic category to which each cluster belongs can be identified based on the changing trend of the number of clusters with the search radius. Specifically, this means identifying whether each cluster forms a building, tree, power line, tower, billboard, streetlight, etc. A pre-trained identification model can be used as input, taking the sequence formed by the search radius and the number of its associated clusters as input, and outputting the characteristic category to which the cluster belongs.

[0068] Once the feature class to which each cluster belongs is identified, the 3D points contained in each cluster that do not belong to the feature class of the object to be optimized can be removed from the 3D point cloud, thus achieving the initial removal of interfering outliers.

[0069] Considering that after removing the aforementioned outliers, there may still be points that are too close together, to reduce the point cloud data volume and improve processing speed, it is preferable to use downsampling to replace the centroids of these closely spaced points, thus reducing redundant points. Furthermore, considering that some noise or artifacts may still exist after removing the outliers (for example, some leaves in a tree may not have been completely removed), it is preferable to use filtering to remove clusters with a smaller total number of 3D points and a smaller overall volume as discrete points, thus completing one optimization of the 3D point cloud.

[0070] Furthermore, the triangular mesh model of the regular object to be optimized can be reconstructed based on the first optimized point cloud obtained after optimization. Preferably, the reconstruction unit 24 uses a point cloud greedy triangulation algorithm to reconstruct the first optimized point cloud to obtain the triangular mesh model of the regular object to be optimized.

[0071] Specifically, the first optimized point cloud can be projected onto a two-dimensional coordinate plane using normals. Then, the projected point cloud is triangulated in the plane to obtain the topological connectivity between points. This triangulation process utilizes a spatial region growing algorithm based on Delaunay triangulation. Finally, the topological connectivity between the original 3D points is determined based on the topological connectivity of the projected points in the plane. The resulting triangular mesh is the reconstructed triangular mesh model. It is understandable that since point cloud indices are configured for each 3D point, the topological connectivity can preferably be represented using these indices. Furthermore, since the projected points are generated through projection and have a certain mapping relationship with the original 3D points, the topological connectivity between projected points can be considered equivalent to the topological connectivity between the individual 3D points.

[0072] After reconstructing the triangular mesh model, the second optimization module can be used to further optimize the data volume of the triangular mesh model. Specifically, the second optimization module 3 includes:

[0073] Coplanar point filtering unit 31 is used to delete coplanar points in the triangular mesh model;

[0074] The smoothing unit 32 is connected to the coplanar point filtering unit 31 and is used to perform bilinear interpolation on the non-edge points retained in the triangular mesh model after deleting each coplanar point to obtain the second optimized point cloud.

[0075] The topology generation unit 33 and the connection smoothing unit 32 are used to process the topology relationships between the three-dimensional points in the obtained second optimized point cloud.

[0076] Specifically, in this embodiment, in the constructed triangular mesh model, there may be an inconspicuous bend on one face. This bend can be considered unnecessary, meaning that all points in that region can be considered coplanar. Based on this, coplanar point filtering is required. Specifically, the coplanar point filtering unit 31 includes:

[0077] The first processing subunit 311 is used to perform a neighborhood search for each vertex of each triangular mesh in the triangular mesh model to obtain other vertices within a preset neighborhood range as neighborhood points, and then calculate the line-to-line angle between each line formed between the vertex and each neighborhood point.

[0078] The judgment subunit 312 is connected to the first processing subunit 311 and is used to treat the vertex as a coplanar point and delete it when any line angle corresponding to the vertex is not within the first preset range, and to generate a judgment signal when all line angles corresponding to the vertex are within the first preset range.

[0079] The second processing subunit 313 is connected to the judgment subunit 312 and is used to perform a neighborhood search on each neighboring point associated with the vertex to obtain the successor neighboring point according to the judgment signal, and then calculate the line angle between each line formed between each neighboring point and the corresponding successor neighboring point.

[0080] The third processing subunit 314 is connected to the first processing subunit 311 and the second processing subunit 313 respectively. It is used to calculate the variance of each line angle corresponding to all neighboring points and all successor neighboring points associated with the vertex, and to delete the vertex as a coplanar point when the variance is within a second preset range.

[0081] Specifically, in this embodiment, the first preset range is preferably [π-1 / 18, π+1 / 18], that is, the line angle is close to 180 degrees, and the second preset range is [0, 5]. First, based on the line angle between the vertex and each neighboring point, it is determined whether it is possible for them to be coplanar. When all line angles are within the first preset range, there may be a situation where multiple consecutive faces bend sequentially. In this case, the line angle alone cannot avoid this situation. In this embodiment, further determination is made based on the variance of the neighboring points and all the line angles corresponding to the neighboring points. If the variance is within the second preset range, it is considered that there is still an inconspicuous bend on one face, rather than a situation where multiple consecutive faces bend sequentially, and the point can be deleted as a coplanar point.

[0082] After deleting coplanar points, low-angle points are filtered out, leaving extreme points and intermediate points in the triangular mesh model. Extreme points can be understood as edge points, and intermediate points as non-edge points. For intermediate points, smoothing is performed using bilinear interpolation. In a preferred embodiment of the present invention, the smoothing unit 32 includes:

[0083] The first marking subunit 321 is used to mark each vertex of the associated line angle within a third preset range as a non-edge point;

[0084] Interpolation processing subunit 322 is connected to the first marking subunit 321. It is used to perform grid convergence on each line angle associated with each non-edge point to obtain multiple convergence values. Then, based on each convergence value, bilinear interpolation processing is performed on each non-edge point to obtain the second optimized point cloud.

[0085] Specifically, in this embodiment, the third preset range is preferably [10°, 60°]. The grid convergence involves classifying line angles to multiples of N degrees. For example, if the line angles are 10, 20, and 25 degrees, and the convergence is set to multiples of 6 degrees, then for 10 degrees, the convergence value after grid convergence is 12; for 20 degrees, it is 18; and for 25 degrees, it is 24. These convergence values ​​are then interpolated into the original 10, 20, and 25 using bilinear interpolation. Each convergence value represents a corresponding 3D point, but these are newly added points. These new points need to be added to the triangular mesh model after deleting coplanar points. Therefore, the topological relationship between the new points and the retained points needs to be reconstructed using the topology relationship generation unit 33. In a preferred embodiment of the present invention, the topology relationship generation unit 33 includes:

[0086] The second marking subunit 331 is used to mark each vertex of the associated line angle within a fourth preset range as an edge point;

[0087] The projection subunit 332 is used to project each three-dimensional point in the second optimized point cloud onto the same two-dimensional plane through normals to form a projection point cloud.

[0088] The regression processing subunit 333 is connected to the second labeling subunit 331 and the projection subunit 332 respectively. It is used to perform linear regression correction on each projection point in the projection point cloud that corresponds to each edge point and belongs to the same cluster, so as to straighten the edges formed by each edge point and obtain the corrected point cloud.

[0089] The topology connection subunit 334 and the connection regression processing subunit 333 are used to triangulate the corrected point cloud in a two-dimensional plane to obtain the topological relationship of each three-dimensional point in the second optimized point cloud.

[0090] Specifically, in this embodiment, the aforementioned fourth preset range is [60°, 90°], applicable to scenarios where the object to be optimized is a building. The greedy point cloud triangulation algorithm is also used to generate the topological relationships. Considering the complexity of regression correction processing for edge points in three-dimensional space, after projecting each three-dimensional point onto a two-dimensional plane, regression correction is still needed for each edge point to simplify the edge straightening operation. Then, the corrected point cloud is triangulated to obtain the topological relationships of each three-dimensional point. Similarly, this topological relationship can be represented using point cloud indexing.

[0091] After obtaining the above topological relationships, the triangular mesh model can be reconstructed using baked texture mapping based on these relationships. Specifically, the white film mesh can first be reconstructed based on the above topological relationships, and then the position information of any point in the triangle within the white film mesh can be obtained using the triangular barycentric coordinate system. Further, a triangle has the following relationship:

[0092] r = λ0*r0 + λ1*r1 + λ2*r2

[0093] r = λ0*r0 + λ1*r1 + λ2*r2

[0094] Among them: λ0+λ1+λ2=1; λ0+λ1+λ2=1; 0≤λ0≤1; 0≤λ0≤1; 0≤λ1≤1; 0≤λ1≤1; 0≤λ2≤1; 0≤λ2≤1.

[0095] The above 'r' represents information about a point on the triangle, and 'r0', 'r1', and 'r2' represent information about the three vertices of the triangle. Based on this, we can replace 'r' with 'position' and 'uv' to obtain the following correspondence:

[0096] p=λ0*p0+λ1*p1+λ2*p2p=λ0*p0+λ1*p1+λ2*p2uv=λ0*uv0+λ1*uv1+λ2*uv2uv=λ0*uv0+λ1*uv1+λ2*uv2

[0097] Based on the two formulas above, once the position and UV information of the three fixed points are known, the position information of the point corresponding to the UV coordinate of any point in the triangle can be obtained. It is possible to obtain the triangle containing the corresponding LightPatch and the position of that point within the triangle based on the UV coordinates of pixels in the lightmap. Furthermore, it is possible to map the baked texture onto the white film mesh to obtain a 3D mesh model of the regular object to be optimized, which serves as the point cloud optimization result for the regular object.

[0098] Furthermore, since textures are usually very large and each is a parent material, this embodiment preferably also includes power-lawing the texture. In short, it is to turn the pixels of the texture into a superposition of multiple powers of two. This pixel can be effectively LODed, thereby reducing the load on video memory and system memory.

[0099] This invention also provides a method for optimizing point clouds of regular objects in large urban scenes, applied to the aforementioned optimization system for point clouds of regular objects in large urban scenes, such as... Figure 2 As shown, the optimization methods for point clouds of regular objects in large urban scenes include:

[0100] Step S1: Obtain multiple oblique photographic images containing regular objects to be optimized in a large city scene, and process each oblique photographic image to obtain the corresponding 3D point cloud.

[0101] Step S2: Select the three-dimensional points representing the regular object to be optimized from the three-dimensional point cloud to obtain the first optimized point cloud, and reconstruct the triangular mesh model of the regular object to be optimized based on the first optimized point cloud.

[0102] Step S3: Smooth the three-dimensional points in the triangular mesh model to obtain the second optimized point cloud, and process the topological relationships between the three-dimensional points in the second optimized point cloud.

[0103] Step S4: Reconstruct the white film mesh of the regular object to be optimized based on the topological relationship, obtain the baked texture of the regular object to be optimized based on each oblique photographic image, and bake the white film mesh based on the baked texture to obtain the three-dimensional mesh model of the regular object to be optimized, which is used as the point cloud optimization result of the regular object to be optimized.

[0104] The above description is merely a preferred embodiment of the present invention and does not limit the implementation and protection scope of the present invention. Those skilled in the art should realize that any equivalent substitutions and obvious changes made using the content of this specification and illustrations should be included within the protection scope of the present invention.

Claims

1. A point cloud optimization system for regular objects in a large urban scene, characterized in that, The method comprises the following steps: An image processing module is configured to acquire multiple oblique photography images containing a regular object to be optimized in a large city market scene, and to obtain a corresponding three-dimensional point cloud according to each of the oblique photography images; A first optimization module is connected to the image processing module and configured to filter out each three-dimensional point representing the regular object to be optimized from the three-dimensional point cloud to obtain a first optimized point cloud, and to reconstruct the triangular mesh model of the regular object to be optimized according to the first optimized point cloud; A second optimization module is connected to the first optimization module and configured to perform smoothing processing on each three-dimensional point in the triangular mesh model to obtain a second optimized point cloud, and to process the topological relationship between each three-dimensional point in the second optimized point cloud; A baking processing module is connected to the image processing module and the second optimization module, respectively, and is configured to reconstruct the white film mesh of the regular object to be optimized according to the topological relationship, to acquire the baking map of the regular object to be optimized according to each of the oblique photography images, and to perform baking on the white film mesh according to the baking map to obtain the three-dimensional mesh model of the regular object to be optimized as the point cloud optimization result of the regular object to be optimized; The first optimization module comprises: A culling unit is configured to cluster the three-dimensional point cloud to obtain multiple clustering clusters, to process the feature class to which each clustering cluster belongs, and to subsequently remove each three-dimensional point contained in each clustering cluster corresponding to the feature class not belonging to the regular object to be optimized from the three-dimensional point cloud; A down-sampling unit is connected to the culling unit and configured to, for each retained clustering cluster, construct the vector between each three-dimensional point in the clustering cluster and a preset reference point, calculate the distance between the vectors associated with each pair of three-dimensional points, extract each three-dimensional point corresponding to a distance less than a preset value as a redundant point, and subsequently replace each redundant point with the barycenter point of each redundant point to down-sample the clustering cluster; A filtering unit is connected to the down-sampling unit and configured to, for each clustering cluster after down-sampling, count the total number of each three-dimensional point contained in the clustering cluster and the radius of the circumscribed sphere of each three-dimensional point, and delete the clustering cluster when the total number is less than a preset number and the radius is less than a preset radius to obtain the first optimized point cloud; A reconstruction unit is connected to the filtering unit and configured to reconstruct the triangular mesh model of the regular object to be optimized according to the first optimized point cloud.

2. The large city market scenario rule object point cloud optimization system of claim 1, wherein, The culling unit comprises: A construction sub-unit is configured to construct a Kd tree according to the three-dimensional point cloud and configure a corresponding point cloud index for each node in the Kd tree; A clustering sub-unit is connected to the construction sub-unit and configured to perform neighbor search clustering on the Kd tree according to a preset clustering point number range and in a preset search radius range in the order of increasing search radius to obtain multiple clustering clusters associated with each search radius. An identifying subunit connected to the clustering subunit is configured to arrange the search radii and the number of the clustering clusters associated with each search radius in sequence according to the order from small to large of the search radii, and input the sequence into a pre-trained identification model to obtain the feature class group to which each clustering cluster belongs; A removing subunit connected to the identifying subunit is configured to remove, according to the feature class group and the point cloud index of each node obtained by identification, each three-dimensional point included in each clustering cluster corresponding to the feature class group that does not belong to the regular object to be optimized in the three-dimensional point cloud.

3. The large city market scenario rule object point cloud optimization system of claim 2, wherein, The number of clustering points ranges from 100 to 25,000.

4. The large city market scenario rule object point cloud optimization system of claim 1, wherein, The reconstruction unit reconstructs the first optimized point cloud by using a point cloud greedy triangulation algorithm to obtain the triangular mesh model of the regular object to be optimized.

5. The large city market scenario rule object point cloud optimization system of claim 1, wherein, The second optimization module comprises: A coplanar point filtering unit configured to delete each coplanar point in the triangular mesh model; A smoothing unit connected to the coplanar point filtering unit and configured to perform bilinear interpolation processing on the non-edge points remaining in the triangular mesh model after each coplanar point is deleted to obtain the second optimized point cloud; A topological relationship generating unit connected to the smoothing unit and configured to process the topological relationship between each three-dimensional point in the second optimized point cloud.

6. The large city market scenario rule object point cloud optimization system of claim 5, wherein, The coplanar point filtering unit comprises: A first processing subunit configured to, for each vertex of each triangular mesh in the triangular mesh model, perform neighborhood search to obtain other vertices within a preset neighborhood range as neighborhood points, and then calculate the line-line angles between each straight line formed between the vertex and each neighborhood point; A judging subunit connected to the first processing subunit and configured to, when any one of the line-line angles corresponding to the vertex is not within a first preset range, delete the vertex as the coplanar point, and generate a judgment signal when each line-line angle corresponding to the vertex is within the first preset range; A second processing subunit connected to the judging subunit and configured to, according to the judgment signal, perform neighborhood search on each neighborhood point associated with the vertex to obtain successor neighborhood points, and then calculate the line-line angles between each straight line formed between each neighborhood point and the corresponding successor neighborhood point; A third processing subunit respectively connected to the first processing subunit and the second processing subunit and configured to calculate the variance of each line-line angle corresponding to all neighborhood points and all successor neighborhood points associated with the vertex, and delete the vertex as the coplanar point when the variance is within a second preset range.

7. The large city market scenario rule object point cloud optimization system of claim 6, wherein, The smoothing unit comprises: A first marking subunit configured to mark each vertex with a line-line angle within a third preset range as a non-edge point; An interpolation processing subunit connected to the first marking subunit and configured to perform grid convergence on each line-line angle associated with each non-edge point to obtain a plurality of convergence values, and then perform bilinear interpolation processing on each non-edge point according to each convergence value to obtain the second optimized point cloud.

8. The large city market scenario rule object point cloud optimization system of claim 6, wherein, The topological relationship generating unit comprises: The second marking subunit is configured to mark each vertex with a line angle within a fourth preset range as an edge point; The projection subunit is configured to project each three-dimensional point in the second optimized point cloud into a same two-dimensional plane through a normal line to form a projected point cloud; The regression processing subunit is respectively connected to the second marking subunit and the projection subunit, and is configured to perform linear regression correction on each projected point in the projected point cloud corresponding to each edge point and belonging to a same cluster to straighten an edge formed by each edge point, to obtain a corrected point cloud; The topology connection subunit is connected to the regression processing subunit, and is configured to triangulate the corrected point cloud in the two-dimensional plane to obtain the topology relationship of each three-dimensional point in the second optimized point cloud.

9. A method for optimizing a point cloud of regular objects in a large city market scenario, the method comprising: receiving a point cloud of regular objects in a large city market scenario; and optimizing the point cloud of regular objects in the large city market scenario. The rule object point cloud optimization method applied to the large city market scene as claimed in any one of claims 1-8 comprises: Step S1, obtaining multiple oblique photography images of a large city market scene containing a rule object to be optimized, and obtaining a corresponding three-dimensional point cloud according to each oblique photography image; Step S2, selecting each three-dimensional point representing the rule object to be optimized from the three-dimensional point cloud to obtain a first optimized point cloud, and reconstructing a triangular mesh model of the rule object to be optimized according to the first optimized point cloud; Step S3, performing smoothing processing on the three-dimensional points in the triangular mesh model to obtain a second optimized point cloud, and processing the topology relationship between each three-dimensional point in the second optimized point cloud; Step S4, reconstructing a white film mesh of the rule object to be optimized according to the topology relationship, obtaining a baked map of the rule object to be optimized according to each oblique photography image, and baking the white film mesh according to the baked map to obtain a three-dimensional mesh model of the rule object to be optimized as a point cloud optimization result of the rule object to be optimized.

Images