Point cloud up-sampling method and device based on multi-scale gaussian process, equipment and medium

Abstract

The application provides a point cloud upsampling method and device based on a multi-scale Gaussian process, equipment and a medium, comprising: obtaining an original point cloud, generating a plurality of scale levels with different spatial resolutions, each scale level corresponding to a voxel, and establishing a spatial index correlation relationship between different scale voxels; for the voxels of each scale level, plane attribute determination is performed according to the geometric distribution characteristics of the internal point cloud to determine the voxels to be reconstructed; local densification reconstruction and sampling processing are performed on each voxel to be reconstructed based on a Gaussian process regression model to determine the upsampled points; and all upsampled points generated at each scale are combined with the original point cloud to output a final dense point cloud. The method can adaptively fuse geometric features of different scales, effectively suppress noise, recover complex geometric structures of an object surface, and improve the accuracy of point cloud upsampling.

Description

Technical Field

[0001] This application relates to the field of 3D computer vision and graphics data processing technology, and in particular to point cloud upsampling methods, devices, equipment and media based on multi-scale Gaussian processes. Background Technology

[0002] With the rapid development of 3D perception technology, acquisition devices such as LiDAR, structured light depth cameras, and multi-view stereo vision are widely used in fields such as autonomous driving, robot navigation, cultural heritage digitization, and industrial inspection. However, due to limitations in the physical resolution of hardware sensors, acquisition distance, and environmental occlusion, the raw point cloud data actually acquired often suffers from sparse distribution, uneven density, and missing areas. This "data sparsity" leads to holes in visualization, resulting in insufficient realism; it also restricts the accuracy and robustness of subsequent high-level vision tasks such as point cloud registration, segmentation, feature extraction, and 3D reconstruction.

[0003] Existing point cloud upsampling methods are mainly divided into traditional methods based on local interpolation and deep learning methods: 1. Limitations of traditional interpolation methods: Although traditional local moving least squares methods or Poisson reconstruction-based methods are relatively simple to compute, they often lead to excessive smoothing of geometric details and loss of angular detail features when dealing with complex topological structures or edge regions. 2. Data dependency and black-box problem of deep learning methods: These methods learn a nonlinear mapping from sparse points to dense points through an end-to-end network architecture. They are highly dependent on large-scale paired training datasets (sparse-dense point cloud pairs), but obtaining high-quality real point cloud data is costly, resulting in poor generalization ability of the model on cross-domain data. At the same time, the "black-box" nature of deep networks makes their output results lack interpretability and makes it difficult to provide uncertainty estimates for predicted points. Therefore, how to improve the accuracy of point cloud upsampling has become a significant technical problem. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a point cloud upsampling method, apparatus, device and medium based on multi-scale Gaussian processes, which can adaptively fuse geometric features of different scales, effectively suppress noise while restoring the complex geometric structure of the object surface and improving the accuracy of point cloud upsampling.

[0005] This application provides a point cloud upsampling method based on a multi-scale Gaussian process, the point cloud upsampling method comprising: The original point cloud is acquired, and multiple scale levels with different spatial resolutions are generated. Each scale level corresponds to its own voxel, and spatial index associations between voxels of different scales are established. For voxels at each scale level, planar attributes are determined based on the geometric distribution characteristics of their internal point clouds to identify the voxels to be reconstructed. Based on the Gaussian process regression model, local dense reconstruction and sampling processing are performed on each voxel to be reconstructed to determine the upsampling point; All upsampled points generated at each scale are merged with the original point cloud to output the final dense point cloud.

[0006] In one possible implementation, determining the voxels to be reconstructed by performing planar attribute determination on the voxel meshes at each scale level based on the geometric distribution characteristics of their internal point clouds includes: For non-empty voxels at coarse and medium scale levels, covariance feature decomposition and flatness threshold determination are used, combined with spatial clustering checks, to screen planar voxels and include them in the set of voxels to be reconstructed, while non-planar voxels are passed down along the hierarchical index to a finer scale for further processing. For non-empty voxels at fine-scale levels, all non-empty voxels are directly identified as voxels to be reconstructed.

[0007] In one possible implementation, the step of selecting planar voxels to be included in the set of voxels to be reconstructed by covariance eigenvalue decomposition and flatness threshold determination, combined with spatial clustering checks, includes: The covariance matrix of the voxel in-point set is determined and eigenvalue decomposition is performed to obtain the flatness index. If the flatness index meets the preset flatness threshold condition, spatial continuity clustering check is performed. If the clustering check passes, the voxel is marked as a planar voxel and identified as the voxel to be reconstructed; otherwise, it is marked as a non-planar voxel.

[0008] In one possible implementation, the spatial continuity clustering check includes: Based on the local neighborhood density distribution of the point cloud within the voxel to be reconstructed, determine whether the point set belongs to the same connected cluster; If the clustering result is a single cluster, then spatial continuity is determined as a planar voxel; otherwise, it is determined as a non-planar voxel.

[0009] In one possible implementation, the step of performing local dense reconstruction and sampling on each voxel to be reconstructed based on the Gaussian process regression model to determine the upsampling point includes: For each reconstructed voxel, a local spatial coordinate system is established based on its internal point cloud. The projection coordinates of the point cloud on the local tangent plane are used as input, and the projection values ​​in the local normal direction are used as output to construct a training dataset for Gaussian process regression. A Gaussian process regression model is established by selecting the radial basis function as the covariance function and introducing a noise term; In the local spatial coordinate system, predictive sampling of regular grid points is performed inside the voxel to be reconstructed, and the predicted mean and predicted variance of each sampling point are calculated using the Gaussian process regression model. Predicted points are selected based on the variance threshold and the predicted variance, and the predicted points are restricted to be within the current voxel bounding box. The selected predicted points are then transformed from the local spatial coordinate system back to the global coordinate system to generate upsampling points.

[0010] In one possible implementation, establishing a local spatial coordinate system for the voxel to be reconstructed based on its internal point cloud includes: A local spatial coordinate system is constructed with the centroid of the point set within the voxel to be reconstructed as the origin, the eigenvector corresponding to the largest eigenvalue of the covariance matrix as the first tangential axis, the eigenvector corresponding to the second largest eigenvalue as the second tangential axis, and the eigenvector corresponding to the smallest eigenvalue as the normal direction axis.

[0011] In one possible implementation, before merging all upsampled points generated at each scale with the original point cloud to output the final dense point cloud, the point cloud upsampling method further includes: Construct an octree spatial index for the original point cloud; For each upsampling point, the octree spatial index is used to quickly search for its nearest neighbor in the original point cloud, and the attributes of the nearest neighbor are assigned to the upsampling point.

[0012] This application also provides a point cloud upsampling device based on a multi-scale Gaussian process, the point cloud upsampling device comprising: The voxelization construction module is used to acquire the original point cloud, generate multiple scale levels with different spatial resolutions, each scale level corresponds to its own voxel, and establish spatial index association relationships between voxels of different scales. The hierarchical planar detection module is used to determine the planar attributes of voxels at each scale level based on the geometric distribution characteristics of their internal point clouds, and to identify the voxels to be reconstructed. The Gaussian fitting module is used to perform local densification reconstruction and sampling processing on each voxel to be reconstructed based on the Gaussian process regression model, and to determine the upsampling point. The merging module is used to merge all upsampled points generated at each scale with the original point cloud to output the final dense point cloud.

[0013] This application embodiment also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the point cloud upsampling method based on the multi-scale Gaussian process described above.

[0014] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the point cloud upsampling method based on a multi-scale Gaussian process as described above.

[0015] This application provides a point cloud upsampling method, apparatus, device, and medium based on a multi-scale Gaussian process. The point cloud upsampling method includes: acquiring an original point cloud; generating multiple scale levels with different spatial resolutions, each scale level corresponding to its own voxels, and establishing spatial index associations between voxels of different scales; determining the planar attributes of voxels at each scale level based on the geometric distribution characteristics of their internal point clouds to identify the voxels to be reconstructed; performing local dense reconstruction and sampling processing on each voxel to be reconstructed based on a Gaussian process regression model to determine upsampling points; and merging all upsampling points generated at each scale with the original point cloud to output the final dense point cloud. This method can adaptively fuse geometric features at different scales, effectively suppressing noise while restoring the complex geometric structures of object surfaces, thus improving the accuracy of point cloud upsampling.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a point cloud upsampling method based on a multi-scale Gaussian process, provided as an embodiment of this application; Figure 2 This is one of the structural schematic diagrams of a point cloud upsampling device based on a multi-scale Gaussian process provided in an embodiment of this application; Figure 3 This is a second schematic diagram of a point cloud upsampling device based on a multi-scale Gaussian process, provided as an embodiment of this application. Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0020] First, the applicable scenarios for this application are introduced. This application can be applied to the field of 3D computer vision and graphics data processing technology.

[0021] Research has revealed that existing point cloud upsampling methods are mainly divided into traditional local interpolation-based methods and deep learning methods: 1. Limitations of traditional interpolation methods: While traditional local moving least squares methods or Poisson reconstruction-based methods are relatively simple to compute, they often lead to excessive smoothing of geometric details and loss of angular detail features when dealing with complex topologies or edge regions. 2. Data dependency and black-box problem of deep learning methods: These methods learn a nonlinear mapping from sparse to dense points through an end-to-end network architecture. They are highly dependent on large-scale paired training datasets (sparse-dense point cloud pairs), but acquiring high-quality real point cloud data is costly, resulting in poor generalization ability of the model on cross-domain data. Simultaneously, the "black-box" nature of deep networks makes their output results lack interpretability and makes it difficult to provide uncertainty estimates for predicted points. Therefore, improving the accuracy of point cloud upsampling has become a significant technical challenge.

[0022] Based on this, this application provides a point cloud upsampling method based on a multi-scale Gaussian process, which can adaptively fuse geometric features of different scales, effectively suppress noise, restore the complex geometric structure of the object surface, and improve the accuracy of point cloud upsampling.

[0023] Please see Figure 1 , Figure 1 This is a flowchart illustrating a point cloud upsampling method based on a multi-scale Gaussian process, provided as an embodiment of this application. Figure 1 As shown in the embodiments of this application, the point cloud upsampling method includes: S101: Obtain the original point cloud, generate multiple scale levels with different spatial resolutions, each scale level corresponds to its own voxels, and establish spatial index associations between voxels of different scales.

[0024] In this step, the original point cloud is acquired, and at least three scale levels with different spatial resolutions are generated. Each scale level corresponds to a voxel grid. A hash index is established for each non-empty voxel, and the spatial affiliation relationship between parent and child voxels is recorded, so that coarse-scale voxels can directly locate all their corresponding child-scale voxels.

[0025] Here, the spatial index association between voxels of different scales is as follows: the voxel resolutions of adjacent scale levels are integer multiples of each other, and each coarse-scale voxel corresponds to a fixed number of sub-scale voxels in space; the index of each scale voxel records the index information of its parent voxel or child voxel to achieve direct cross-scale positioning.

[0026] The voxel resolution is set to three levels: Vcoarse = 0.2m, Vmedium = 0.1m, and Vfine = 0.05m. The corresponding input point cloud is the downsampled point cloud obtained above. To avoid memory overflow caused by pre-allocating a huge 3D array, a spatial hash table is used to dynamically manage non-empty voxels.

[0027] Here, for any point P(x,y,z), its voxel index (i,j,k) at level k is calculated as follows:

[0028] in, , , The minimum boundary values ​​of the point cloud in the X, Y, and Z directions for the entire scene. For the first k The voxel side length of the hierarchy (scale).

[0029] Specifically, shift-based XOR hashing is used to compress the three-dimensional index into a one-dimensional key value:

[0030] here, , , It is a pre-selected large prime constant.

[0031] S102: For voxels at each scale level, determine the planar attributes based on the geometric distribution characteristics of their internal point clouds to identify the voxels to be reconstructed.

[0032] In this step, starting from the coarsest scale, the planar attributes of each non-empty voxel at the current scale are determined based on the geometric distribution characteristics of its internal point cloud, thus identifying the voxels to be reconstructed.

[0033] It should be noted that the voxels to be reconstructed consist of planar voxels and non-empty voxels.

[0034] In one possible implementation, determining the voxels to be reconstructed by performing planar attribute determination on the voxel meshes at each scale level based on the geometric distribution characteristics of their internal point clouds includes: (1): For non-empty voxels at coarse and medium scale levels, planar voxels are selected and included in the set of voxels to be reconstructed by covariance feature decomposition and flatness threshold determination, combined with spatial clustering check, while non-planar voxels are passed down to a finer scale for further processing along the hierarchical index.

[0035] Here, for non-empty voxels at coarse and medium scale levels, planar voxels are identified through covariance feature decomposition, flatness threshold determination, and spatial clustering checks, and are then included in the set of voxels to be reconstructed. Non-planar voxels are then passed down along the hierarchical index to a finer scale for further processing.

[0036] It should be noted that coarse-scale non-planar objects enter the medium-scale sub-voxel mesh, while medium-scale non-planar objects enter the fine-scale sub-voxel mesh.

[0037] In one possible implementation, the step of selecting planar voxels to be included in the set of voxels to be reconstructed by covariance eigenvalue decomposition and flatness threshold determination, combined with spatial clustering checks, includes: The covariance matrix of the point set within the voxel is determined and eigenvalue decomposition is performed to obtain the flatness index. If the flatness index meets the preset flatness threshold condition, a spatial continuity clustering check is performed. If the clustering check passes, the voxel is marked as a planar voxel and the planar voxel is determined as the voxel to be reconstructed; otherwise, it is marked as a non-planar voxel.

[0038] Here, for any non-empty voxel V at the current scale, the set of points P={p1,p2,...,pn} contained within it is extracted. If the number of points n<3, it is directly determined to be non-planar or invalid. The covariance matrix is ​​determined by the following formula:

[0039] Where C is the covariance matrix, and n is the total number of point clouds contained in any non-empty voxel. For each i-th point cloud, The center of mass.

[0040] Here, the covariance matrix C is decomposed into eigenvalues ​​λ1≥λ2≥λ3≥0 and their corresponding eigenvectors e1,e2,e3. These three eigenvalues ​​represent the variance of the point cloud distribution in three orthogonal directions. The ratio of the eigenvalues ​​is used to quantify the "flatness" of the plane. An ideal two-dimensional plane has minimal variance in the normal direction (λ3≈0), while the variances in the two directions within the plane are relatively large and close. The flatness Splanar is determined by the following formula: Splanar=λ2 / λ1. When Splanar>100, it indicates that λ1 and λ2 are extremely close (i.e., the distribution within the plane is uniform, with no significant linear dominance), and λ3 is much smaller than the former two.

[0041] In one possible implementation, the spatial continuity clustering check includes: determining whether a set of points belongs to the same connected cluster based on the local neighborhood density distribution of the point cloud within the voxel to be reconstructed; if the clustering result is a single cluster, then the spatial continuity is determined to be a planar voxel, otherwise it is determined to be a non-planar voxel.

[0042] Here, relying solely on PCA eigenvalues ​​can easily misclassify multiple parallel but separate planes or sparse noise as a single plane. Therefore, DBSCAN clustering is introduced to check spatial continuity. The DBSCAN algorithm is executed in the local spatial coordinate system of voxel V. The parameters are set as follows: neighborhood radius ( ): Set to 0.5 times the current voxel resolution (e.g., at a coarse scale). =0.1m). Minimum number of points (MinPts): Set to 3, count the number of clustering results Nclusters. If Nclusters=1: it means that all points within the voxel are spatially continuous and connected, satisfying the "single clustering" condition. If Nclusters>1: it means that there are outliers or multiple separate geometric surfaces, and it is determined to be non-planar.

[0043] (2): For non-empty voxels at fine-scale levels, all non-empty voxels are directly identified as voxels to be reconstructed.

[0044] Here, a coarse-to-fine sequential decision-making mechanism is used to determine the geometric properties of voxels at three scales: coarse, medium, and fine. Only voxels marked as "non-planar" will flow into the next level, thus achieving efficient hierarchical processing. While processing large planes such as walls and floors at the large scale, detailed structures such as railings and corners are processed at the small scale, ensuring both processing efficiency and accuracy.

[0045] This application introduces a planar voxel selection mechanism at coarse and mesoscale levels, only transferring non-planar voxels to finer-scale processing, avoiding redundant computations of Gaussian regression at the highest resolution for all voxels. Point clouds in planar regions are reconstructed at large-scale voxels with lower computational cost, while fine-scale processing is only enabled for complex regions when necessary. This "on-demand refinement" strategy significantly reduces overall computational overhead and improves processing efficiency compared to uniformly applying fine-scale processing to all regions.

[0046] S103: Based on the Gaussian process regression model, local dense reconstruction and sampling processing are performed on each voxel to be reconstructed to determine the upsampling point.

[0047] In this step, a Gaussian process regression model is used to generate sampling points within each voxel to be reconstructed at a preset density. The sampling points are then filtered based on the output prediction confidence and converted back to the global coordinate system to obtain upsampling points.

[0048] In one possible implementation, the method involves performing local densification reconstruction and sampling on each voxel to be reconstructed based on a Gaussian process regression model to determine the upsampling point, including: a: For the reconstructed voxels, a local spatial coordinate system is established based on their internal point cloud. The projection coordinates of the point cloud on the local tangent plane are used as input and the projection values ​​in the local normal direction are used as output to construct the training dataset for Gaussian process regression.

[0049] Here, for each voxel to be reconstructed entering the reconstruction process, a local spatial coordinate system is established with the centroid of its internal point set as the origin, the two principal tangents obtained from principal component analysis as the plane coordinate axes, and the normal as the regression target axis. The projection values ​​of the voxel's point cloud on the plane coordinate axes are used as training inputs, and the projection values ​​on the normal axis are used as training outputs to construct the training dataset for the Gaussian process regression model.

[0050] In one possible implementation, establishing a local spatial coordinate system for the voxel to be reconstructed based on its internal point cloud includes: A local spatial coordinate system is constructed with the centroid of the point set within the voxel to be reconstructed as the origin, the eigenvector corresponding to the largest eigenvalue of the covariance matrix as the first tangential axis, the eigenvector corresponding to the second largest eigenvalue as the second tangential axis, and the eigenvector corresponding to the smallest eigenvalue as the normal direction axis.

[0051] b: Select the radial basis function as the covariance function and introduce a noise term to establish a Gaussian process regression model.

[0052] Here, the RBF kernel (radial basis function) is selected and a noise term is introduced, as shown in the following formula:

[0053] Where l is the length scale, controlling the smoothness of the surface, and is set to 0.5 times the voxel resolution. The amplitude of the control function value can usually be set to 1. Let x be the covariance between any two input points x and x′. l For length scale, For noise variance, δxx′ Let be the Kronecker Delta function. Assume the observations follow a Gaussian distribution. ,in To collect noise variance observations y and predicted values ​​for point clouds Follows a joint Gaussian distribution:

[0054] here, Let n×n be the covariance matrix between all known points. Given point X and predicted point The covariance of , where I is the identity matrix. Let be the covariance between the predicted point and itself.

[0055] c: In the local spatial coordinate system, predictive sampling of regular grid points is performed inside the voxel to be reconstructed, and the predicted mean and predicted variance of each sampling point are calculated using the Gaussian process regression model.

[0056] Here, the predicted mean and variance are obtained: Mean: ;variance: .

[0057] d: Based on the variance threshold and the predicted variance, the predicted points are selected and restricted to be within the current voxel bounding box. The selected predicted points are then transformed from the local spatial coordinate system back to the global coordinate system to generate upsampling points.

[0058] Here, the prediction points are filtered according to the preset variance thresholds for each scale, and the prediction points are constrained to fall within the current voxel space boundary. The filtered prediction points are then converted back to the global coordinate system to obtain the upsampled points.

[0059] In this context, a variance tolerance threshold of 0.04 is set for planar regions (coarse / medium scale) and a variance tolerance threshold of 0.02 is set for non-planar regions (fine scale), retaining only prediction points with variance less than the set threshold.

[0060] S104: Merge all upsampled points generated at each scale with the original point cloud to output the final dense point cloud.

[0061] In this step, all scales are traversed, and all generated upsampled points are merged with the original point cloud to form the final dense point cloud.

[0062] In one possible implementation, before merging all upsampled points generated at each scale with the original point cloud to output the final dense point cloud, the point cloud upsampling method further includes: Construct an octree spatial index for the original point cloud; for each upsampled point, use the octree spatial index to quickly search for its nearest neighbor in the original point cloud, and assign the attributes of the nearest neighbor to the upsampled point.

[0063] Here, an octree spatial index of the original point cloud is constructed; for each newly generated upsampled point, the octree spatial index is used to quickly search for its nearest neighbor in the original point cloud; the color, intensity, or normal vector attribute of the nearest neighbor is assigned to the upsampled point.

[0064] This application constructs an octree spatial index for the original point cloud, quickly searches for the nearest neighbor of each newly generated sampling point in the original point cloud, and assigns corresponding attributes (color, intensity, normal vector, etc.). This attribute inheritance method ensures that the upsampling results retain geometric accuracy while fully preserving the physical attribute information of the original measurement data, making it easy for downstream tasks to use directly.

[0065] This application provides a point cloud upsampling method based on a multi-scale Gaussian process. The method includes: acquiring an original point cloud; generating multiple scale levels with different spatial resolutions, each scale level corresponding to its own voxels, and establishing spatial index associations between voxels of different scales; determining the planar attributes of voxels at each scale level based on the geometric distribution characteristics of their internal point clouds to identify the voxels to be reconstructed; performing local dense reconstruction and sampling processing on each voxel to be reconstructed based on a Gaussian process regression model to determine upsampling points; and merging all upsampling points generated at each scale with the original point cloud to output the final dense point cloud. This method adaptively fuses geometric features at different scales, effectively suppressing noise while restoring complex geometric structures on object surfaces, thus improving the accuracy of point cloud upsampling.

[0066] Please see Figure 2 , Figure 3 , Figure 2 This is one of the structural schematic diagrams of a point cloud upsampling device based on a multi-scale Gaussian process provided in an embodiment of this application; Figure 3 This is a second schematic diagram of a point cloud upsampling device based on a multi-scale Gaussian process, provided as an embodiment of this application. Figure 3 As shown, the point cloud upsampling device 200 based on a multi-scale Gaussian process includes: The voxelization construction module 210 is used to acquire the original point cloud, generate multiple scale levels with different spatial resolutions, each scale level corresponds to its own voxel, and establish spatial index association relationships between voxels of different scales. The hierarchical plane detection module 220 is used to determine the plane attributes of voxels at each scale level based on the geometric distribution characteristics of their internal point clouds, and to identify the voxels to be reconstructed. The Gaussian fitting module 230 is used to perform local densification reconstruction and sampling processing on each voxel to be reconstructed based on the Gaussian process regression model, and to determine the upsampling point. The merging module 240 is used to merge all the upsampled points generated at each scale with the original point cloud to output the final dense point cloud.

[0067] Furthermore, the hierarchical planar detection module 220 is used to determine the planar attributes of the voxel meshes at each scale level based on the geometric distribution characteristics of their internal point clouds, thereby identifying the voxels to be reconstructed. For non-empty voxels at coarse and medium scale levels, covariance feature decomposition and flatness threshold determination are used, combined with spatial clustering checks, to screen planar voxels and include them in the set of voxels to be reconstructed, while non-planar voxels are passed down along the hierarchical index to a finer scale for further processing. For non-empty voxels at fine-scale levels, all non-empty voxels are directly identified as voxels to be reconstructed.

[0068] Furthermore, the hierarchical plane detection module 220 is used to screen planar voxels into the set of voxels to be reconstructed by using covariance feature decomposition and flatness threshold determination, combined with spatial clustering checks. The covariance matrix of the voxel in-point set is determined and eigenvalue decomposition is performed to obtain the flatness index. If the flatness index meets the preset flatness threshold condition, spatial continuity clustering check is performed. If the clustering check passes, the voxel is marked as a planar voxel and identified as the voxel to be reconstructed; otherwise, it is marked as a non-planar voxel.

[0069] Furthermore, the hierarchical plane detection module 220 for the spatial continuity clustering check includes: Based on the local neighborhood density distribution of the point cloud within the voxel to be reconstructed, determine whether the point set belongs to the same connected cluster; If the clustering result is a single cluster, then spatial continuity is determined as a planar voxel; otherwise, it is determined as a non-planar voxel.

[0070] Furthermore, the Gaussian fitting module 230 is used to perform local densification reconstruction and sampling processing on each voxel to be reconstructed based on the Gaussian process regression model, and to determine the upsampling point: For each reconstructed voxel, a local spatial coordinate system is established based on its internal point cloud. The projection coordinates of the point cloud on the local tangent plane are used as input, and the projection values ​​in the local normal direction are used as output to construct a training dataset for Gaussian process regression. A Gaussian process regression model is established by selecting the radial basis function as the covariance function and introducing a noise term; In the local spatial coordinate system, predictive sampling of regular grid points is performed inside the voxel to be reconstructed, and the predicted mean and predicted variance of each sampling point are calculated using the Gaussian process regression model. Predicted points are selected based on the variance threshold and the predicted variance, and the predicted points are restricted to be within the current voxel bounding box. The selected predicted points are then transformed from the local spatial coordinate system back to the global coordinate system to generate upsampling points.

[0071] Furthermore, the Gaussian fitting module 230 is used to establish a local spatial coordinate system for the voxel to be reconstructed based on its internal point cloud: A local spatial coordinate system is constructed with the centroid of the point set within the voxel to be reconstructed as the origin, the eigenvector corresponding to the largest eigenvalue of the covariance matrix as the first tangential axis, the eigenvector corresponding to the second largest eigenvalue as the second tangential axis, and the eigenvector corresponding to the smallest eigenvalue as the normal direction axis.

[0072] Furthermore, such as Figure 3 As shown, the point cloud upsampling device 200 based on a multi-scale Gaussian process also includes an attribute assignment module 250, which is used for: Construct an octree spatial index for the original point cloud; For each upsampling point, the octree spatial index is used to quickly search for its nearest neighbor in the original point cloud, and the attributes of the nearest neighbor are assigned to the upsampling point.

[0073] This application provides a point cloud upsampling device based on a multi-scale Gaussian process. The device includes: a voxelization construction module for acquiring the original point cloud, generating multiple scale levels with different spatial resolutions, each scale level corresponding to its own voxels, and establishing spatial index associations between voxels of different scales; a hierarchical plane detection module for determining the planar attributes of voxels at each scale level based on the geometric distribution characteristics of their internal point clouds, thus identifying the voxels to be reconstructed; a Gaussian fitting module for performing local dense reconstruction and sampling processing on each voxel to be reconstructed based on a Gaussian process regression model, determining the upsampling points; and a merging module for merging all upsampling points generated at each scale with the original point cloud to output the final dense point cloud. This device adaptively fuses geometric features at different scales, effectively suppressing noise while restoring the complex geometric structures of object surfaces, thus improving the accuracy of point cloud upsampling.

[0074] Please refer to 4. Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0075] The storage 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 and the storage 420 communicate via a link 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 1 The steps of the point cloud upsampling method based on the multi-scale Gaussian process in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0076] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the point cloud upsampling method based on multi-scale Gaussian process in the method embodiment shown are specifically implemented in the method embodiment and will not be repeated here.

[0077] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0078] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0079] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0080] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0081] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A point cloud upsampling method based on a multi-scale Gaussian process, characterized in that, The point cloud upsampling method includes: The original point cloud is acquired, and multiple scale levels with different spatial resolutions are generated. Each scale level corresponds to its own voxel, and spatial index associations between voxels of different scales are established. For voxels at each scale level, planar attributes are determined based on the geometric distribution characteristics of their internal point clouds to identify the voxels to be reconstructed. Based on the Gaussian process regression model, local dense reconstruction and sampling processing are performed on each voxel to be reconstructed to determine the upsampling point; All upsampled points generated at each scale are merged with the original point cloud to output the final dense point cloud.

2. The point cloud upsampling method according to claim 1, characterized in that, The process of determining the voxels to be reconstructed by analyzing the planar attributes of the voxel meshes at each scale level based on the geometric distribution characteristics of their internal point clouds includes: For non-empty voxels at coarse and medium scale levels, covariance feature decomposition and flatness threshold determination are used, combined with spatial clustering checks, to screen planar voxels and include them in the set of voxels to be reconstructed, while non-planar voxels are passed down along the hierarchical index to a finer scale for further processing. For non-empty voxels at fine-scale levels, all non-empty voxels are directly identified as voxels to be reconstructed.

3. The point cloud upsampling method according to claim 2, characterized in that, The process of selecting planar voxels to be reconstructed by using covariance eigenvalue decomposition and flatness threshold determination, combined with spatial clustering checks, includes: The covariance matrix of the voxel in-point set is determined and eigenvalue decomposition is performed to obtain the flatness index. If the flatness index meets the preset flatness threshold condition, spatial continuity clustering check is performed. If the clustering check passes, the voxel is marked as a planar voxel and identified as the voxel to be reconstructed; otherwise, it is marked as a non-planar voxel.

4. The point cloud upsampling method according to claim 3, characterized in that, The spatial continuity clustering check includes: Based on the local neighborhood density distribution of the point cloud within the voxel to be reconstructed, determine whether the point set belongs to the same connected cluster; If the clustering result is a single cluster, then spatial continuity is determined as a planar voxel; otherwise, it is determined as a non-planar voxel.

5. The point cloud upsampling method according to claim 1, characterized in that, The Gaussian process regression model is used to perform local dense reconstruction and sampling for each voxel to be reconstructed, determining the upsampling points, including: For each reconstructed voxel, a local spatial coordinate system is established based on its internal point cloud. The projection coordinates of the point cloud on the local tangent plane are used as input, and the projection values ​​in the local normal direction are used as output to construct a training dataset for Gaussian process regression. A Gaussian process regression model is established by selecting the radial basis function as the covariance function and introducing a noise term; In the local spatial coordinate system, predictive sampling of regular grid points is performed inside the voxel to be reconstructed, and the predicted mean and predicted variance of each sampling point are calculated using the Gaussian process regression model. Predicted points are selected based on the variance threshold and the predicted variance, and the predicted points are restricted to be within the current voxel bounding box. The selected predicted points are then transformed from the local spatial coordinate system back to the global coordinate system to generate upsampling points.

6. The point cloud upsampling method according to claim 5, characterized in that, The establishment of a local spatial coordinate system for the voxel to be reconstructed based on its internal point cloud includes: A local spatial coordinate system is constructed with the centroid of the point set within the voxel to be reconstructed as the origin, the eigenvector corresponding to the largest eigenvalue of the covariance matrix as the first tangential axis, the eigenvector corresponding to the second largest eigenvalue as the second tangential axis, and the eigenvector corresponding to the smallest eigenvalue as the normal direction axis.

7. The point cloud upsampling method according to claim 1, characterized in that, Before merging all upsampled points generated at each scale with the original point cloud to output the final dense point cloud, the point cloud upsampling method further includes: Construct an octree spatial index for the original point cloud; For each upsampling point, the octree spatial index is used to quickly search for its nearest neighbor in the original point cloud, and the attributes of the nearest neighbor are assigned to the upsampling point.

8. A point cloud upsampling device based on a multi-scale Gaussian process, characterized in that, The point cloud upsampling device includes: The voxelization construction module is used to acquire the original point cloud, generate multiple scale levels with different spatial resolutions, each scale level corresponds to its own voxel, and establish spatial index association relationships between voxels of different scales. The hierarchical planar detection module is used to determine the planar attributes of voxels at each scale level based on the geometric distribution characteristics of their internal point clouds, and to identify the voxels to be reconstructed. The Gaussian fitting module is used to perform local densification reconstruction and sampling processing on each voxel to be reconstructed based on the Gaussian process regression model, and to determine the upsampling point. The merging module is used to merge all upsampled points generated at each scale with the original point cloud to output the final dense point cloud.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the point cloud upsampling method based on a multi-scale Gaussian process as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the point cloud upsampling method based on a multi-scale Gaussian process as described in any one of claims 1 to 7.

Images