A three-dimensional point cloud data reconstruction processing method and system

Abstract

The present application relates to the field of automatic driving and robot environment perception technology, in particular to a three-dimensional point cloud data reconstruction processing method and system, comprising: obtaining original sparse point cloud data and sensor scanning trajectory information; and based on local geometric tensor and density distribution, constructing a local feature entropy field, which is used to quantitatively represent the information disorder degree and feature saliency of the local area of the point cloud; based on the local feature entropy field, reconstructing the confidence weight for representing the probability of each point belonging to the real geometric feature or random noise; an anisotropic confidence evaluation matrix, and a multi-scale adaptive grid growth including dynamically adjusting the grid size and topological structure according to the reconstructed confidence weight; combining the original sparse point cloud data and the initial three-dimensional reconstruction model, and based on the semantic residual, locally refining the initial three-dimensional reconstruction model to output the final three-dimensional model; the present application effectively avoids the misjudgment and filtering of high-frequency sparse key features.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving and robot environmental perception technology, specifically a three-dimensional point cloud data reconstruction processing method and system. Background Technology

[0002] 3D point cloud data reconstruction is mainly used for environmental perception and modeling of equipment such as unmanned delivery logistics vehicles or industrial inspection robots. Its technical essence is to process the raw sparse point cloud data collected by sensors to generate a continuous 3D physical model.

[0003] Existing point cloud reconstruction and filtering techniques typically employ globally indiscriminate smoothing algorithms or single-resolution mesh generation methods. These traditional methods can achieve basic 3D morphological construction and eliminate some environmental noise, addressing the initial smoothing and basic noise reduction issues of point cloud data in conventional scenarios. However, in unstructured road driving scenarios, sensors often experience sampling non-uniformity interference due to vehicle motion. Existing global smoothing and single-resolution algorithms suffer from dynamic mismatch and computational redundancy limitations, easily leading to the misjudgment and filtering of high-frequency sparse key geometric features. Furthermore, the unavoidable low-pass smoothing effect in conventional mesh generation often smooths real physical edges into rounded corners, causing the loss of key topological structures. In addition, traditional global reconstruction error correction mechanisms consume enormous amounts of computational power.

[0004] Therefore, how to avoid filtering out sparse key features, suppress the smooth transition of physical edges, and achieve adaptive and efficient allocation and fine reconstruction of computing resources in complex scenarios has become a problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for reconstructing three-dimensional point cloud data, and to solve the following technical problems:

[0006] It breaks through the dynamic mismatch problem of indiscriminate processing of traditional global filtering, avoids the misjudgment and filtering out of high-frequency sparse key features by traditional smoothing algorithms, thereby effectively suppressing reconstruction distortion and improving the decision robustness of downstream environmental perception tasks in complex logistics scenarios.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] A method for reconstructing 3D point cloud data includes the following steps:

[0009] The process involves acquiring raw sparse point cloud data and sensor scanning trajectory information; for target points in the raw sparse point cloud data, calculating the rate of change of the normal vector and the curvature tensor as local geometric tensors, and combining this with the sensor scanning trajectory information to calculate the reciprocal of the local point cloud density as the density distribution; and constructing a local feature entropy field by weighting the normal vector rate of change, the curvature tensor, and the reciprocal of the density, wherein the local feature entropy field is used to quantify the information disorder and feature saliency of local regions of the point cloud.

[0010] Based on the local feature entropy field, the reconstruction confidence weight of each point in the original sparse point cloud data is calculated to generate an anisotropic confidence evaluation matrix, wherein the reconstruction confidence weight is used to characterize the probability that each point belongs to the true geometric features or random noise.

[0011] Based on the anisotropic confidence assessment matrix, multi-scale adaptive mesh growth is performed to generate an initial 3D reconstruction model, wherein the multi-scale adaptive mesh growth includes dynamically adjusting the mesh size and topology according to the reconstruction confidence weight;

[0012] By combining the original sparse point cloud data with the initial 3D reconstruction model, semantic residuals are calculated, and the initial 3D reconstruction model is locally refined based on the semantic residuals to output the final 3D model.

[0013] Preferably, the local geometric tensor and density distribution of the original sparse point cloud data are calculated, and a local feature entropy field is constructed based on the local geometric tensor and the density distribution, including:

[0014] For each target point in the original sparse point cloud data, search for its nearest neighbor set within a preset neighborhood.

[0015] The rate of change of the normal vector and the curvature tensor of the target point and the set of nearest neighbors are calculated as the local geometric tensor, and the local point cloud density of the target point is calculated as the density distribution in combination with the sensor scanning trajectory information;

[0016] Based on the normal vector change rate, the curvature tensor, and the local point cloud density, the feature entropy value of the target point is calculated, and the feature entropy values ​​of all target points are mapped into a three-dimensional salient feature map as the local feature entropy field.

[0017] Preferably, based on the local feature entropy field, the reconstruction confidence weight of each point in the original sparse point cloud data is calculated to generate an anisotropic confidence evaluation matrix, including:

[0018] Regions in the three-dimensional saliency feature map with entropy values ​​less than a preset feature entropy threshold are identified as low-entropy regions, and regions with feature entropy values ​​greater than or equal to the preset feature entropy threshold are identified as high-entropy regions.

[0019] For the high-entropy region, the eigenvalues ​​of the covariance matrix of the nearest neighbor set are further extracted to determine its topological continuity. If the proportion of the largest eigenvalue in the sum of eigenvalues ​​exceeds the continuity lower limit and satisfies the preset topological continuity condition, it is determined to be a complex feature region and given a high confidence weight. If the proportion of the largest eigenvalue does not satisfy the preset topological continuity condition, it is determined to be an outlier noise region and given a low confidence weight.

[0020] The low-entropy region is determined to be a flat safe region and assigned a medium confidence weight to form the anisotropic confidence assessment matrix.

[0021] Preferably, based on the anisotropy confidence evaluation matrix, multi-scale adaptive mesh growth is performed, including:

[0022] In the flat, safe area, a first-scale grid is used for rapid fitting to reduce data storage requirements;

[0023] In the complex feature region, a second-scale mesh is used for fine fitting, the mesh size is reduced to construct physical edges in situ, and an edge constraint operator is introduced to calculate the maximum direction of the local point cloud normal vector gradient. Based on this direction, a geometric offset is applied to the mesh vertices to force the second-scale mesh to fit the high-frequency feature edges, wherein the second scale is smaller than the first scale.

[0024] In the outlier noise region, mesh generation is suppressed according to the low confidence weight to remove background noise.

[0025] Preferably, the semantic residual is calculated by combining the original sparse point cloud data with the initial 3D reconstruction model, including:

[0026] Based on the sensor scanning trajectory information, the original acquisition viewpoint is determined, and the initial 3D reconstruction model is projected back to the original acquisition viewpoint to generate a reconstruction depth map;

[0027] The original sparse point cloud data is projected onto the original acquisition viewpoint to generate an original point cloud projection map, and the geometric deviation between the reconstructed depth map and the original point cloud projection map under the corresponding original acquisition viewpoint is calculated. The key topological features of the reconstructed depth map and the original point cloud projection map are extracted respectively, and the matching degree of the key topological features is extracted.

[0028] The semantic residual is generated by weighting the geometric deviation with the matching degree, wherein the semantic residual is used to indicate regions where key features are lost or overly smoothed.

[0029] Preferably, the initial 3D reconstruction model is locally refined based on the semantic residual, including:

[0030] In regions where the semantic residual is greater than or equal to a preset residual threshold, a local reconstruction mechanism is automatically triggered.

[0031] Increase the grid resolution within the region and adjust the reconstruction confidence weights, then perform the multi-scale adaptive grid growth again until the semantic residual is less than the preset residual threshold.

[0032] Preferably, the original sparse point cloud data comes from autonomous driving environmental perception devices or handheld 3D scanning devices;

[0033] The final 3D model is used for obstacle identification or reverse engineering modeling in downstream tasks.

[0034] A 3D point cloud data reconstruction and processing system includes the following modules:

[0035] The entropy field construction module is used to acquire raw sparse point cloud data and sensor scanning trajectory information; calculate the local geometric tensor and density distribution of the raw sparse point cloud data, and construct a local feature entropy field based on the local geometric tensor and density distribution, wherein the local feature entropy field is used to quantify the information disorder and feature saliency of local regions of the point cloud.

[0036] The weight evaluation module is used to calculate the reconstruction confidence weight of each point in the original sparse point cloud data based on the local feature entropy field, and generate an anisotropic confidence evaluation matrix, wherein the reconstruction confidence weight is used to characterize the probability that each point belongs to the true geometric features or random noise.

[0037] The mesh reconstruction module is used to perform multi-scale adaptive mesh growth based on the anisotropic confidence evaluation matrix to generate an initial three-dimensional reconstruction model, wherein the multi-scale adaptive mesh growth includes dynamically adjusting the mesh size and topology according to the reconstruction confidence weight;

[0038] The feedback optimization module is used to combine the original sparse point cloud data with the initial 3D reconstruction model, calculate the semantic residual, and refine the initial 3D reconstruction model locally based on the semantic residual to output the final 3D model.

[0039] The beneficial effects of this invention are:

[0040] 1) This invention constructs a local feature entropy field by calculating the local geometric tensor and density distribution, and generates an anisotropic confidence evaluation matrix accordingly. This mechanism quantitatively characterizes the information disorder and feature entropy field of the local region, breaking the dynamic mismatch problem caused by the indiscriminate processing of traditional global filtering. By reconstructing the confidence weight, differentiated computing power allocation is achieved, effectively avoiding the misjudgment and filtering out of high-frequency sparse key features.

[0041] 2) This invention performs multi-scale adaptive mesh growth based on the anisotropic confidence evaluation matrix; it uses large-scale fast fitting in flat safe areas, and small-scale fine fitting in complex feature areas and introduces edge constraint operators; the edge constraint operators force the mesh to fit high-frequency edges, breaking the conventional low-pass smoothing effect and preventing physical edges from being smoothed into rounded corners, thereby accurately preserving the key topological structure.

[0042] 3) This invention combines the original data with the initial model to calculate semantic residuals, which are used to indicate the feature loss area and perform local refinement; the local reconstruction mechanism is automatically triggered only in areas where the semantic residual is greater than or equal to a preset threshold, thereby improving the local grid resolution and confidence weight; this closed-loop mechanism avoids the huge computational power consumption caused by global recalculation, and accurately focuses resources on the feature damage area, thereby achieving efficient allocation of computational power. Attached Figure Description

[0043] The invention will now be further described with reference to the accompanying drawings;

[0044] Figure 1 This is a flowchart illustrating a three-dimensional point cloud data reconstruction processing method provided in an embodiment of this application;

[0045] Figure 2 This is a schematic diagram of a three-dimensional point cloud data reconstruction processing system provided in an embodiment of this application. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Please see Figure 1 This embodiment provides a three-dimensional point cloud data reconstruction processing method and system, the execution subject of which is a computing unit mounted in an unmanned delivery logistics vehicle or an industrial inspection robot; the system acquires raw sparse point cloud data and sensor scanning trajectory information;

[0048] The sensor scanning trajectory information includes six-degree-of-freedom pose data over time, aiming to eliminate sampling non-uniformity interference caused by the vehicle driving on unstructured roads in the logistics park; the system calculates the local geometric tensor and density distribution of the original sparse point cloud data, and constructs a local feature entropy field based on the local geometric tensor and density distribution; the local feature entropy field quantitatively characterizes the feature uncertainty and feature entropy field of the local region of the point cloud, breaking the dynamic mismatch problem of indiscriminate processing of traditional global filtering;

[0049] In the flat asphalt areas of the logistics park, the point cloud arrangement follows a strict pattern and the feature uncertainty is at a low level. However, at the edges of scattered cardboard boxes or the sharp chamfers of mechanical parts, the geometric shape changes drastically, and the feature uncertainty increases. Based on the local feature entropy field, the system calculates the reconstruction confidence weight of each point in the original sparse point cloud data and generates an anisotropic confidence assessment matrix.

[0050] The reconstruction confidence weight represents the probability that each point belongs to the true geometric feature or random noise. This matrix enables differentiated allocation of computing resources and avoids the misjudgment and filtering out of high-frequency sparse key features by traditional smoothing algorithms. Based on the anisotropic confidence evaluation matrix, the system performs multi-scale adaptive mesh growth to generate the initial three-dimensional reconstruction model.

[0051] Multi-scale adaptive mesh growth dynamically adjusts the mesh size and topology based on the reconstruction confidence weights. In low-weight regions with redundant information, the mesh size is enlarged to reduce memory overhead, while in information-dense feature regions, the mesh size is reduced to construct physical edges in situ.

[0052] By combining the original sparse point cloud data with the initial 3D reconstruction model, the system calculates the semantic residual and refines the initial 3D reconstruction model locally based on the semantic residual, outputting the final 3D model. This process suppresses reconstruction distortion and improves the decision robustness of downstream environmental perception tasks in complex logistics scenarios.

[0053] In a preferred embodiment of the present invention, this embodiment is a further specification of the step of constructing a local feature entropy field; for each target point in the original sparse point cloud data, the system uses a spatial index structure to search for the set of nearest neighbors in its preset neighborhood;

[0054] In this process, the search radius of the preset neighborhood is not a fixed constant, but is dynamically deduced by multiplying the inherent angular resolution of the sensor with the absolute distance from the target point to the sensor's origin, thereby ensuring that a stable number of nearest neighbors can be identified in both sparse regions at long distances and dense regions at close distances. The system calculates the rate of change of the normal vector and the curvature tensor of the target point and the set of nearest neighbors as local geometric tensors. This operation aims to capture the local curvature of the surface, and the region with the larger the normal vector deflection angle contains richer geometric and topological information.

[0055] The system combines sensor scanning trajectory information to calculate the local point cloud density of the target point as the density distribution. Since the beam emitted by the lidar has a fixed angular resolution, the point cloud of the object surface is naturally sparser the farther away from the sensor. The system extracts the absolute distance from the target point to the sensor's emission origin as the basic input. To prevent the risk of division by zero overflow at extremely close distances, the system adds the absolute distance to the sensor's inherent minimum blind zone distance and uses the sum of the two as a compensation coefficient to multiply into the initial density value, thereby eliminating false density fluctuations caused by the detection distance and providing a stable calculation benchmark for subsequent reciprocal operations.

[0056] Based on the rate of change of the normal vector, the curvature tensor, and the local point cloud density, the system calculates the feature entropy value of the target point. The specific calculation logic is as follows: assign a positive basic weight to the rate of change of the normal vector to characterize the positive contribution of geometrical abrupt changes to feature uncertainty; extract the eigenvalues ​​of the curvature tensor to calculate the principal curvature of the surface as a scalar measure, and assign it a positive morphological weight to capture the local severe bending structure; at the same time, assign a positive compensation weight to the inverse of the compensated local point cloud density, so that the feature entropy value is amplified accordingly in the sparse point cloud region.

[0057] In this weighted solution, the system adopts a quantization allocation strategy that combines static and dynamic methods. Since the rate of change of the normal vector is the most direct physical quantity reflecting local geometric changes, the system assigns it a fixed base weight of 0.5. The morphological weight and compensation weight are dynamically allocated based on the overall sparsity of the current point cloud frame, and the sum of the two is strictly maintained at 0.5.

[0058] In the specific simulation, the system no longer relies on the global average density, but instead calculates the local density benchmark based on the spatial layering of the target point. The system uses the absolute distance from the sensor's origin to dynamically divide the three-dimensional space of the current frame into multiple sub-regions: near field, mid field, and far field, and extracts the average point cloud density of each sub-region as the regional density benchmark. When the regional density benchmark of the target point's region is higher than the preset density threshold, it indicates that the local region has been sufficiently sampled. The system increases the shape weight of the target point to 0.4 and reduces the compensation weight to 0.1.

[0059] Conversely, when the regional density benchmark of the target point is lower than the preset sparsity threshold, the system reduces the shape weight of the target point to 0.1 and increases the compensation weight to 0.4, thereby amplifying the impact of density changes on feature uncertainty.

[0060] When the regional density benchmark of the target point's region is between the preset sparse threshold and dense threshold, the system uses linear interpolation to dynamically calculate the morphological weight and compensation weight to ensure that the sum of the two is strictly kept at 0.5, thereby achieving a smooth transition of weights.

[0061] Before performing the final weighted summation, in order to eliminate the arithmetic conflict caused by the three different physical quantities of the normal vector change rate, principal curvature and density reciprocal, the system performs dimensionless mapping on these three physical quantities respectively.

[0062] Specifically, the system extracts the global maximum and minimum values ​​of the normal vector change rate, principal curvature, and inverse density of all target points in the current point cloud frame, and uses range scaling logic to linearly map these three physical quantities of the target points to a standardized range of 0 to 1. The system sums the weighted values ​​obtained by multiplying the normalized normal vector change rate by the basic weight, the weighted values ​​obtained by multiplying the normalized principal curvature by the shape weight, and the weighted values ​​obtained by multiplying the normalized inverse density by the compensation weight to obtain the final feature entropy value.

[0063] Under this compensation and weighting mechanism, the compensation coefficient is positively correlated with the absolute distance. When the distance is increased, the compensation coefficient increases synchronously to offset the physical sparsity caused by beam divergence. When a point cloud has both a large normal vector deflection and a very high principal curvature, such as exhibiting acute geometric features, and the local point cloud density is extremely low due to the long detection distance, i.e., the inverse of density is extremely large, the system will deduce and calculate a very high feature entropy value according to the weighted flow rule of the positive weights mentioned above.

[0064] This quantization and calculation mechanism ensures that tiny, critical features at a distance are accurately identified as high-information regions, preventing them from being misjudged and filtered out as sparse noise. The solution process achieves a comprehensive weighted evaluation of geometric severity and sampling reliability. The system maps the feature entropy values ​​of all target points into a three-dimensional saliency feature map, providing underlying data support for subsequent nonlinear cascade reconstruction.

[0065] In a preferred embodiment of the present invention, this embodiment is a further specification of the step of generating anisotropic confidence assessment matrix; the system divides the three-dimensional saliency feature map into regions through a multi-level dynamic threshold mechanism, identifies regions in the three-dimensional saliency feature map whose feature entropy value is less than a preset feature entropy threshold as low-entropy regions, and identifies regions whose feature entropy value is greater than or equal to the preset feature entropy threshold as high-entropy regions;

[0066] In this segmentation logic, the preset feature entropy threshold is not a fixed empirical value, but rather a distribution histogram of feature entropy values ​​of all target points in the current frame's 3D salient feature map. The maximum inter-class variance method is used to dynamically calculate the segmentation value that maximizes the variance between the background flat area and the foreground feature area, which is then used as the preset feature entropy threshold to avoid environmental mismatch.

[0067] For high-entropy regions, the system further judges their topological continuity to distinguish between real physical edges and random outliers. The system performs principal component analysis on the nearest neighbor point set in the high-entropy region to extract the eigenvalues ​​of the covariance matrix. If the proportion of the largest eigenvalue in the sum of eigenvalues ​​exceeds the continuity lower limit, it indicates that the point cloud in the region has obvious linear extension or planar extension. The system determines that it meets the preset topological continuity condition, marks it as a complex feature region and assigns it a high confidence weight, forcing subsequent algorithms to retain key features such as the edge of a logistics vehicle bumper.

[0068] The continuity lower limit is based on the prior setting of the sensor's sampling resolution and the physical structure of typical obstacles; the proportion of the largest eigenvalue of an ideal linear distribution approaches the theoretical value of 1, while the three eigenvalues ​​of a completely randomly diverging noise sphere tend to be equal, with a proportion close to the theoretical value of 0.33.

[0069] The system sets the continuity lower limit strictly between 0.33 and 1; specifically, based on the prior scan data of typical linear obstacles in logistics parks, such as isolation wire mesh or fences, the system statistically analyzes the distribution pattern of the eigenvalues ​​of their covariance matrix. In actual business simulation, the system sets the continuity lower limit to 0.75.

[0070] When feature extraction reveals that the proportion of the largest feature value exceeds the continuity lower limit of 0.75, the system determines that the point cloud has a strong directional clustering in a single dimension and confirms it as a real physical edge; if the proportion of the largest feature value does not reach the continuity lower limit of 0.75, it indicates that the point cloud in this area is spherically divergent and has no obvious directionality. The system determines that it does not meet the preset topological continuity condition, marks it as an outlier noise area and assigns it a low confidence weight to suppress the interference of floating noise points caused by rain and snow.

[0071] For the low-entropy region, due to its high consistency of normal vectors, the system determines it as a flat safe region and assigns it a medium confidence weight, thereby forming an anisotropic confidence evaluation matrix. This embodiment demonstrates the adaptability of feature extraction under complex meteorological conditions and verifies the effectiveness of the matrix assignment strategy.

[0072] In a preferred embodiment of the present invention, this embodiment is a further specification of the multi-scale adaptive mesh growth step; the system performs multi-scale adaptive mesh growth based on the anisotropic confidence evaluation matrix, which changes the limitations of computational redundancy caused by the traditional single resolution; in the flat safe area, the system uses the first-scale mesh for fast fitting to reduce the amount of data storage and uses large-scale patches to cover redundant planar data.

[0073] In complex feature regions, the system uses a second-scale grid for fine fitting and introduces an edge constraint operator to force the second-scale grid to fit the high-frequency feature edges. The second scale is smaller than the first scale. The edge constraint operator calculates the maximum direction of the local point cloud normal vector gradient during the grid vertex generation stage and forces the vertices of the second-scale grid to apply a geometric offset to the maximum gradient direction to prevent physical edges from being smoothed into rounded corners.

[0074] The core technical motivation of the edge constraint operator is to break the unavoidable low-pass smoothing effect in conventional mesh generation. In the specific deduction, the system extracts the eigenvector corresponding to the largest eigenvalue of the normal gradient matrix as the direction of the maximum gradient, which clearly points to the steepest crease region of the geometric surface in physical space.

[0075] The system sets the magnitude of the geometric offset to be dynamically positively correlated with the magnitude of the normal vector gradient. Specifically, the system uses nonlinear mapping logic to determine the geometric offset, takes the magnitude of the normal vector gradient as input, and outputs the normalized offset coefficient through a sigmoid activation mechanism.

[0076] In the quantitative derivation of this S-shaped activation mechanism, the system constructs a mapping rule with clear physical boundaries; the system sets a reference gradient magnitude that represents a typical right-angle edge as the activation center point, calculates the difference between the magnitude of the input normal vector gradient and the reference gradient magnitude, and multiplies it by a preset scaling factor to control the steepness of the activation curve, and constructs a smooth transition logic based on the exponential operation of the natural constant.

[0077] Specifically, the system presets the scaling factor to 5.0 to ensure that the normalized offset coefficient can be quickly flipped within a very small range of gradient magnitude deviation from the reference value; in particular, when the input gradient magnitude is exactly equal to the reference gradient magnitude, the activation mechanism strictly outputs a normalized offset coefficient of 0.5.

[0078] When the input gradient magnitude is lower than the baseline and the difference continues to increase, the output value decays smoothly and monotonically and approaches 0 infinitely, avoiding microscopic distortion in flat regions; when the input gradient magnitude increases sharply and is higher than the baseline, the output value increases smoothly and monotonically and approaches 1 infinitely, driving the mesh vertices to produce significant displacement, thereby accurately anchoring the mesh vertices that were originally scattered in the beveled and smooth transition area to the real physical edge structure, realizing the in-situ restoration of key edges from the topological bottom layer; multiplying this coefficient by the average side length of the current mesh yields the absolute geometric offset;

[0079] Meanwhile, to prevent excessive mesh vertex offset due to extreme local gradients, which could lead to patch self-intersection or topology flip, the system applies a physical boundary interception mechanism to the geometric offset; the system extracts the average side length of the current second-scale mesh in real time and forcibly truncates the maximum allowed geometric offset to 0.5 times the average side length.

[0080] This dynamic limiting strategy based on its own scale ensures that vertices are always safely confined within a legal topological bounding box as they move toward the real physical edge. In outlier noise regions, the system suppresses mesh generation based on low confidence weights to remove background noise, determines that there is no need for reconstruction in the region, and directly cuts off the triangulation process in the region. This mechanism realizes nonlinear adaptive allocation of computing power in spatial distribution.

[0081] In a preferred embodiment of the present invention, this embodiment is a further specification of the step of calculating semantic residuals; the system combines the original sparse point cloud data with the initial 3D reconstruction model to calculate semantic residuals in order to verify whether the reconstruction model has lost key physical properties;

[0082] Based on the sensor scanning trajectory information, the system projects the initial 3D reconstruction model back to the original acquisition viewpoint to generate a dense reconstruction depth map; at the same time, it projects the original sparse point cloud data to the corresponding original acquisition viewpoint, and uses morphological closing operation or inverse distance weighted interpolation algorithm to densify the discrete depth holes generated by the projection, generating a continuous original point cloud projection map.

[0083] The system extracts the absolute difference between the depth values ​​at corresponding pixel coordinates between the reconstructed depth map and the original point cloud projection map as the geometric deviation. It uses the edge detection operator to extract the edge gradient direction of the reconstructed depth map and the original point cloud projection map respectively, and calculates the cosine similarity of the gradient vectors of the two as the matching degree.

[0084] In obstacle avoidance scenarios for unmanned delivery vehicles, the system not only considers the absolute deviation of depth at a single point, but also whether topological features such as thin wire mesh or small reflective cones match the original input.

[0085] The system generates semantic residuals by weighting geometric deviation and matching degree. The semantic residuals are used to indicate areas where key features are lost or over-smoothed. The specific weighting calculation logic is as follows: before weighting, the system divides the geometric deviation, i.e., the absolute difference, by the maximum bounding box size of the spatial depth of the viewpoint to achieve a dimensionless mapping of the geometric deviation, ensuring that its value strictly falls within the range of 0 to 1. This eliminates the interference of the absolute depth span of near and far objects on the semantic residual results, making it compatible with the matching degree in weighted calculation.

[0086] The system assigns positive geometric weights to the dimensionless geometric deviations to quantify the degree of spatial position distortion. Simultaneously, it extracts the difference between the perfect match state and the cosine similarity as a topological distortion index and assigns it positive topological weights. In this weighted calculation, the geometric and topological weights are not static constants but are dynamically modulated by the depth span of the current viewpoint. The system extracts the maximum depth value under the current projection viewpoint. When the maximum depth value is small, it indicates that the obstacle is close at hand. The system increases the geometric weights and correspondingly decreases the topological weights because the risk of minor collisions in absolute spatial position is higher at close range.

[0087] Conversely, when the maximum depth value is large, it represents long-distance detection. Due to the increase in the inherent ranging variance of lidar, the reliability of absolute geometric depth decreases. The system automatically reduces the geometric weight and simultaneously increases the topological weight, making the system rely more on the topological shape of the object to determine the reconstruction quality at long distances.

[0088] In the quantization deduction of this dynamic modulation mechanism, the system sets a constraint that the sum of geometric weights and topological weights is always 1, and introduces the effective detection distance of the sensor as a quantization benchmark; the system calculates the ratio of the current maximum depth value to the effective detection distance, and strictly limits the ratio to the range of 0 to 1; the system directly uses the ratio as the value of the topological weight, and uses the result of subtracting the ratio from 1 as the value of the geometric weight;

[0089] In this embodiment, through this quantized linear mapping design, when the maximum depth value approaches the very close distance, the geometric weight approaches the theoretical upper limit of 1, and the topological weight approaches 0, and the system relies entirely on the absolute geometric deviation for residual evaluation; when the maximum depth value reaches or exceeds the effective detection distance, the ratio is truncated to 1, at which point the topological weight reaches the upper limit of 1, the geometric weight drops to 0, and the system relies entirely on the feature matching degree for evaluation.

[0090] The system adds the two weighted results to generate semantic residuals; this index marks the critical state of the model's local fidelity; in this quantization inference mechanism, the perfect matching state is strictly defined as the theoretical upper limit that the edge gradient of the reconstructed depth map is ideally parallel to the edge gradient of the original point cloud projection map, that is, the state where the cosine similarity value is 1; this quantization linear mapping is designed to subtract the actual calculated cosine similarity value from 1.

[0091] If the reconstructed surface undergoes severe structural distortion, causing the reconstructed edge gradient to be orthogonal or even inversely related to the original true gradient, the difference will be significantly amplified, driving a sharp increase in the semantic residual value after topological weighting. This quantization boundary decomposition based on vector direction similarity transforms abstract topological distortion into deterministic values ​​within the range of 0 to 2, enabling the system to accurately capture and significantly amplify the microscopic deviations of the reconstructed model at weak features such as thin wire mesh.

[0092] In a preferred embodiment of the present invention, this embodiment is a further specification of the local refinement step; based on semantic residuals, the system executes a precise error correction closed loop; the system scans the semantic residuals of each region, and automatically triggers a local reconstruction mechanism in regions where the semantic residuals are greater than or equal to a preset residual threshold; wherein, the preset residual threshold is not arbitrarily specified, but is a fixed boundary value determined by the system through offline statistics based on the maximum tolerance of geometric distortion to the downstream obstacle recognition task.

[0093] This mechanism avoids the huge computational cost of global reconstruction, focusing computing resources only on the local bounding box where features are damaged; the system increases the grid resolution within the region and adjusts the reconstruction confidence weights, dividing the current semantic residual value by a preset residual threshold to obtain a gain coefficient, and multiplying the original reconstruction confidence weights by this gain coefficient to improve the fidelity priority of the point cloud in that region.

[0094] Specifically, if the preset residual threshold is set to 0.15, when the actual semantic residual of a certain region reaches 0.30, the system calculates a gain coefficient of 2.0, and then multiplies the original reconstruction confidence weight of the region, for example, the initial evaluation value of 0.4, by the gain coefficient to update it to 0.8, thereby significantly improving the resource allocation priority of the local bounding box in subsequent reconstruction. During the weight adjustment calculation, the system sets a theoretical confidence upper limit of 1. If the product of the original reconstruction confidence weight and the gain coefficient exceeds the upper limit, the system forcibly executes a truncation strategy to anchor the weight at the upper limit of 1, preventing single-point abnormal collapse of computing power allocation in the iterative optimization.

[0095] Meanwhile, the system does not blindly increase the grid resolution within the region, but rather dynamically reduces the subdivision edge length by a fixed attenuation ratio at the current grid scale, for example, reducing the current grid scale to 0.5 times the original scale;

[0096] The system uses the updated high-resolution parameters and high-confidence weights to perform multi-scale adaptive grid growth again until the semantic residual is less than the preset residual threshold; the stepwise halving of the grid scale and the strict truncation mechanism of the weights initially construct the convergence system of local reconstruction.

[0097] To prevent the risk of semantic residuals failing to fall below the threshold in extremely complex regions with continuous optical interference, leading to infinite subdivision and memory overflow, the system has set up an additional limit protection mechanism based on the physical resolution limit of the sensor.

[0098] When the mesh scale is dynamically reduced multiple times and reaches the preset minimum physical safety boundary size, the system will forcibly terminate the iterative growth instructions within the local bounding box and freeze the current topology state, regardless of whether the semantic residual meets the standard.

[0099] In this extreme protection mechanism, the preset minimum physical safety boundary size is strictly equal to the theoretical lower limit of the actual sampling point spacing of the sensor at the current distance. When the grid density touches this limit, it indicates that there are no real physical details to extract. The forced freeze operation effectively prevents meaningless sub-pixel level interpolation. This resource control mechanism not only avoids endless iterative fitting, but also ensures the balance between memory security and geometric reconstruction accuracy of the vehicle terminal system when dealing with extreme long-tail conditions.

[0100] In a preferred embodiment of the present invention, this embodiment is a further specification of the application scenario; the original sparse point cloud data comes from an autonomous driving environment perception device or a handheld 3D scanning device, and the final 3D model is used for obstacle recognition or reverse engineering modeling in downstream tasks; on the vehicle terminal interaction interface of the unmanned delivery logistics vehicle, the system presents functional partitions, the left half of the interface uses a global perspective to render the 3D model of the basic road surface and large buildings, and the right half of the interface peels off and highlights the complex feature areas that trigger the local refinement mechanism, such as tire fragments left on the road surface;

[0101] When the system continuously identifies a large area of ​​outlier noise at a specific angle, it determines that there is physical occlusion and automatically generates a lens mud and dirt adhesion maintenance prompt; the system collects specific scene data that frequently triggers high semantic residuals during its life cycle, such as speed bumps made of new reflective materials, and packages them back to the cloud server.

[0102] The cloud utilizes long-tail operating data to calibrate the parameters in the local feature entropy field calculation formula offline, and distributes updates via over-the-air download technology, enabling the adaptive capability of the reconstruction algorithm to continuously evolve in actual business operations.

[0103] Please see Figure 2 A three-dimensional point cloud data reconstruction and processing system for executing the above method, comprising four core collaborative modules;

[0104] The entropy field construction module acquires the original sparse point cloud data and sensor scanning trajectory information, calculates the local geometric tensor and density distribution of the original sparse point cloud data, and constructs a local feature entropy field based on the local geometric tensor and density distribution, transforming the disordered point cloud coordinates into a three-dimensional saliency feature map with physical meaning; the weight evaluation module calculates the reconstruction confidence weight of each point in the original sparse point cloud data based on the local feature entropy field, generates an anisotropic confidence evaluation matrix, and acts as a strategy allocator in the system to determine the tilt direction of computing power resources;

[0105] The mesh reconstruction module performs multi-scale adaptive mesh growth based on the anisotropic confidence evaluation matrix to generate an initial 3D reconstruction model, and controls the underlying graphics interface to complete the topological transformation from discrete point cloud to continuous triangular facets; the feedback optimization module combines the original sparse point cloud data and the initial 3D reconstruction model to calculate semantic residuals, and performs local refinement of the initial 3D reconstruction model based on the semantic residuals to output the final 3D model; the modules transfer tensor data with zero copy through video memory, which improves the response latency of complex nonlinear reconstruction algorithms in high real-time business scenarios.

[0106] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A method of reconstructing three-dimensional point cloud data, characterized by, Includes the following steps: The process involves acquiring raw sparse point cloud data and sensor scanning trajectory information; for target points in the raw sparse point cloud data, calculating the rate of change of the normal vector and the curvature tensor as local geometric tensors, and combining this with the sensor scanning trajectory information to calculate the reciprocal of the local point cloud density as the density distribution; and constructing a local feature entropy field by weighting the normal vector rate of change, the curvature tensor, and the reciprocal of the density, wherein the local feature entropy field is used to quantify the information disorder and feature saliency of local regions of the point cloud. Based on the local feature entropy field, the reconstruction confidence weight of each point in the original sparse point cloud data is calculated to generate an anisotropic confidence evaluation matrix, wherein the reconstruction confidence weight is used to characterize the probability that each point belongs to the true geometric features or random noise. Based on the anisotropic confidence assessment matrix, multi-scale adaptive mesh growth is performed to generate an initial 3D reconstruction model, wherein the multi-scale adaptive mesh growth includes dynamically adjusting the mesh size and topology according to the reconstruction confidence weight; By combining the original sparse point cloud data with the initial 3D reconstruction model, semantic residuals are calculated, and the initial 3D reconstruction model is locally refined based on the semantic residuals to output the final 3D model. Combining the original sparse point cloud data with the initial 3D reconstruction model, the semantic residual is calculated, including: Based on the sensor scanning trajectory information, the original acquisition viewpoint is determined, and the initial 3D reconstruction model is projected back to the original acquisition viewpoint to generate a reconstruction depth map; The original sparse point cloud data is projected onto the original acquisition viewpoint to generate an original point cloud projection map, and the geometric deviation between the reconstructed depth map and the original point cloud projection map under the corresponding original acquisition viewpoint is calculated. The key topological features of the reconstructed depth map and the original point cloud projection map are extracted respectively, and the matching degree of the key topological features is extracted. The semantic residual is generated by weighting the geometric deviation with the matching degree, wherein the semantic residual is used to indicate regions where key features are lost or overly smoothed.

2. The method for reconstructing three-dimensional point cloud data according to claim 1, characterized in that, The calculation of the local geometric tensor and density distribution of the original sparse point cloud data, and the construction of a local feature entropy field based on the local geometric tensor and density distribution, includes: For each target point in the original sparse point cloud data, search for its nearest neighbor set within a preset neighborhood. The rate of change of the normal vector and the curvature tensor of the target point and the set of nearest neighbors are calculated as the local geometric tensor, and the local point cloud density of the target point is calculated as the density distribution in combination with the sensor scanning trajectory information; Based on the normal vector change rate, the curvature tensor, and the local point cloud density, the feature entropy value of the target point is calculated, and the feature entropy values ​​of all target points are mapped into a three-dimensional salient feature map as the local feature entropy field.

3. The method for reconstructing three-dimensional point cloud data according to claim 2, characterized in that, The step of calculating the reconstruction confidence weight of each point in the original sparse point cloud data based on the local feature entropy field and generating an anisotropic confidence evaluation matrix includes: Regions in the three-dimensional saliency feature map with feature entropy values ​​less than a preset feature entropy threshold are identified as low-entropy regions, and regions with feature entropy values ​​greater than or equal to the preset feature entropy threshold are identified as high-entropy regions. For the high-entropy region, the eigenvalues ​​of the covariance matrix of the nearest neighbor set are further extracted to determine its topological continuity. If the proportion of the largest eigenvalue in the sum of eigenvalues ​​exceeds the continuity lower limit and satisfies the preset topological continuity condition, it is determined to be a complex feature region and given a high confidence weight. If the proportion of the largest eigenvalue does not satisfy the preset topological continuity condition, it is determined to be an outlier noise region and given a low confidence weight. The low-entropy region is determined to be a flat safe region and assigned a medium confidence weight to form the anisotropic confidence assessment matrix.

4. The three-dimensional point cloud data reconstruction processing method according to claim 3, characterized in that, The step of performing multi-scale adaptive mesh growth based on the anisotropic confidence evaluation matrix includes: In the flat, safe area, a first-scale grid is used for rapid fitting to reduce data storage requirements; In the complex feature region, a second-scale mesh is used for fine fitting, the mesh size is reduced to construct physical edges in situ, and an edge constraint operator is introduced to calculate the maximum direction of the local point cloud normal vector gradient. Based on this direction, a geometric offset is applied to the mesh vertices to force the second-scale mesh to fit the high-frequency feature edges, wherein the second scale is smaller than the first scale. In the outlier noise region, mesh generation is suppressed according to the low confidence weight to remove background noise.

5. The method for reconstructing three-dimensional point cloud data according to claim 1, characterized in that, The local refinement of the initial 3D reconstruction model based on the semantic residual includes: In regions where the semantic residual is greater than or equal to a preset residual threshold, a local reconstruction mechanism is automatically triggered. Increase the grid resolution within the region and adjust the reconstruction confidence weights, then perform the multi-scale adaptive grid growth again until the semantic residual is less than the preset residual threshold.

6. The method for reconstructing three-dimensional point cloud data according to claim 1, characterized in that, The original sparse point cloud data comes from autonomous driving environmental perception devices or handheld 3D scanning devices. The final 3D model is used for obstacle identification or reverse engineering modeling in downstream tasks.

7. A three-dimensional point cloud data reconstruction processing system, characterized in that, Includes the following modules: The entropy field construction module is used to acquire raw sparse point cloud data and sensor scanning trajectory information; calculate the local geometric tensor and density distribution of the raw sparse point cloud data, and construct a local feature entropy field based on the local geometric tensor and density distribution, wherein the local feature entropy field is used to quantify the information disorder and feature saliency of local regions of the point cloud. The weight evaluation module is used to calculate the reconstruction confidence weight of each point in the original sparse point cloud data based on the local feature entropy field, and generate an anisotropic confidence evaluation matrix, wherein the reconstruction confidence weight is used to characterize the probability that each point belongs to the true geometric features or random noise. The mesh reconstruction module is used to perform multi-scale adaptive mesh growth based on the anisotropic confidence evaluation matrix to generate an initial three-dimensional reconstruction model, wherein the multi-scale adaptive mesh growth includes dynamically adjusting the mesh size and topology according to the reconstruction confidence weight; The feedback optimization module is used to combine the original sparse point cloud data with the initial 3D reconstruction model, calculate the semantic residual, and refine the initial 3D reconstruction model locally based on the semantic residual to output the final 3D model. Combining the original sparse point cloud data with the initial 3D reconstruction model, the semantic residual is calculated, including: Based on the sensor scanning trajectory information, the original acquisition viewpoint is determined, and the initial 3D reconstruction model is projected back to the original acquisition viewpoint to generate a reconstruction depth map; The original sparse point cloud data is projected onto the original acquisition viewpoint to generate an original point cloud projection map, and the geometric deviation between the reconstructed depth map and the original point cloud projection map under the corresponding original acquisition viewpoint is calculated. The key topological features of the reconstructed depth map and the original point cloud projection map are extracted respectively, and the matching degree of the key topological features is extracted. The semantic residual is generated by weighting the geometric deviation with the matching degree, wherein the semantic residual is used to indicate regions where key features are lost or overly smoothed.

Images