Display method and system for real scene three-dimensional achievement product based on laser radar

Abstract

This invention belongs to the field of 3D modeling technology and discloses a method and system for displaying real-scene 3D product results based on LiDAR. The method includes: scanning the target product with a 3D scanning device to obtain real-scene 3D data of the product; performing multi-step denoising, layered collaborative hole filling, and optimization on the 3D data to obtain preprocessed 3D data; using the preprocessed 3D data, constructing a 3D model of the target product through 3D modeling software; creating a virtual display environment in a virtual reality engine and importing the 3D model into the environment; designing interactive functions for the virtual reality scene and receiving user feedback data in real time to optimize the 3D model of the target product. This invention combines multi-step denoising and layered collaborative hole filling to effectively filter out noise and repair low-density areas, thereby reducing the error of traditional point cloud registration.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, and more specifically, to a method and system for displaying 3D real-world results products based on LiDAR. Background Technology

[0002] With the development of technology, product display methods are becoming increasingly diverse, but traditional display methods have certain limitations. For example, two-dimensional images and videos cannot fully and three-dimensionally showcase products. Real-world 3D renderings have a high degree of realism and three-dimensionality, but their application in the field of product display is not yet widespread.

[0003] Chinese patent application CN118840492A discloses an intelligent VR 3D digital product display system and method: acquiring product images and point cloud datasets of a target product; extracting corner points of the target product from the product images; extracting the product outline of the target product from the corner points; matching the point cloud coordinates corresponding to the product outline in the point cloud dataset based on the product outline; performing point cloud registration on the point cloud coordinates to obtain initial registered point cloud coordinates; performing color matching on the initial registered point cloud coordinates to obtain matched point cloud coordinates; performing point cloud filtering on the matched point cloud coordinates to obtain target point cloud coordinates; calculating the center point of a surface based on the target point cloud coordinates; generating a surface support domain based on the surface center point; fitting the 3D surface of the target product based on the surface support domain; performing 3D modeling on the 3D surface to obtain a 3D model of the target product. This invention extracts the product outline based on the product corner points; determines the corresponding point cloud coordinates of the product outline in the point cloud dataset; performs point cloud registration on the point cloud coordinates to obtain initial registered point cloud coordinates, which can effectively reduce the complexity of point cloud coordinates and remove noise from the point cloud data; performs color matching and point cloud filtering on the initial registered point cloud coordinates, which can make the real features in the point cloud data more obvious and obtain high-quality target point cloud coordinates, which is beneficial to improving the accuracy of subsequent 3D model construction.

[0004] While the above methods can meet the needs of most scenarios, research and practical application of these methods and existing technologies have revealed at least the following shortcomings:

[0005] Relying on image-based contour extraction without incorporating 3D scanning equipment results in insufficient data integrity; point cloud registration relies on static parameters, making it difficult to handle complex geometric structures or reflective surfaces, leading to cumulative registration errors.

[0006] In view of this, the present invention proposes a method and system for displaying real-world 3D results products based on lidar to solve the above problems. Summary of the Invention

[0007] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a method for displaying 3D real-scene results products based on lidar, comprising the following steps:

[0008] The target product is scanned using a 3D scanning device to obtain real-world 3D data of the product;

[0009] Multi-step denoising, layered collaborative hole filling, and optimization are performed on the 3D data to obtain preprocessed 3D data;

[0010] Using the preprocessed 3D data, a 3D model of the target product is constructed using 3D modeling software;

[0011] Create a virtual display environment in the virtual reality engine and import the 3D model into the virtual display environment;

[0012] Design interactive functions for virtual reality scenes and receive user feedback data in real time to optimize the 3D model of the target product.

[0013] Furthermore, methods for hierarchical collaborative hole-filling of 3D data include:

[0014] Calculate the average distance of the k-neighborhood point set of each denoised 3D data point, where k represents the number of neighboring points. If the average distance of the i-th point is greater than twice the global average distance, then the i-th point is marked as a low-density point. Identify continuous low-density regions as holes through connected component analysis, and divide the holes according to the voxel resolution. Perform hierarchical collaborative hole filling on the divided holes.

[0015] Furthermore, methods for identifying continuous low-density regions as holes through connected component analysis include:

[0016] The denoised 3D data space is divided into cubic voxels with fixed side lengths v;

[0017] Count the number of denoised 3D data points contained within each voxel;

[0018] Voxels with densities below a preset density threshold are marked as low-density candidate voxels; the average distance between the S neighboring points of each voxel is calculated, and the low-density threshold is dynamically adjusted by an amplification factor based on the average distance between the S neighboring points.

[0019] Traverse all voxels and use unlabeled voxels as seed points;

[0020] Starting from any seed point, expand to all connected neighborhoods and count the low-density voxels belonging to the same connected region;

[0021] Record the set of voxels and its volume for each connected region;

[0022] Remove connected regions and boundary contact voxels with volumes smaller than a preset volume threshold; for the remaining connected regions, extract the surface mesh of all low-density candidate voxels as the hole boundaries to obtain the holes.

[0023] Furthermore, the method of layered collaborative hole filling after the holes have been divided includes:

[0024] For the boundary points of small holes, dynamic voxel partitioning is performed in the hole region using preset rules, and missing data is filled based on a weighted average of the coordinates of neighboring points and normal vectors; a local implicit function is constructed by combining the voxel edge length and hole coordinates to calculate the SDF value; for each voxel in the small hole, the SDF value of the corresponding 8 vertices is calculated; the intersection points of the isosurface and voxel edges are determined according to the predefined basic configuration; a continuous triangular mesh is generated through the lookup table corresponding to the basic configuration; and the triangular mesh is smoothed using Laplace.

[0025] For the central hole, the boundary point features of the central hole are encoded as position embeddings, curvature encoding is introduced, and attention weights are calculated. The attention weights are aggregated through MLP to obtain the hole-filling coordinates, and the hole-filling coordinates are forced to meet preset constraints.

[0026] For large holes, a GAN is constructed, and the coordinates of the hole boundary, Gaussian curvature, and normal vector are used as inputs to the generator to obtain filling data points; topological constraints are constructed, and the large holes are filled based on the trained GAN.

[0027] In the boundary area of ​​different holes, a distance-weighted fusion method is used to fuse and optimize the hole-filling data;

[0028] The hole-filling results are optimized by taking into account the surface texture of the product.

[0029] Furthermore, methods for obtaining Gaussian curvature include:

[0030] Obtain J neighboring points of the boundary point, calculate the centroid of the neighboring points, and translate the boundary point according to the difference between its coordinates and the centroid. Perform principal component extraction on the translated boundary point to obtain the normal vector and two tangent directions of the tangent plane. Establish a local coordinate system based on the normal vector and the two tangent directions, and project the neighboring points onto the tangent plane formed by the two tangent directions to obtain the projected coordinates and the distance to the tangent plane. Fit a quadratic function of the distance to the tangent plane with respect to the projected coordinates, and obtain the coefficients of the quadratic function by solving the least squares method.

[0031] Obtain the second derivative matrix of the quadratic surface from the coefficients of the quadratic function, solve for the two eigenvalues ​​of the second derivative matrix, calculate the product of the two eigenvalues, and obtain the Gaussian curvature of the boundary points.

[0032] Furthermore, the training method for the GAN includes:

[0033] The coordinates, Gaussian curvature, and normal vector of the hole boundary are used as inputs to the generator to obtain filling data points as the generator data. The discriminator receives the generator data and determines the probability that the generated data belongs to the real data.

[0034] A batch of real data is randomly selected from the preset training set. The selected real data and the generated data are used as the input of the discriminator. The discriminator outputs the corresponding discrimination probability.

[0035] The optimization loss of GAN is designed, including constraint set loss and adversarial loss, and the network parameters of GAN are updated. The process continues until GAN ​​reaches the preset optimal performance condition, and the GAN corresponding to the preset optimal performance condition is taken as the final output GAN.

[0036] Furthermore, methods for collaboratively optimizing hole-filling results by incorporating product surface texture include:

[0037] For each point, calculate the average distance to Q neighboring points, and delete outliers whose distance exceeds the sum of the global mean and twice the standard deviation;

[0038] Within the patching area and the surrounding area of ​​twice the hole radius, missing points are supplemented by uniform resampling, based on the average spacing of the three-dimensional data within the hole radius.

[0039] Obtain E neighboring points of the hole-filling point, and correct the hole-filling point based on the distance-weighted average of the coordinates of the E neighboring points; where the weighting weight is calculated based on the distance between the neighboring points and the hole-filling point;

[0040] Obtain the RGB color values ​​of the Y neighboring points of each 3D data point, take the median value of each channel, and replace the color of the current point.

[0041] For patching points in colored patching regions, obtain the colors of the H nearest neighbors and perform a weighted average; for patching points without color, generate the color that appears most frequently based on the color distribution of the neighboring points; the color of the patching point satisfies the preset color constraint.

[0042] Furthermore, multi-step denoising methods for 3D data include:

[0043] S1: Obtain N neighboring points for each data point by calculating Euclidean distance. If the distance from a data point to any neighboring point exceeds a preset neighboring distance threshold, the data point is determined to be an outlier and deleted. Iterate through and filter all outliers to obtain the filtered 3D data.

[0044] S2: For the data points in the filtered 3D data, take the M nearest neighbors by Euclidean distance and sort them in ascending order of distance; replace the coordinates of the current data point with the coordinates of the sorted intermediate points, and iterate through the filtered 3D data to obtain coarsely denoised data.

[0045] S3: Perform Gaussian filtering on the coarsely denoised data, and use the Gaussian kernel function to perform a weighted average of the neighborhood points to obtain finely denoised data;

[0046] S4: For each finely denoised data point, the principal component analysis of the covariance matrix of the k-neighbor points is used to take the eigenvector corresponding to the smallest eigenvalue as the normal vector; the neighboring points are projected onto the tangent plane of the current point, and the centroid of the neighboring points in the tangent plane is calculated; the smooth displacement is calculated only in the normal direction in combination with the smoothing factor, and the finely denoised data points are updated based on the smooth traversal to obtain the denoised 3D data.

[0047] Furthermore, in the multi-step denoising method for 3D data, after each step S1-S3, the point cloud feature points of the current step are extracted, the number of feature points in the current stage is compared with the feature point data before filtering, and the retention rate is calculated. If the retention rate is lower than a preset retention threshold, error compensation is triggered.

[0048] Calculate and obtain the covariance matrix of the feature point displacements before and after filtering;

[0049] Based on the retention rate and covariance matrix, adjust the next-level filtering parameters according to the preset dynamic parameter adjustment rules;

[0050] Methods for optimizing 3D data include:

[0051] A weighting function is constructed based on the Gaussian curvature of the 3D data points, and different smoothing intensities are applied in the normal and tangential directions; specifically, the smoothing intensity is reduced for points with high curvature in the normal direction, and the smoothing intensity is increased for points with low curvature in the tangential direction.

[0052] The feature vector of each point in the 3D data is calculated, and principal component analysis is performed on the feature vector to reduce the dimensionality to a 3D feature space. The reduced feature vector is then classified using FCM clustering optimized by Lagrange multipliers, combined with preset high and low clustering thresholds. The classification results include planar points, transition points, and feature points. 3D data points classified as feature points are extracted, and a feature protection mask is generated. If a 3D data point is a feature point, the feature protection mask is 1; otherwise, it is 0. During filtering, feature points are only allowed to move along the normal direction within a preset change region, with a fixed tangential position.

[0053] Methods for obtaining real-world 3D data of a product include:

[0054] Preliminary product data is obtained through sensor scanning, and corresponding parameter combinations are selected based on a pre-established knowledge base.

[0055] Collect the scanning parameter range of the R group of scanning devices;

[0056] The initial product data obtained by sensor scanning and the scanning parameter range of R group scanning equipment are used as inputs to the parameter adjustment model to obtain the optimal parameter combination;

[0057] The initial data of the product is adjusted and tested. For areas that meet the preset adjustment requirements, compensation testing is performed in combination with the preset adjustment strategy. The compensation testing includes reflection pre-compensation and occlusion pre-compensation.

[0058] The method for displaying 3D real-scene results based on LiDAR, which is based on LiDAR, includes:

[0059] Real-scene scanning module: Uses 3D scanning equipment to scan the target product and obtain real-scene 3D data of the product;

[0060] Preprocessing module: Performs multi-step denoising, layered collaborative hole filling, and optimization on the 3D data to obtain preprocessed 3D data;

[0061] Model building module: Using preprocessed 3D data, construct a 3D model of the target product using 3D modeling software;

[0062] Virtual Display Module: Creates a virtual display environment in the virtual reality engine and imports 3D models into the virtual display environment;

[0063] Human-computer interaction module: Used to design interactive functions for virtual reality scenes and receive user feedback data in real time to optimize the 3D model of the target product.

[0064] The technical effects and advantages of this invention, based on a lidar-based real-scene 3D product display method and system, are as follows:

[0065] This invention directly acquires real-world 3D data of a product using a 3D scanning device, avoiding reliance on 2D image contour extraction. It combines preliminary sensor data with a parameter adjustment model to dynamically adapt scanning parameters, achieving complete data acquisition of complex geometric structures and reflective / occluded areas, ensuring the comprehensiveness and authenticity of the 3D data from the source. By combining multi-step denoising and layered collaborative hole filling, noise is effectively filtered out and low-density areas are repaired, reducing errors in traditional point cloud registration. Simultaneously, feature point protection masking technology based on Lagrange multiplier-optimized FCM clustering ensures the complete preservation of geometric features of complex surfaces, avoiding the cumulative errors of traditional static registration in high-curvature areas, improving the accuracy of 3D model construction, and providing a high-precision data foundation for virtual display. Attached Figure Description

[0066] Figure 1This is a flowchart of the method for displaying real-world 3D results products based on lidar, according to Embodiment 1 of the present invention.

[0067] Figure 2 This is a schematic diagram of the product display method according to Embodiment 1 of the present invention;

[0068] Figure 3 This is a schematic diagram of the method flow of Embodiment 2 of the present invention;

[0069] Figure 4 This is a schematic diagram of the system structure of Embodiment 3 of the present invention. Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] Example 1:

[0072] Please see Figure 1 , Figure 2 As shown, this embodiment provides a method for displaying 3D real-world results products based on LiDAR, including the following steps:

[0073] The target product is scanned using a 3D scanning device to obtain real-world 3D data of the product;

[0074] Methods for obtaining real-world 3D data of a product include:

[0075] Preliminary product data, such as dimensions, material reflectivity, and surface roughness, is acquired through sensor scanning. Dimensions can be measured using a laser rangefinder, and material reflectivity can be detected using a spectrometer. For example, reflectivity greater than 80% is considered specular, reflectivity between 30% and 80% is semi-diffuse, and reflectivity less than 30% is diffuse. Corresponding parameter combinations are selected based on a pre-established knowledge base. For instance, for reflective metal parts, a structured light scanner with a polarizing filter is selected; low-power blue light is chosen to reduce specular reflection; a 5° scanning interval is selected to densely cover the refractive details of the reflective surface; and HDR mode is enabled for texture acquisition to enhance highlight and shadow details.

[0076] Collect the scanning parameter range of the R group of scanning devices;

[0077] The initial product data obtained by sensor scanning and the scanning parameter range of R group scanning equipment are used as inputs to the parameter adjustment model to obtain the optimal parameter combination;

[0078] The initial product data is adjusted and tested. For areas that meet the preset adjustment requirements, compensation testing is performed using preset adjustment strategies; this includes pre-compensation for glare and pre-compensation for occlusion. For example, if the simulation shows that the reflectivity of a curved surface is >85% and the incident angle is <30°, which is prone to specular reflection, an orthogonal viewing angle is added to the scanning parameters of that area in advance. For example, if the original viewing angles are 0° and 45°, a -15° side-light viewing angle is added; the light source power is dynamically reduced by 20%, and "phase-shifting structured light" is enabled to reduce glare interference through stripe coding. For example, if the dimensional parameters show the presence of a deep hole (i.e., hole diameter <10mm and depth >50mm), the scanning angle interval of that area is automatically reduced to 3°, and the device's "macro focus" mode is triggered.

[0079] The above steps abandon traditional image contour extraction and directly acquire real-world 3D data, including dimensions, reflectivity, and surface roughness, through 3D scanning equipment. Combined with a pre-established knowledge base, it intelligently matches equipment, light sources, scanning intervals, and texture acquisition modes to achieve complete data acquisition for complex structures such as reflective metal parts and deep holes. This avoids contour extraction deviations caused by missing image information from the source, improving the completeness of 3D data. The sensor's initial data-driven parameter adjustment model dynamically generates optimal scanning parameters. Combined with reflectivity pre-compensation and occlusion pre-compensation, it significantly improves point cloud quality, such as increasing point cloud density in reflective areas and reducing data loss in occluded areas. This provides a high-precision data foundation for subsequent registration, fundamentally reducing cumulative errors caused by insufficient static parameter adaptation and improving the registration accuracy of complex structures.

[0080] Multi-step denoising, layered collaborative hole filling, and optimization are performed on the 3D data to obtain preprocessed 3D data;

[0081] Methods for multi-step denoising of 3D data include:

[0082] S1: Obtain N neighboring points for each data point by calculating Euclidean distance. If the distance from a data point to any neighboring point exceeds a preset neighboring distance threshold, the data point is determined to be an outlier and deleted. Iterate through and filter all outliers to obtain the filtered 3D data.

[0083] S2: For the data points in the filtered 3D data, take the M nearest neighbors by Euclidean distance and sort them in ascending order of distance; replace the coordinates of the current data point with the coordinates of the sorted intermediate points, and iterate through the filtered 3D data to obtain coarsely denoised data.

[0084] S3: Perform Gaussian filtering on the coarsely denoised data, and use the Gaussian kernel function to perform a weighted average of the neighborhood points to obtain finely denoised data;

[0085] S4: For each finely denoised data point, principal component analysis of the covariance matrix of k-neighbor points is used to take the eigenvector corresponding to the smallest eigenvalue as the normal vector; the neighboring points are projected onto the tangent plane of the current point, and the centroid of the neighboring points in the tangent plane is calculated; the smoothing displacement is calculated only in the normal direction in combination with the smoothing factor, which is usually set to 0.1-0.5. The finely denoised data points are updated based on the smoothing traversal to obtain the denoised 3D data.

[0086] After each filtering step, feature points of the current point cloud are extracted, the number of feature points in the current stage is compared with the feature point data before filtering, and the retention rate is calculated. If the retention rate is lower than the preset retention threshold, error compensation is triggered.

[0087] The covariance matrix of the feature point displacements before and after filtering is calculated. The matrix has a dimension of 3×3 and represents the three-dimensional spatial error distribution.

[0088] Based on the retention rate and covariance matrix, the next-level filtering parameters are adjusted according to the preset dynamic parameter adjustment rules.

[0089] The parameter adjustment rules are as follows:

[0090] S1: If the retention rate is lower than the preset threshold, then reduce the neighborhood distance threshold;

[0091] S2: If the maximum value of the main diagonal of the covariance matrix is ​​>0.1mm, reduce the number of neighborhood points, such as from 50 to 30;

[0092] S3: If normal smoothing causes feature offset >0.05mm, reduce the smoothing factor, such as from 0.3 to 0.1.

[0093] The aforementioned multi-step denoising and dynamic parameter adjustment process progressively purifies the data through outlier filtering, median filtering, Gaussian filtering, and normal smoothing. It dynamically optimizes parameters using feature point retention rate and covariance matrix, efficiently removing noise and improving surface smoothness while precisely protecting key features such as edges and high curvature, preventing detail loss. This ensures that the 3D reality data retains the geometric accuracy and feature details of the real scene after denoising, providing high-quality, high-fidelity foundational data for results presentation. This results in more realistic and detailed presentations, meeting the needs of engineering surveying, urban planning, and cultural tourism experiences for high-precision, high-realism 3D reality results. It enhances the accuracy and immersiveness of visualization, improving the application value and presentation effect of the results.

[0094] Methods for layered collaborative hole-filling of 3D data include:

[0095] Calculate the average distance of the k-neighborhood point set for each denoised 3D data point, where k represents the number of neighboring points. If the average distance of the i-th point is greater than twice the global average distance, then the i-th point is marked as a low-density point. Identify continuous low-density regions as holes through connected component analysis, and divide the holes according to their diameter. Then, fill the holes after division.

[0096] Methods for identifying continuous low-density regions as voids through connected component analysis include:

[0097] The denoised 3D data space is divided into cubic voxels with fixed side lengths v;

[0098] Count the number of denoised 3D data points contained within each voxel;

[0099] Voxels with densities below a preset density threshold are marked as low-density candidate voxels; the average distance between the S neighboring points of each voxel is calculated, and the low-density threshold is dynamically adjusted based on the average distance between the S neighboring points using an amplification factor; the amplification factor can be obtained by optimization using a natural heuristic optimization algorithm.

[0100] Traverse all voxels and use unlabeled voxels as seed points;

[0101] Starting from any seed point, expand to all connected neighborhoods and count the low-density voxels belonging to the same connected region; in three-dimensional space, each voxel is considered connected to its 26-neighbor or 6-neighbor neighborhoods. The 26-neighbor neighborhoods include adjacent voxels in the up / down / front / back / left / right directions, and the 6-neighbor neighborhoods only include the 6 neighboring neighborhoods on the face; record the voxel set and volume of each connected region.

[0102] Remove connected regions and boundary contact voxels with volumes smaller than a preset volume threshold; for the remaining connected regions, extract the surface mesh of all low-density candidate voxels as the hole boundaries to obtain the holes.

[0103] Based on the average distance of k-neighborhoods and connected region analysis, it can accurately identify continuous low-density holes, avoid missed detections, and solve the problem of incomplete models caused by missing data.

[0104] Methods for filling the divided holes include:

[0105] For the boundary points of small holes, dynamic voxel partitioning is performed in the hole region using preset rules. For example, when the hole diameter is less than 10mm, the voxel side length is v / 2; otherwise, the voxel side length is v. Missing data is filled using a weighted average of the coordinates of neighboring points and the normal vector. A local implicit function is constructed by combining the voxel side length and the hole coordinates to calculate the SDF value. The isosurface threshold is set to zero, i.e., SDF(x) = 0. For each voxel in the small hole, the SDF value of the corresponding 8 vertices is calculated. Based on predefined basic configurations, such as utilizing the rotation and reflection symmetry of 3D voxels, 256 configurations are merged into 15 basic configurations to determine the intersection points of the isosurface and the voxel edges. Through the lookup table corresponding to the basic configuration, each configuration corresponds to a unique combination of edge intersection points and triangle connection methods to generate a continuous triangular mesh. The triangular mesh is then smoothed using Laplace.

[0106] For the central hole, the boundary point features of the central hole are encoded by position embedding, curvature encoding is introduced, and attention weights are calculated. The attention weights are aggregated by MLP to obtain the hole-filling coordinates, and the hole-filling coordinates are forced to meet preset constraints, such as the distance between the hole-filling coordinates and the boundary points of the central hole being less than twice the average distance between the boundary points of the central hole.

[0107] For large holes, a GAN is constructed, and the coordinates of the hole boundary, Gaussian curvature, and normal vector are used as inputs to the generator to obtain filling data points as the generator data. The discriminator receives the generator data and determines the probability that the generated data belongs to the real data.

[0108] A batch of real data is randomly selected from the preset training set. The selected real data and the generated data are used as the input of the discriminator. The discriminator outputs the corresponding discrimination probability.

[0109] Design the optimization loss of GAN, including constraint set loss and adversarial loss, and update the network parameters of GAN; stop when GAN reaches the preset best performance condition, and take the GAN corresponding to the preset best performance condition as the final output GAN; add topological constraints, such as forcing the difference between the Euler feature number of the hole-filling point and the original data to be <2%, and fill large holes based on the final output GAN.

[0110] In the boundary area of ​​different holes, a distance-weighted fusion method is used to fuse and optimize the hole-filling data;

[0111] The hole-filling results are optimized by taking into account the surface texture of the product.

[0112] The aforementioned hole-filling methods for different scales employ dynamic voxel partitioning and SDF calculation, attention mechanisms and constraint optimization, and GAN generation combined with topological constraints to precisely repair local details, balance accuracy and efficiency, and handle complex topological deficiencies. Simultaneously, fusion optimization and texture collaboration in boundary regions ensure a high degree of geometric continuity and texture consistency between the filled data and the original data. This process eliminates hole defects in the displayed 3D model, presenting a complete surface; retains key features, such as sharp edges for small holes and reasonable topology for large hole areas, avoiding distorted filling; and ensures natural texture transitions, with seamless integration between the filled area and the original texture, enhancing visual realism. Ultimately, the hole-filled 3D data is more realistic, richer in detail, and more topologically compliant in the final presentation, meeting the needs of engineering, cultural tourism, and planning fields for complete, accurate, and high-fidelity 3D scene displays. This enhances the visualization quality and application value of the model, providing viewers or users with a more immersive and realistic 3D scene experience.

[0113] Differentiated hole-filling strategies are adopted for small, medium, and large holes to ensure that the hole-filling area is highly consistent with the surrounding geometric features, providing a complete and feature-aligned data foundation for registration and reducing the error of missing corresponding points caused by holes.

[0114] Methods for collaboratively optimizing hole-filling results by incorporating product surface texture include:

[0115] For each point, calculate the average distance of Q neighboring points (Q=20), and delete outliers whose distance exceeds the sum of the global mean and twice the standard deviation;

[0116] Within the patching area and the surrounding area of ​​twice the hole radius, missing points are supplemented by uniform resampling, based on the average spacing of the three-dimensional data within the hole radius.

[0117] Obtain E neighboring points of the hole-filling point, and correct the hole-filling point based on the distance-weighted average of the coordinates of the E neighboring points; where the weighting weight is calculated based on the distance between the neighboring points and the hole-filling point;

[0118] Obtain the RGB color values ​​of Y neighboring points (Y=15) of each 3D data point, take the median value of each channel, and replace the color of the current point;

[0119] For patching points in colored areas, obtain the colors of the H nearest neighbors and perform a weighted average; for patching points without color, generate the color that appears most frequently based on the color distribution of the neighbors; the color of the patching point satisfies the preset color constraints, such as the RGB difference between the new point color and the neighboring point colors being <15.

[0120] The above method, combined with the collaborative optimization of product surface texture, systematically improves the display quality of real-world 3D results through precise repair in both geometric and color dimensions: outlier removal and uniform resampling ensure that the point cloud density of the patched area is consistent with the surrounding area, avoiding noise or holes in the model caused by sparse point clouds during display; patch point correction uses neighborhood weighted averaging to highly integrate the geometric features of the patched area with the original data, eliminating structural distortion of the displayed model; color median replacement and weighted averaging ensure natural texture transitions, and colorless patch points generate colors that conform to the real material based on the neighborhood color distribution, ultimately forming a geometrically complete and textured 3D model, significantly improving the immersion and credibility of virtual displays, especially suitable for detailed display and interactive experience optimization of high-end products.

[0121] Methods for optimizing 3D data include:

[0122] Obtain J neighboring points of a 3D data point, calculate the centroid of the neighboring points, and translate the 3D data point according to the difference between its coordinates and the centroid. Perform principal component extraction on the translated 3D data to obtain the normal vector and two tangent directions of the tangent plane. Establish a local coordinate system based on the normal vector and the two tangent directions, and project the neighboring points onto the tangent plane formed by the two tangent directions to obtain the projected coordinates and the distance to the tangent plane. Fit a quadratic function of the distance to the tangent plane with respect to the projected coordinates, and obtain the coefficients of the quadratic function by solving the least squares method.

[0123] The second derivative matrix of the quadratic surface is obtained from the coefficients of the quadratic function. The two eigenvalues ​​of the second derivative matrix are solved, and the product of the two eigenvalues ​​is calculated to obtain the Gaussian curvature.

[0124] A weighting function is constructed based on Gaussian curvature, and different smoothing intensities are applied in the normal and tangential directions; specifically, the smoothing intensity is reduced for high curvature points in the normal direction, and the smoothing intensity is increased for low curvature points in the tangential direction.

[0125] The feature vector of each point in the 3D data is calculated, and principal component analysis is performed on the feature vector to reduce the dimensionality to a 3D feature space. The dimensionality-reduced feature vector is classified by FCM clustering optimized by Lagrange multipliers, combined with preset high and low clustering thresholds. The classification results include planar points, transition points, and feature points. 3D data points classified as feature points are extracted, and feature protection masks are generated. If a 3D data point is a feature point, the feature protection mask is 1; otherwise, it is 0. During filtering, feature points are only allowed to move along the normal direction within a preset change area. If the displacement is less than 0.05 times the average spacing of the 3D data points, the tangential position is strictly fixed to avoid the edges becoming "blunted".

[0126] The aforementioned 3D data optimization method systematically improves the display accuracy and detail realism of real-world 3D data through a triple mechanism of geometric feature quantification analysis, adaptive smoothing, and feature point protection: A Gaussian curvature-based weighted smoothing strategy eliminates surface noise while preserving key features such as edges and bosses, avoiding edge blurring caused by traditional uniform smoothing; a feature protection mask generated by Lagrange multiplier-optimized FCM clustering ensures that details such as bolt holes and chamfers are only allowed to undergo minute normal displacements during filtering, improving the integrity of geometric features. The resulting 3D data possesses both smooth surfaces and sharp features, accurately presenting the structural design details of products in virtual displays. Combined with realistic rendering, it significantly enhances the user's immersive experience of product form and materials, making it particularly suitable for scenarios requiring high-precision detail display, such as industrial design and cultural heritage digitization.

[0127] Using the preprocessed 3D data, a 3D model of the target product is constructed using 3D modeling software;

[0128] Methods for constructing a 3D model of a target product include:

[0129] Acquire the preprocessed 3D data and align it with the coordinate system of the 3D modeling software;

[0130] Set the modeling scale and standardize the model units;

[0131] Construct the corresponding 3D model using 3D modeling software;

[0132] Material mapping is performed on the corresponding 3D model according to the product's material category, and lighting rendering is performed according to the lighting scene to obtain the 3D model of the target product.

[0133] Create a virtual display environment in the virtual reality engine and import the 3D model into that environment;

[0134] Methods for creating virtual display environments include:

[0135] Create virtual display environments in a virtual reality engine.

[0136] Import the constructed 3D model into the virtual display environment.

[0137] Design virtual showrooms or display rooms based on product characteristics.

[0138] Design interactive functions for virtual reality scenes and receive user feedback data in real time to optimize the 3D model of the target product.

[0139] Interactive functions designed for virtual reality scenarios include model rotation, scaling, movement, and decomposition; user interfaces and control options are provided for easy operation; user feedback data is received and analyzed to optimize product design and improve user experience.

[0140] The display system designed for virtual reality scenarios integrates 3D models, virtual scenes, and interactive functions. It supports users to perform operations such as rotating, scaling, moving, and decomposing models through an intuitive user interface. It also provides control options to receive user feedback data, which can help optimize product design and improve user experience.

[0141] The system deeply integrates 3D models with virtual scenes and optimizes performance to ensure a smooth user experience across multiple platforms, including PCs, mobile devices, and large screens. Users can browse products comprehensively in the virtual environment via display devices and remotely access the system via the internet. In terms of security, the system employs data encryption, user authentication, and access control measures to ensure secure data transmission and that user operations comply with permission guidelines, providing users with a safe, efficient, and immersive virtual display environment.

[0142] Example 2:

[0143] Please see Figure 3 As shown, this embodiment provides a method for intelligent hole repair applied to Embodiment 1, including the following steps:

[0144] A GAN generator is built based on GNN, representing 3D data as a graph structure. Nodes are 3D data coordinates, and edge weights are a joint measure of Euclidean distance and curvature difference. Local geometric features and global structural features are extracted through GCN layers. Local geometric features include Gaussian curvature and normal vectors, while global structural features include the ring structure around large holes. The hole boundary points and their F-order neighborhood features are used as input to the generator, and the output is the coordinates of the hole-filling points. The hole-filling points are forced to meet preset large hole-filling constraints, such as: the angle between the normal vector of the hole-filling point and the boundary point is <15°; and the spacing between points in the hole-filling region is within 80% to 120% of the global average spacing.

[0145] Example 3:

[0146] Please see Figure 4 As shown, this embodiment provides a LiDAR-based real-scene 3D result product display system, including:

[0147] Real-scene scanning module: Uses 3D scanning equipment to scan the target product and obtain real-scene 3D data of the product;

[0148] Preprocessing module: Performs multi-step denoising, layered collaborative hole filling, and optimization on the 3D data to obtain preprocessed 3D data;

[0149] Model building module: Using preprocessed 3D data, construct a 3D model of the target product using 3D modeling software;

[0150] Virtual Display Module: Creates a virtual display environment in the virtual reality engine and imports 3D models into that environment;

[0151] Human-computer interaction module: Used to design interactive functions for virtual reality scenes and to receive user feedback data in real time to optimize the 3D model of the target product.

[0152] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0153] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for displaying real-world 3D results products based on lidar, characterized in that: Includes the following steps: The target product is scanned using a 3D scanning device to obtain real-world 3D data of the product; Multi-step denoising, layered collaborative hole filling, and optimization are performed on the 3D data to obtain preprocessed 3D data; Methods for layered collaborative hole-filling of 3D data include: Calculate the average distance of the k-neighborhood point set of each denoised 3D data point, where k represents the number of neighboring points. If the average distance of the i-th point is greater than twice the global average distance, then the i-th point is marked as a low-density point. Identify continuous low-density regions as holes through connected component analysis, and divide the holes according to the voxel resolution. The divided holes are then filled in a layered, coordinated manner: For the boundary points of small holes, dynamic voxel partitioning is performed in the hole region using preset rules, and missing data is filled based on a weighted average of the coordinates of neighboring points and normal vectors; a local implicit function is constructed by combining the voxel edge length and hole coordinates to calculate the SDF value; for each voxel in the small hole, the SDF value of the corresponding 8 vertices is calculated; the intersection points of the isosurface and voxel edges are determined according to the predefined basic configuration; a continuous triangular mesh is generated through the lookup table corresponding to the basic configuration; and the triangular mesh is smoothed using Laplace. For the central hole, the boundary point features of the central hole are encoded as position embeddings, curvature encoding is introduced, and attention weights are calculated. The attention weights are aggregated through MLP to obtain the hole-filling coordinates, and the hole-filling coordinates are forced to meet preset constraints. For large holes, a GAN is constructed, and the coordinates of the hole boundary, Gaussian curvature, and normal vector are used as inputs to the generator to obtain filling data points; topological constraints are constructed, and the large holes are filled based on the trained GAN. In the boundary area of ​​different holes, a distance-weighted fusion method is used to fuse and optimize the hole-filling data; The hole-filling results are optimized in conjunction with the product's surface texture. Using the preprocessed 3D data, a 3D model of the target product is constructed using 3D modeling software; Create a virtual display environment in the virtual reality engine and import the 3D model into the virtual display environment; Design interactive functions for virtual reality scenes and receive user feedback data in real time to optimize the 3D model of the target product.

2. The method for displaying 3D real-scene results products based on lidar according to claim 1, characterized in that, Methods for identifying continuous low-density regions as voids through connected component analysis include: The denoised 3D data space is divided into cubic voxels with fixed side lengths v; Count the number of denoised 3D data points contained within each voxel; Voxels with densities below a preset density threshold are marked as low-density candidate voxels; the average distance between the S neighboring points of each voxel is calculated, and the low-density threshold is dynamically adjusted by an amplification factor based on the average distance between the S neighboring points. Traverse all voxels and use unlabeled voxels as seed points; Starting from any seed point, expand to all connected neighborhoods and count the low-density voxels belonging to the same connected region; Record the set of voxels and its volume for each connected region; Remove connected regions and boundary contact voxels with volumes smaller than a preset volume threshold; for the remaining connected regions, extract the surface mesh of all low-density candidate voxels as the hole boundaries to obtain the holes.

3. The method for displaying 3D real-scene results products based on lidar according to claim 1, characterized in that, Methods for obtaining Gaussian curvature include: Obtain J neighboring points of the boundary point, calculate the centroid of the neighboring points, and translate the boundary point according to the difference between its coordinates and the centroid. Perform principal component extraction on the translated boundary point to obtain the normal vector and two tangent directions of the tangent plane. Establish a local coordinate system based on the normal vector and the two tangent directions, and project the neighboring points onto the tangent plane formed by the two tangent directions to obtain the projected coordinates and the distance to the tangent plane. Fit a quadratic function of the distance to the tangent plane with respect to the projected coordinates, and obtain the coefficients of the quadratic function by solving the least squares method. Obtain the second derivative matrix of the quadratic surface from the coefficients of the quadratic function, solve for the two eigenvalues ​​of the second derivative matrix, calculate the product of the two eigenvalues, and obtain the Gaussian curvature of the boundary points.

4. The method for displaying 3D real-scene results products based on lidar according to claim 1, characterized in that, The training methods for the GAN include: The coordinates, Gaussian curvature, and normal vector of the hole boundary are used as inputs to the generator to obtain filling data points as the generator data. The discriminator receives the generator data and determines the probability that the generated data belongs to the real data. A batch of real data is randomly selected from the preset training set. The selected real data and the generated data are used as the input of the discriminator. The discriminator outputs the corresponding discrimination probability. The optimization loss of GAN is designed, including constraint set loss and adversarial loss, and the network parameters of GAN are updated. The process continues until GAN ​​reaches the preset optimal performance condition, and the GAN corresponding to the preset optimal performance condition is taken as the final output GAN.

5. The method for displaying 3D real-scene results products based on lidar according to claim 1, characterized in that, Methods for collaboratively optimizing hole-filling results by incorporating product surface texture include: For each point, calculate the average distance to Q neighboring points, and delete outliers whose distance exceeds the sum of the global mean and twice the standard deviation; Within the patching area and the surrounding area of ​​twice the hole radius, missing points are supplemented by uniform resampling, based on the average spacing of the three-dimensional data within the hole radius. Obtain E neighboring points of the hole-filling point, and correct the hole-filling point based on the distance-weighted average of the coordinates of the E neighboring points; where the weighting weight is calculated based on the distance between the neighboring points and the hole-filling point; Obtain the RGB color values ​​of the Y neighboring points of each 3D data point, take the median value of each channel, and replace the color of the current point. For patching points in colored patching regions, obtain the colors of the H nearest neighbors and perform a weighted average; for patching points without color, generate the color that appears most frequently based on the color distribution of the neighboring points; the color of the patching point satisfies the preset color constraint.

6. The method for displaying 3D real-scene results products based on lidar according to claim 1, characterized in that, Methods for multi-step denoising of 3D data include: S1: Obtain N neighboring points for each data point by calculating Euclidean distance. If the distance from a data point to any neighboring point exceeds a preset neighboring distance threshold, the data point is determined to be an outlier and deleted. Iterate through and filter all outliers to obtain the filtered 3D data. S2: For the data points in the filtered 3D data, take the M nearest neighbors by Euclidean distance and sort them in ascending order of distance; replace the coordinates of the current data point with the coordinates of the sorted intermediate points, and iterate through the filtered 3D data to obtain coarsely denoised data. S3: Perform Gaussian filtering on the coarsely denoised data, and use the Gaussian kernel function to perform a weighted average of the neighborhood points to obtain finely denoised data; S4: For each finely denoised data point, the principal component analysis of the covariance matrix of the k-neighbor points is used to take the eigenvector corresponding to the smallest eigenvalue as the normal vector; the neighboring points are projected onto the tangent plane of the current point, and the centroid of the neighboring points in the tangent plane is calculated; the smooth displacement is calculated only in the normal direction in combination with the smoothing factor, and the finely denoised data points are updated based on the smooth traversal to obtain the denoised 3D data.

7. The method for displaying 3D real-scene results products based on lidar according to claim 1, characterized in that, In the multi-step denoising method for 3D data, after each step S1-S3, the point cloud feature points of the current step are extracted, the number of feature points in the current stage is compared with the feature point data before filtering, and the retention rate is calculated. If the retention rate is lower than a preset retention threshold, error compensation is triggered. Calculate and obtain the covariance matrix of the feature point displacements before and after filtering; Based on the retention rate and covariance matrix, adjust the next-level filtering parameters according to the preset dynamic parameter adjustment rules; Methods for optimizing 3D data include: A weighting function is constructed based on the Gaussian curvature of the 3D data points, and different smoothing intensities are applied in the normal and tangential directions. The feature vector of each point in the 3D data is calculated, and principal component analysis is performed on the feature vector to reduce the dimensionality to a 3D feature space. The reduced feature vector is then classified using FCM clustering optimized by Lagrange multipliers, combined with preset high and low clustering thresholds. The classification results include planar points, transition points, and feature points. 3D data points classified as feature points are extracted, and a feature protection mask is generated. If a 3D data point is a feature point, the feature protection mask is 1; otherwise, it is 0. During filtering, feature points are only allowed to move along the normal direction within a preset change region, with a fixed tangential position. Methods for obtaining 3D reality data of products include: Preliminary product data is obtained through sensor scanning, and corresponding parameter combinations are selected based on a pre-established knowledge base. Collect the scanning parameter range of the R group of scanning devices; The initial product data obtained by sensor scanning and the scanning parameter range of R group scanning equipment are used as inputs to the parameter adjustment model to obtain the optimal parameter combination; The initial data of the product is adjusted and tested. For areas that meet the preset adjustment requirements, compensation testing is performed in combination with the preset adjustment strategy. The compensation testing includes reflection pre-compensation and occlusion pre-compensation.

8. A lidar-based real-scene 3D result product display system, implementing the lidar-based real-scene 3D result product display method according to any one of claims 1-7, characterized in that, include: Real-scene scanning module: Uses 3D scanning equipment to scan the target product and obtain real-scene 3D data of the product; Preprocessing module: Performs multi-step denoising, layered collaborative hole filling, and optimization on the 3D data to obtain preprocessed 3D data; Model building module: Using preprocessed 3D data, construct a 3D model of the target product using 3D modeling software; Virtual Display Module: Creates a virtual display environment in the virtual reality engine and imports 3D models into the virtual display environment; Human-computer interaction module: Used to design interactive functions for virtual reality scenes and receive user feedback data in real time to optimize the 3D model of the target product.

Images