Three-dimensional model reconstruction method based on point cloud data processing

Abstract

The invention relates to the technical field of three-dimensional model reconstruction, and discloses a three-dimensional model reconstruction method based on point cloud data processing, which comprises the following steps: step 1, point cloud data acquisition and multi-modal fusion: acquiring point cloud data through a laser radar, an RGB camera and multi-view three-dimensional reconstruction to obtain a point cloud set, aligning the point cloud data with an RGB image to obtain a point cloud image set; establishing a mapping relation between point cloud coordinates and image pixel coordinates through an internal parameter matrix and an external parameter matrix of the camera to obtain aligned multi-modal point cloud data; 2, preprocessing the point cloud data: performing cleaning and downsampling processing on the multi-modal point cloud data obtained in the step 1; by combining point cloud data obtained by a laser radar, an RGB camera and multi-view three-dimensional reconstruction, accurate alignment of different sensor data is realized, high-precision and complete multi-modal point cloud data is obtained, blind areas of all sensors are effectively supplemented, and the precision and reliability of a three-dimensional reconstruction result are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional model reconstruction, and specifically to a three-dimensional model reconstruction method based on point cloud data processing. Background Art

[0002] With the rapid development of computer vision, sensor technology and three-dimensional modeling technology, point cloud data processing plays an increasingly important role in various applications, especially in the fields of three-dimensional model reconstruction, virtual reality, industrial design, autonomous driving, robot perception and digital protection of cultural heritage. As a high-precision spatial data representation method, point cloud data can intuitively display the three-dimensional shape and spatial distribution of an object, facilitating the accurate reconstruction of a three-dimensional model. However, in practical applications, the processing of point cloud data faces many challenges. In particular, problems such as how to improve the quality of point cloud data, reduce noise, fill in missing data, and optimize the model surface are difficult problems that need to be solved urgently.

[0003] Currently, the acquisition technology of point cloud data mainly relies on multiple sensors such as lidar, RGB cameras and multi-view stereo reconstruction. Lidar obtains the spatial information of the target object through the emission and reflection of laser beams, and can efficiently obtain accurate three-dimensional point cloud data. RGB cameras capture the color and texture information of the object through images, and are often used to enhance the visual effect of point cloud data. The multi-view stereo reconstruction technology reconstructs three-dimensional point clouds from image data captured from different perspectives, and is widely used in the modeling of irregular or complex scenes.

[0004] However, the existing point cloud data processing methods have the following problems:

[0005] Data collected by different sensors have different coordinate systems and resolutions, resulting in differences between multi-sensor data. Existing data fusion methods cannot accurately align the data, thus affecting the accuracy and effect of subsequent three-dimensional reconstruction.

[0006] During the actual acquisition process, due to occlusion, sensor blind spots, and environmental factors, there are missing areas in the point cloud data. Existing point cloud data completion methods mostly rely on simple interpolation techniques, which are prone to introducing morphological distortions, resulting in an unnatural transition between the completed part and the original data, and affecting the integrity and accuracy of the model.

[0007] Due to the measurement errors of sensors and environmental interference factors, the point cloud data will contain noise points and redundant points, increasing the computational complexity and affecting the finally reconstructed three-dimensional model. Traditional noise reduction and downsampling methods usually have problems such as incomplete processing or inability to retain key features.

[0008] Existing 3D surface reconstruction techniques can generate basic 3D models, but lack optimization of topological structures and processing of details. The generated surfaces are not smooth and lack a sense of reality, resulting in the final 3D models not meeting the requirements of high-precision and high-quality applications.

[0009] Therefore, those skilled in the art provide a 3D model reconstruction method based on point cloud data processing to solve the above-mentioned problems. Summary of the Invention

[0010] Aiming at the deficiencies of the prior art, the present invention provides a 3D model reconstruction method based on point cloud data processing to solve the problems raised in the above background art.

[0011] To achieve the above objectives, the present invention is realized through the following technical solutions: A 3D model reconstruction method based on point cloud data processing, including:

[0012] Step 1: Acquisition and multi-modal fusion of point cloud data: Obtain point cloud data through lidar, RGB cameras, and multi-view stereo reconstruction to get a point cloud set. Then align the point cloud data with the RGB images, and establish a mapping relationship between the point cloud coordinates and the image pixel coordinates through the internal and external parameter matrices of the cameras to obtain the aligned multi-modal point cloud data;

[0013] Step 2: Preprocessing of point cloud data: Clean and downsample the multi-modal point cloud data obtained in Step 1. First, remove the noise points in the point cloud data according to the normal vector differences and local densities of the points, and then use downsampling techniques to uniformly process the point cloud data, retaining the feature points in the key areas. At the same time, perform rigid body transformation on the multi-view point cloud data to achieve preliminary registration, and obtain the registered and denoised point cloud data;

[0014] Step 3: Feature extraction and segmentation: Extract the geometric features in the denoised point cloud data in Step 2, including normal vectors and curvatures, to describe the local surface directions and surface change rates of the points. At the same time, combine the RGB image data to form a high-dimensional feature vector of the point cloud. On the basis of feature extraction, divide the point cloud into multiple regions according to geometric characteristics;

[0015] Step 4: Completion of missing point cloud data: Complement the missing data in the point cloud regions after segmentation in Step 3. For small-scale missing data regions, complete them by neighborhood interpolation. For large-scale missing data, establish an implicit representation model, map the point cloud data into an implicit function to describe the continuous distribution of the point cloud in space, and achieve data completion;

[0016] Step 5, 3D surface generation: Based on the point cloud data completed in Step 4, use the triangulation method to generate an initial 3D surface model, and construct the 3D spatial representation of the point cloud. Then, optimize and adjust the topological structure of the surface model to make the 3D surface smoother, and add texture information to the surface by combining multi-modal image data, and finally generate a textured 3D model;

[0017] Step 6, post-processing: Post-process the 3D surface model generated in Step 5, repair the non-manifold regions and holes in the topological structure, then optimize the distribution of surface points by the surface smoothing method, reduce the error of the rough surface, and convert the model into a standard 3D model file format suitable for engineering applications for visualization and further processing.

[0018] Preferably, in the said Step 1, the intrinsic matrix is a matrix describing the camera imaging geometry, representing the mapping relationship between the physical points in the 3D space projected onto the 2D image plane. The camera intrinsic matrix is expressed as:

[0019]

[0020] where, f x is the focal length of the camera in the horizontal direction, f y is the focal length of the camera in the vertical direction, c x is the optical center coordinate of the camera, c y is the optical center coordinate of the camera;

[0021] In the said Step 1, the extrinsic matrix is used to describe the transformation relationship between the camera coordinate system and the world coordinate system, representing the position and orientation of the camera in the 3D space. The specific form is:

[0022]

[0023] where, r 11 、r 12 、r 13 、r 21 、r 22 、r 23 、r 31 、r 32 and r 33 are all elements of the rotation matrix, and t x 、t y and t z are all translation vectors.

[0024] Preferably, in the said Step 2, the removal of noise points adopts a weighted method based on the normal vector difference of points. The calculation formula for the normal vector difference weight w ij is:

[0025]

[0026] Among them, n i and n j are the normal vectors of points p i and p j , σ n is the weight adjustment coefficient, and n i -n j is the Euclidean distance between the normal vectors.

[0027] Preferably, in step 3, the extraction of the geometric features includes describing the local surface morphology of points by calculating the normal vector and curvature, and the calculation methods of the normal vector and curvature are as follows:

[0028] Normal vector:

[0029] Among them, N i is the normal vector of point i in the point cloud, p i is the coordinate of point i, p j is the coordinate of point j, p j -p i is the Euclidean distance between point i and neighboring point j;

[0030] Curvature:

[0031] Among them, K i is the curvature of point i, p j -p i is the Euclidean distance between point i and neighboring point j, p i is the coordinate of point i, p j is the coordinate of point j.

[0032] Preferably, in step 3, the high-dimensional feature vector of the point cloud includes the normal vector and curvature, and feature fusion is performed by combining RGB image data. The expression of the high-dimensional feature vector:

[0033] f i =[N i , K i , I(p i )],

[0034] Among them, f i is the high-dimensional feature vector of point i, I(p i ) represents the color value of the corresponding pixel of point i in the RGB image, K i is the curvature of point i, and N i is the normal vector of point i in the point cloud.

[0035] Preferably, in step 4, for the completion of the large-scale missing data area, the implicit representation model adopted is a deep learning method based on a convolutional neural network, and data completion is performed by learning the spatial distribution and local features of the point cloud.

[0036] Preferably, the implicit representation model is trained by minimizing the following loss function:

[0037]

[0038] where L is the value of the loss function, D represents the set of known point cloud data, i and j are the data point indices in the dataset, is the predicted value of data point i, is the predicted value of data point j, and p i is the coordinate of point i.

[0039] Preferably, during the generation process of the three-dimensional surface model, the triangulation method used is an algorithm based on Poisson reconstruction, which can recover a smooth three-dimensional surface from incomplete point cloud data;

[0040] The surface optimization process is achieved by minimizing the surface smoothing energy, and the calculation formula of the surface smoothing energy is: E = ∑ i，j∈T ∥p i - p j ∥ 2 ,

[0041] where T represents the set of triangular patches, p i is the coordinate of point i, p j is the coordinate of point j, and E is the total energy.

[0042] Preferably, in step 6, the model repair and optimization process includes hole filling and topological structure repair. The method used is a repair method based on finite element analysis, which restores surface continuity by simulating a physical force field to stretch and smooth the surface.

[0043] Preferably, finite element analysis usually solves physical problems by simulating the response of an object under external forces. In the repair of point cloud data and topological structure repair, finite element analysis is used to simulate the deformation of the object surface under force to achieve surface smoothing and hole filling;

[0044] The finite element repair formula is as follows: Ku = F,

[0045] where K is the stiffness matrix, u is the displacement vector, and F is the force vector;

[0046] During the surface repair process, especially in surface smoothing and hole filling, the continuity of the surface is restored by the method of energy minimization. The formula for energy minimization is:

[0047]

[0048] where E is the total energy, Ω is the surface area, and μ and λ are material parameters. is the displacement gradient, is the second derivative of displacement.

[0049] The present invention provides a three-dimensional model reconstruction method based on point cloud data processing. It has the following beneficial effects:

[0050] 1. By combining the point cloud data obtained from lidar, RGB cameras, and multi-view stereo reconstruction, and using the internal and external parameter matrices of the cameras for precise alignment and optimization, the present invention realizes the effective fusion of data from different sensors, precise coordinate mapping and alignment, solves the problem of data differences between different sensors, ensures the accurate fusion of the complete information of each object and surface in the three-dimensional space, effectively compensates for the blind spots and errors brought by various sensors, and makes the final three-dimensional reconstruction result more accurate and comprehensive. Especially in complex environments or changing scenarios, it ensures the high quality and reliability of three-dimensional reconstruction.

[0051] 2. By establishing an implicit representation model, the present invention maps discontinuous or missing point cloud data into an implicit function, describes the geometric distribution of the point cloud in a continuous manner in space, and realizes the precise restoration of the missing part of the point cloud data through the continuity and smoothness of the implicit function in a large-scale missing area. At the same time, it avoids morphological distortion or unnatural transition during the filling process, obtains a smooth and natural transition filling result, effectively reduces the reconstruction defects caused by interpolation errors or local data inconsistency in traditional methods, and ensures the integrity and continuity of the point cloud data in the missing area.

[0052] 3. By combining the normal vector difference and local density analysis of the point cloud data, the present invention intelligently identifies and removes noise points. At the same time, by adopting a fine downsampling technique, the features of important regions are retained, while redundant points in irrelevant regions are effectively removed. In the preprocessing stage of the point cloud data, through noise reduction and downsampling methods, efficient and precise cleaning of the point cloud data is achieved, the processing speed is improved, and the local features and geometric details of the point cloud data are effectively retained.

[0053] 4. By combining optimized topological structure adjustment and texture mapping techniques during the generation of the three-dimensional surface, the present invention realizes high-quality smoothing of the three-dimensional model surface and restoration of realistic textures, obtains a textured three-dimensional model with high visual effects and realism, and is applicable to precise industrial design and virtual reality applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flowchart of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0055] To enable those skilled in the art to understand the solution of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are partial embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present invention.

[0056] The following will describe the present invention in detail with reference to the accompanying drawings:

[0057] Embodiment:

[0058] Please refer to the atta Figure 1 , the embodiment of the present invention provides a three-dimensional model reconstruction method based on point cloud data processing, including:

[0059] Step 1, acquisition and multi-modal fusion of point cloud data: Obtain point cloud data through lidar, RGB camera and multi-view stereo reconstruction to obtain a point cloud set. Then, align the point cloud data with the RGB image, and establish a mapping relationship between the point cloud coordinates and the image pixel coordinates through the internal parameter matrix and external parameter matrix of the camera to obtain the aligned multi-modal point cloud data;

[0060] Step 2, preprocessing of point cloud data: Clean and downsample the multi-modal point cloud data obtained in Step 1. First, remove the noise points in the point cloud data according to the normal vector difference and local density of the points, and then use the downsampling technology to homogenize the point cloud data, retaining the feature points in the key areas. At the same time, perform rigid body transformation on the multi-view point cloud data to achieve preliminary registration, and obtain the registered and denoised point cloud data;

[0061] Step 3, feature extraction and segmentation: Extract the geometric features in the denoised point cloud data obtained in Step 2, including normal vectors and curvatures, to describe the local surface direction and surface change rate of the points. At the same time, combine the RGB image data to form a high-dimensional feature vector of the point cloud. On the basis of feature extraction, divide the point cloud into multiple regions according to geometric characteristics;

[0062] Step 4, completion of missing point cloud data: Complement the missing data in the point cloud regions after segmentation in Step 3. For small-scale missing data regions, complete them by neighborhood interpolation. For large-scale missing data, establish an implicit representation model, map the point cloud data to an implicit function to describe the continuous distribution of the point cloud in space, and achieve data completion;

[0063] Step 5, 3D surface generation: Based on the point cloud data completed in Step 4, use the triangulation method to generate an initial 3D surface model, and construct the 3D spatial representation of the point cloud. Then, optimize and adjust the topological structure of the surface model to make the 3D surface smoother, and add texture information to the surface by combining multi-modal image data. Finally, generate a textured 3D model;

[0064] Step 6, post-processing: Perform post-processing on the 3D surface model generated in Step 5 to repair non-manifold regions and holes in the topological structure. Then, optimize the distribution of surface points by the surface smoothing method to reduce the error of the rough surface, and convert the model into a standard 3D model file format suitable for engineering applications for visualization and further processing.

[0065] Benefits of Step 1:

[0066] By aligning the point cloud data with the RGB image, fuse the data from different sensors into the same coordinate system, solve the problem of data differences between different sensors, ensure the unity of spatial information and visual information during the reconstruction process, and effectively improve the accuracy and reliability of the final 3D reconstruction model.

[0067] Benefits of Step 2:

[0068] By analyzing the normal vector differences and local density, effectively remove noise points, retain the valid point cloud data, improve the data quality, and reduce the error sources.

[0069] Homogenize the point cloud data by the downsampling method, make the data point distribution more uniform, and at the same time retain the feature points in the key areas to avoid redundant data during processing and reduce the computational burden.

[0070] Perform rigid body transformation on the multi-viewpoint cloud data to achieve preliminary registration, so that the data from different viewpoints can be effectively matched in the same coordinate system, providing an accurate basis for subsequent point cloud processing and model generation.

[0071] Benefits of Step 3:

[0072] Through the geometric features of the normal vector and curvature, the local surface direction and surface change rate of the point cloud data can be accurately described, and the geometric shape and details of the object can be effectively characterized.

[0073] Combine the RGB image information and the geometric features of the point cloud data to form a high-dimensional feature vector, which can provide more abundant information to support the subsequent segmentation and reconstruction processes.

[0074] Divide the point cloud into regions through geometric features, which can independently process different regions, make the subsequent processing more efficient, and can handle complex or irregular scenes.

[0075] Benefits of Step 4:

[0076] Processing small - scale missing data through neighborhood interpolation can effectively fill local gaps, ensure the continuity of point - cloud data, and reduce the unnatural transition phenomenon during the filling process.

[0077] Using an implicit representation model to process large - scale missing data, the implicit function can continuously describe the spatial distribution, avoid the morphological distortion problem in traditional interpolation methods, and achieve a smoother and more natural filling result.

[0078] The filling of missing data can ensure the continuity and integrity of the point - cloud model, avoid the three - dimensional reconstruction defects caused by missing parts in traditional methods, and improve the accuracy and usability of the model.

[0079] Benefits of step 5:

[0080] Generating a preliminary three - dimensional surface model through triangulation can quickly construct a basic three - dimensional mesh structure, laying a foundation for subsequent refinement and optimization.

[0081] Adjusting the topological structure of the surface model through an optimization algorithm makes the three - dimensional surface smoother, removes the acute angles or irregular faces generated by preliminary triangulation, and improves the visual effect and geometric accuracy of the model.

[0082] Combining RGB image data to add texture information to the three - dimensional surface enhances the visual effect and realism of the model, which is especially suitable for industrial design and virtual reality applications that require high - fidelity.

[0083] Benefits of step 6:

[0084] Post - processing the generated three - dimensional surface model to repair non - manifold regions and holes in the topological structure, ensuring the geometric correctness and usability of the model, and avoiding affecting subsequent applications due to topological defects.

[0085] Optimizing the point distribution on the point - cloud surface through surface smoothing methods reduces the errors caused by rough surfaces, makes the three - dimensional model smoother, and is suitable for high - precision engineering applications.

[0086] Converting the finally generated three - dimensional model into a standard three - dimensional model file format suitable for engineering applications facilitates subsequent visualization, virtual reality display, manufacturing, and analysis applications.

[0087] In summary, the present invention provides a complete and efficient three - dimensional model reconstruction process, covering the entire process from the acquisition, pre - processing, and filling of missing data of point - cloud data to the generation and post - processing of the final three - dimensional surface. The design of each step can improve data quality, reduce noise and errors, enhance the accuracy, integrity, and visual effect of the final three - dimensional model, and meet the requirements for high - quality three - dimensional reconstruction in the fields of industrial design, virtual reality, and autonomous driving.

[0088] In Step 1, the intrinsic matrix is a matrix that describes the imaging geometry of the camera, representing the mapping relationship from physical points in three-dimensional space to the two-dimensional image plane. The camera intrinsic matrix is expressed as:

[0089]

[0090] where f x is the focal length of the camera in the horizontal direction, and f y is the focal length of the camera in the vertical direction, and c x is the optical center coordinate of the camera, and c y is the optical center coordinate of the camera;

[0091] In Step 1, the extrinsic matrix is used to describe the transformation relationship between the camera coordinate system and the world coordinate system, representing the position and orientation of the camera in three-dimensional space. The specific form is:

[0092]

[0093] where r 11 , r 12 , r 13 , r 21 , r 22 , r 23 , r 31 , r 32 and r 33 are all elements of the rotation matrix, and t x , t y and t z are all translation vectors.

[0094] The intrinsic matrix and the extrinsic matrix play a crucial role in the point cloud data processing method of the present invention. The intrinsic matrix ensures the accurate alignment of the point cloud data and the RGB image by precisely describing the imaging geometric relationship of the camera. The extrinsic matrix ensures the precise registration of multi-view point cloud data by describing the transformation relationship between the camera and the world coordinate system, thereby improving the accuracy and quality of 3D model reconstruction. The combination of the intrinsic matrix and the extrinsic matrix provides reliable technical support for achieving high-precision and multi-modal fusion 3D reconstruction.

[0095] In Step 2, the removal of noise points adopts a weighted method based on the normal vector difference of points. The normal vector difference weight w ij is calculated by the formula:

[0096]

[0097] where n i and n j are the normal vectors of points p i and p j , σ n is the weight adjustment coefficient, and ni -n j is the Euclidean distance between the normal vectors.

[0098] In step 2, removing noise points through a weighted method based on the normal vector differences of points has significant advantages. The calculation of the normal vector difference weights can effectively evaluate the changes in the surface morphology between adjacent points. By performing weighted calculations on the Euclidean distance between the normal vectors, it is possible to more accurately identify and remove noise points that are significantly inconsistent with the surrounding point cloud structure. Compared with traditional methods based on distance or local density, the weighted method of normal vector differences can more sensitively capture the changes in the surface morphology in the point cloud data and more efficiently remove the outlier noise introduced by sensor errors or environmental factors. Improving the accuracy of noise point removal can retain the details and geometric features of the key areas, thereby ensuring the accuracy and quality of the final point cloud data and reducing the error propagation in subsequent processing.

[0099] In step 3, the extraction of geometric features includes describing the local surface morphology of points by calculating the normal vector and curvature. The calculation methods of the normal vector and curvature are as follows:

[0100] Normal vector:

[0101] where N i is the normal vector of point i in the point cloud, p i is the coordinate of point i, p j is the coordinate of point j, p j -p i is the Euclidean distance between point i and neighboring point j;

[0102] Curvature:

[0103] where K i is the curvature of point i, p j -p i is the Euclidean distance between point i and neighboring point j, p i is the coordinate of point i, p j is the coordinate of point j.

[0104] In step 3, geometric features of the point cloud data are extracted by calculating the normal vector and curvature, which can effectively describe the local surface morphology of points. The method has significant advantages. The normal vector calculation can accurately reflect the local geometric structure of the object surface in the point cloud by evaluating the surface direction of each point in the local area, which is crucial for subsequent point cloud segmentation and reconstruction. The calculation method of the normal vector is based on the geometric relationship between adjacent points, which helps to distinguish different surface features and improve the recognition ability of complex shapes in the point cloud. The curvature calculation can effectively capture the degree of curvature of the object surface by measuring the geometric change rate between a point and its neighboring points, providing important information for details, edges, and complex surfaces in the point cloud. The combination of the normal vector and curvature helps to accurately identify and describe the shape features of different regions in the point cloud, improve the detail retention ability of the point cloud data, and at the same time provide a key basis for point cloud segmentation and subsequent processing, enhancing the accuracy and reliability of 3D reconstruction.

[0105] In step 3, the high-dimensional feature vector of the point cloud includes the normal vector and curvature, and feature fusion is performed by combining RGB image data. The expression of the high-dimensional feature vector:

[0106] f i =[N i ,K i ,I(p i )],

[0107] where f i is the high-dimensional feature vector of point i, I(p i ) represents the color value of the corresponding pixel of point i in the RGB image, K i is the curvature of point i, and N i is the normal vector of point i in the point cloud.

[0108] In step 3, by combining the normal vector, curvature, and RGB image data to form the high-dimensional feature vector of the point cloud, the feature fusion method has significant advantages. The construction of the high-dimensional feature vector retains the geometric information of the point cloud data and combines the color information in the RGB image, providing a richer and more comprehensive description. Multimodal feature fusion can enhance the distinguishability between different points in the point cloud, especially in areas with complex surfaces and color differences. By supplementing the deficiencies of geometric features with color information, it improves the recognition and classification ability of the point cloud. Especially in the reconstruction of complex scenes, using the color information of the RGB image effectively helps to distinguish different regions with similar geometric structures, reduces errors or ambiguities caused by single geometric features, and realizes more accurate point cloud segmentation and feature extraction, making the point cloud data show stronger adaptability and accuracy in diverse and complex actual scenes, providing a more accurate basis for subsequent 3D model reconstruction.

[0109] In step 4, for the completion of a large-scale missing data region, the implicit representation model adopted is a deep learning method based on a convolutional neural network, which performs data completion by learning the spatial distribution and local features of the point cloud;

[0110] The implicit representation model is trained by minimizing the following loss function:

[0111]

[0112] where L is the value of the loss function, D represents the set of known point cloud data, i and j are the data point indices in the dataset, is the predicted value of data point i, is the predicted value of data point j, and p i is the coordinate of point i.

[0113] In step 4, by adopting an implicit representation model based on a convolutional neural network to complete a large-scale missing data region, the completion effect of the point cloud data can be significantly improved. The implicit representation model utilizes deep learning technology to effectively predict and complete the point cloud data in the missing region by learning the spatial distribution and local geometric features of the point cloud. The core advantage of this method is that the implicit representation model can automatically capture the complex structure and continuity of the point cloud data. By minimizing the loss function, it can generate coherent and smooth completion results within the missing data region, maintaining the geometric consistency and local details of the data. Compared with traditional interpolation methods, the deep learning method shows stronger robustness and accuracy when dealing with large-scale missing data, especially avoiding problems such as morphological distortion or unnatural transitions caused by traditional methods. By modeling the spatial characteristics of the point cloud, the implicit representation method can better restore the true shape of the missing part, making the completed point cloud data more complete and natural, and ultimately improving the accuracy and quality of 3D model reconstruction.

[0114] During the generation process of the 3D surface model, the triangulation method used is an algorithm based on Poisson reconstruction, which can restore a smooth 3D surface from incomplete point cloud data;

[0115] The process of surface optimization is achieved by minimizing the surface smoothing energy, and the formula for calculating the surface smoothing energy is: E = ∑ i，j∈T ∥p i - p j ∥ 2 ,

[0116] where T represents the set of triangular patches, p i is the coordinate of point i, p j is the coordinate of point j, and E is the total energy.

[0117] During the generation of a 3D surface model, a triangulation method based on Poisson reconstruction and a surface optimization process are used. By minimizing the surface smoothing energy, the quality of the generated surface can be significantly improved. The Poisson reconstruction algorithm can effectively recover the smooth 3D surface in incomplete point cloud data, and is particularly suitable for dealing with surface missing problems caused by limitations in data acquisition or sensor blind spots. By solving the Poisson equation, this algorithm can generate a smooth and continuous surface globally, enabling irregular or sparse point cloud data to be converted into a 3D model with physical consistency. The minimization of the surface smoothing energy further optimizes the surface quality of the model. By balancing the geometric relationships between points, it reduces noise and unnatural transitions on the surface, ensuring a smooth and natural surface. This optimization process effectively removes rough, distorted, or incoherent parts of the surface, enhancing the visual effect and realism of the 3D model. Therefore, the accuracy and usability of the final 3D model can be significantly improved, especially in applications such as industrial design and virtual reality that require high-precision and high-quality surface reconstruction, demonstrating strong advantages.

[0118] In step 6, the model repair and optimization process includes hole filling and topology repair. The method used is a repair method based on finite element analysis. By simulating the physical force field to stretch and smooth the surface, the surface continuity is restored.

[0119] Finite element analysis usually solves physical problems by simulating the response of an object under external forces. In the repair of point cloud data and topology repair, finite element analysis is used to simulate the deformation of the object's surface under force to achieve surface smoothing and hole filling.

[0120] The finite element repair formula is as follows: Ku = F,

[0121] where K is the stiffness matrix, u is the displacement vector, and F is the force vector;

[0122] During the surface repair process, especially in surface smoothing and hole filling, the continuity of the surface is restored by the method of energy minimization. The formula for energy minimization is:

[0123]

[0124] where E is the total energy, Ω is the surface area, μ and λ are material parameters, is the displacement gradient, is the second derivative of the displacement.

[0125] In step 6, model repair and optimization are carried out through a repair method based on finite element analysis, which has significant advantages especially in hole filling and topological structure repair. Finite element analysis can effectively restore the continuity and smoothness of the surface by simulating the response of the object surface under external forces. Through the calculation of the stiffness matrix, displacement vector, and force vector, the finite element method can simulate the physical deformation of the surface during stretching and smoothing, and accurately fill the missing areas and repair the holes in the topological structure. The key advantage of this method is that it can perform surface repair according to physical laws, avoiding artificial repair marks or unnatural morphological transitions in traditional methods.

[0126] During the surface smoothing and hole filling process, the repair method based on energy minimization can effectively restore surface continuity. By minimizing the total energy involved in the repair process, finite element analysis ensures that the repaired surface is not only smooth but also natural, avoiding geometric distortion caused by unreasonable repair. The accurate calculation of material parameters, displacement gradients, and second-order displacement derivatives ensures the physical rationality of surface deformation, making the repaired model have higher accuracy and consistency. It is particularly suitable for engineering applications that require high-precision and high-quality surface repair, can significantly improve the quality of 3D models, making them more suitable for practical use, and shows great potential especially in virtual reality, industrial design, and other fields that require precise 3D reconstruction.

[0127] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A three-dimensional model reconstruction method based on point cloud data processing, characterized in that: include: Step 1: Collection and multimodal fusion of point cloud data: Obtain point cloud data through lidar, RGB camera and multi-view stereo reconstruction to obtain a point cloud set, then align the point cloud data with the RGB image, and establish a mapping relationship between point cloud coordinates and image pixel coordinates through the camera's intrinsic parameter matrix and extrinsic parameter matrix to obtain aligned multimodal point cloud data; Step 2, preprocessing of point cloud data: clean and downsample the multimodal point cloud data obtained in step 1. First, remove the noise points in the point cloud data according to the normal vector difference and local density of the points. Then, use the downsampling technology to homogenize the point cloud data and retain the feature points in the key areas. At the same time, perform rigid body transformation on the multi-view point cloud data to achieve preliminary registration and obtain the registered denoised point cloud data. Step 3, feature extraction and segmentation: Extract geometric features from the denoised point cloud data in step 2, including normal vectors and curvatures, to describe the local surface direction and surface change rate of the points. At the same time, combine the RGB image data to form a high-dimensional feature vector of the point cloud. Based on the feature extraction, divide the point cloud into multiple regions according to the geometric characteristics. Step 4, missing point cloud data completion: the missing data in the point cloud area segmented in step 3 is completed. For small-scale missing data areas, the data is completed by neighborhood interpolation. For large-scale missing data, an implicit representation model is established to map the point cloud data to an implicit function to describe the continuous distribution of the point cloud in space and achieve data completion. Step 5, 3D surface generation: Based on the point cloud data completed in step 4, the initial 3D surface model is generated by triangulation method, and the 3D space representation of the point cloud is constructed. Then, the topological structure of the surface model is optimized and adjusted to make the 3D surface smoother, and texture information is added to the surface by combining multimodal image data, and finally a textured 3D model is generated; Step 6, post-processing: Post-process the 3D surface model generated in step 5, repair the non-manifold areas and holes in the topological structure, and then optimize the distribution of surface points through surface smoothing methods to reduce the error of the rough surface, and convert the model into a standard 3D model file format suitable for engineering applications for visualization and further processing.

2. The three-dimensional model reconstruction method based on point cloud data processing according to claim 1, characterized in that: The intrinsic parameter matrix in step 1 is a matrix that describes the camera imaging geometry, and represents the mapping relationship between the physical point in the three-dimensional space and the two-dimensional image plane. The camera intrinsic parameter matrix is ​​expressed as: Among them, f x is the focal length of the camera in the horizontal direction, f y is the focal length of the camera in the vertical direction, c x is the coordinate of the optical center of the camera, c y is the coordinate of the optical center of the camera; The external parameter matrix in step 1 is used to describe the transformation relationship between the camera coordinate system and the world coordinate system, indicating the position and orientation of the camera in three-dimensional space. The specific form is: Among them, r 11 、r 12 、r 13 、r 21 、r 22 、r 23 、r 31 、r 32 and r 33 are all rotation matrix elements, t x ,t y and t z are translation vectors.

3. The three-dimensional model reconstruction method based on point cloud data processing according to claim 1, characterized in that: In step 2, the noise points are removed by using a weighted method based on the normal vector difference of the points, and the normal vector difference weight w ij The calculation formula is: Among them, n i and n j For point p i and p j The normal vector, σ n is the weight adjustment coefficient, n i -n j is the Euclidean distance between normal vectors.

4. The three-dimensional model reconstruction method based on point cloud data processing according to claim 1, characterized in that: In step 3, the extraction of the geometric features includes describing the local surface morphology of the point by calculating the normal vector and the curvature, and the calculation method of the normal vector and the curvature is: Normal vector: Among them, N i is the normal vector of point i in the point cloud, p i is the coordinate of point i, p j is the coordinate of point j, p j -p i is the Euclidean distance between point i and its neighboring point j; Curvature: Among them, K i is the curvature of point i, p j -p i is the Euclidean distance between point i and its neighboring point j, p i is the coordinate of point i, p j are the coordinates of point j.

5. The three-dimensional model reconstruction method based on point cloud data processing according to claim 1, characterized in that: In step 3, the high-dimensional feature vector of the point cloud includes the normal vector and the curvature, and the feature fusion is performed in combination with the RGB image data. The expression of the high-dimensional feature vector is: f i =[N i ,K i ,I(p i )], Among them, f i is the high-dimensional feature vector of point i, I(p i ) represents the color value of the corresponding pixel at point i in the RGB image, K i is the curvature of point i, N i is the normal vector of point i in the point cloud.

6. The three-dimensional model reconstruction method based on point cloud data processing according to claim 1, characterized in that: In step 4, the implicit representation model used to complete the large-scale missing data area is a deep learning method based on a convolutional neural network, which completes the data by learning the spatial distribution and local features of the point cloud.

7. The three-dimensional model reconstruction method based on point cloud data processing according to claim 6, characterized in that: The implicit representation model is trained by minimizing the following loss function: Among them, L is the value of the loss function, D represents the set of known point cloud data, i and j are the data point indices in the data set, is the predicted value for data point i, is the predicted value for data point j, p i is the coordinate of point i.

8. The three-dimensional model reconstruction method based on point cloud data processing according to claim 1, characterized in that: In the process of generating the three-dimensional surface model, the triangulation method used is an algorithm based on Poisson reconstruction, which can restore a smooth three-dimensional surface in incomplete point cloud data; The surface optimization process is achieved by minimizing the surface smoothing energy, which is calculated by the formula: E = ∑ i，j∈T ∥p i -p j ∥ 2 , Where T represents a set of triangular faces, p i is the coordinate of point i, p j are the coordinates of point j, and E is the total energy.

9. The three-dimensional model reconstruction method based on point cloud data processing according to claim 1, characterized in that: In step 6, the model repair and optimization process includes hole filling and topological structure repair, and the method adopted is a repair method based on finite element analysis, which stretches and smoothes the surface by simulating the physical force field to restore the surface continuity.

10. The three-dimensional model reconstruction method based on point cloud data processing according to claim 9, characterized in that: The finite element analysis is usually used to solve physical problems by simulating the response of an object under external force. In point cloud data repair and topological structure repair, finite element analysis is used to simulate the deformation of the object surface after being subjected to force, so as to achieve surface smoothing and hole filling. The finite element repair formula is as follows: Ku = F, Where K is the stiffness matrix, u is the displacement vector, and F is the force vector; In the process of surface repair, especially surface smoothing and hole filling, the continuity of the surface is restored by energy minimization. The formula for energy minimization is: Where E is the total energy, Ω is the surface area, μ and λ are material parameters, is the displacement gradient, is the second derivative of the displacement.

Images