Large component three-dimensional reconstruction method based on multi-view point cloud registration

Abstract

The invention relates to the technical field of large component three-dimensional reconstruction, and discloses a large component three-dimensional reconstruction method based on multi-view point cloud registration, which comprises the following steps: step 1, carrying out point cloud data acquisition on a target large component from different views to obtain point cloud data of the target large component; each visual angle covers different areas of the component, the adjacent areas are overlapped, and the overlapping degree is 20%-50%. According to the large component three-dimensional reconstruction method based on multi-view point cloud registration, the problems that a large component is large in size and complex in structure are solved through multi-view collection of point cloud data of different areas, the overlapping degree of adjacent areas is 20%-50%, the integrity and continuity of data are ensured, an iterative nearest point algorithm or an improved version of the iterative nearest point algorithm is adopted, and the reconstruction efficiency is improved. Accurate alignment of different view angle data is ensured, semantic segmentation and feature point matching optimization are carried out in combination with a deep learning technology, and the speed and efficiency of data processing are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction of large components, and in particular to a three-dimensional reconstruction method of large components based on multi-view point cloud registration. Background Art

[0002] In the fields of industrial manufacturing, construction, and cultural relics protection, the three-dimensional reconstruction of large components is of great significance. Traditional methods usually rely on single-perspective point cloud acquisition, but this method has obvious limitations when processing large components. Due to the large size and complex structure of large components, a single perspective is difficult to cover the entire component, and it cannot effectively handle occluded areas and complex surfaces.

[0003] Existing 3D reconstruction methods mainly rely on a single data source (such as laser or stereo vision), which makes it difficult to handle the complexity of large targets and the consistency of multi-view data. In addition, traditional methods often ignore the fusion of multi-region data during point cloud data processing, resulting in insufficient accuracy and completeness of the reconstructed model. Summary of the Invention

[0004] The present invention is proposed in view of the fact that the above-mentioned existing 3D reconstruction methods mainly rely on a single data source (such as laser or stereo vision), which makes it difficult to handle the complexity of large targets and the consistency of multi-view data. In addition, traditional methods often ignore the fusion of multi-region data during point cloud data processing, resulting in insufficient accuracy and integrity of the reconstructed model.

[0005] Therefore, the purpose of the present invention is to provide a three-dimensional reconstruction method for large components based on multi-view point cloud registration, which aims to solve the problem of large size and complex structure of large components, with the overlap of adjacent areas being 20%-50%, ensuring the integrity and continuity of the data.

[0006] To solve the above technical problems, the present invention provides the following technical solution: a method for 3D reconstruction of large components based on multi-view point cloud registration, comprising the following steps:

[0007] Step 1: Collect point cloud data of the target large component from different perspectives. Each perspective covers a different area of the component, and there is overlap between adjacent areas with an overlap of 20%-50%;

[0008] Step 2: Select one point cloud data as a reference point cloud, and register the remaining point cloud data with the reference point cloud. The registration method includes the iterative closest point (ICP) algorithm or its improved version.

[0009] Step 3: De-noise the registered point cloud data by using Gaussian filtering, median filtering or statistical filtering methods to remove noise points.

[0010] Step 4: Perform filling processing on the denoised point cloud data, and use radial basis function interpolation, nearest neighbor interpolation, or interpolation methods based on machine learning to fill the missing areas;

[0011] Step 5: Perform smoothing processing on the filled point cloud data, and use the moving average method, surface fitting, or curvature-based smoothing algorithm to smooth the point cloud;

[0012] Step 6: Based on the processed point cloud dataset, use a triangular mesh reconstruction algorithm to generate a three-dimensional model. The triangular mesh reconstruction algorithm includes the Poisson reconstruction algorithm, Alpha shape algorithm, or spherical harmonics method;

[0013] Step 7: Perform texture mapping on the three-dimensional model, and map the texture data collected by a color camera or multispectral camera to the surface of the three-dimensional model through UV coordinate mapping or feature point alignment method to generate the final three-dimensional reconstruction model.

[0014] As a preferred solution of the three-dimensional reconstruction method for large components based on multi-viewpoint cloud registration according to the present invention, wherein: the acquisition device includes a laser scanner, a depth camera, or an RGB-D camera.

[0015] As a preferred solution of the three-dimensional reconstruction method for large components based on multi-viewpoint cloud registration according to the present invention, wherein: the improved version of the registration method includes the ICP algorithm based on feature point matching, the ICP algorithm combined with global optimization, or the registration algorithm based on deep learning.

[0016] As a preferred solution of the three-dimensional reconstruction method for large components based on multi-viewpoint cloud registration according to the present invention, wherein: in the step 7, the texture data is collected by a color camera or multispectral camera, and the texture data is mapped to the surface of the three-dimensional model through UV coordinate mapping or feature point alignment method.

[0017] As a preferred solution of the three-dimensional reconstruction method for large components based on multi-viewpoint cloud registration according to the present invention, wherein: the method is applicable to fields such as large mechanical components, building structures, cultural relics protection, etc., and is particularly applicable to large components with a size greater than 1 meter.

[0018] As a preferred solution of the three-dimensional reconstruction method for large components based on multi-viewpoint cloud registration according to the present invention, wherein: it further includes a three-dimensional reconstruction system, and the three-dimensional reconstruction system includes:

[0019] Data acquisition module: used to collect point cloud data of the target large component from different viewpoints;

[0020] Registration module: used to register the collected point cloud data with the reference point cloud;

[0021] Data processing module: used for denoising, filling, and smoothing the registered point cloud data;

[0022] Three-dimensional model reconstruction module: used for generating a three-dimensional model based on the processed point cloud data set;

[0023] Texture mapping module: used for mapping texture data onto the surface of the three-dimensional model to generate the final three-dimensional reconstruction model.

[0024] As a preferred solution of the three-dimensional reconstruction method for large components based on multi-view point cloud registration according to the present invention, wherein: the three-dimensional reconstruction system further includes an optimization module for optimizing the generated three-dimensional model, and the optimization processing includes model simplification, detail enhancement, or lighting adjustment.

[0025] Compared with the prior art, the present invention has at least the following beneficial effects:

[0026] 1. The present invention collects point cloud data of different regions through multiple views, solves the problems of large size and complex structure of large components, the overlap degree of adjacent regions is 20%-50%, ensuring the integrity and continuity of the data, and adopts the iterative closest point algorithm or its improved version to ensure the precise alignment of data from different views, and combines deep learning technology for semantic segmentation and feature point matching optimization, significantly improving the speed and efficiency of data processing. At the same time, this method is applicable to fields such as large mechanical components, building structures, and cultural relic protection, especially applicable to large components with a size greater than 1 meter.

[0027] 2. The present invention improves the quality of point cloud data through processing steps such as denoising, filling, and smoothing, and the finally generated three-dimensional model has higher accuracy and integrity. And through texture mapping, the collected image data is mapped onto the surface of the three-dimensional model to generate the final three-dimensional reconstruction model with a realistic sense and visual effect. Description of the Drawings

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. Among them:

[0029] Figure 1 It is the overall flow schematic diagram of the three-dimensional reconstruction method for large components based on multi-view point cloud registration of the present invention.

[0030] Figure 2 It is the system framework schematic diagram of the three-dimensional reconstruction method for large components based on multi-view point cloud registration of the present invention. Detailed Embodiments

[0031] To make the above objects, features, and advantages of the present invention more apparent and understandable, the following provides a detailed description of the specific embodiments of the present invention in conjunction with the accompanying drawings of the specification.

[0032] In the following description, many specific details are set forth in order to provide a thorough understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the spirit of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0033] Embodiment

[0034] Refer to Figure 1-2 , which is an embodiment of the present invention, provides a three-dimensional reconstruction method for large components based on multi-viewpoint cloud registration. This three-dimensional reconstruction method for large components based on multi-viewpoint cloud registration includes the following steps:

[0035] Step 1: Collect point cloud data of the target large component from different viewpoints. Each viewpoint covers different regions of the component, and there is an overlap between adjacent regions, with the overlap degree being 20%-50%.

[0036] Step 2: Select one point cloud data as the reference point cloud, and register the remaining point cloud data with the reference point cloud. The registration method includes the Iterative Closest Point (ICP) algorithm or its improved version.

[0037] Step 3: Denoise the registered point cloud data, and use Gaussian filtering, median filtering, or statistics-based filtering methods to remove noise points.

[0038] Step 4: Fill in the denoised point cloud data, and use radial basis function interpolation, nearest neighbor interpolation, or machine learning-based interpolation methods to fill in the missing regions.

[0039] Step 5: Smooth the filled point cloud data, and use the moving average method, surface fitting, or curvature-based smoothing algorithm to smooth the point cloud.

[0040] Step 6: Based on the processed point cloud data set, use a triangular mesh reconstruction algorithm to generate a three-dimensional model. The triangular mesh reconstruction algorithm includes the Poisson reconstruction algorithm, Alpha shape algorithm, or spherical harmonics method.

[0041] Step 7: Perform texture mapping on the three-dimensional model, and map the texture data collected by a color camera or a multispectral camera to the surface of the three-dimensional model through UV coordinate mapping or feature point alignment-based methods to generate the final three-dimensional reconstruction model.

[0042] During use:

[0043] 1. Data collection

[0044] Use a laser scanner, depth camera, or RGB-D camera to collect point cloud data of the target large component from different perspectives. Each perspective covers different areas of the component, and there is an overlap of 20%-50% between adjacent areas to ensure the integrity and continuity of the data. The recommended acquisition sequence starts from perspective A and proceeds in the order of A-H. The angle between perspective A and perspectives B and H is approximately 30°, the angle between perspective E and perspectives F and D is approximately 30°, and the angle between the remaining perspectives is approximately 60°. In perspectives C and G, the distance between the sensor and the target object is approximately 1.5m, and in the remaining perspectives, the distance between the sensor and the target object is approximately 1.2m to obtain the 3D information of the point cloud within the maximum range.

[0045] 2. Point cloud preprocessing

[0046] 2.1 Remove background points

[0047] Distinguish the point cloud of the target object from other point clouds through conditional filtering. Set appropriate thresholds, consider the points that meet the thresholds as points in the target object and save them, and remove the points outside the threshold range.

[0048] 2.2 Remove noise points

[0049] Use statistical filtering to remove outliers. Calculate the average distance from each point to k neighboring points. This average distance follows a Gaussian distribution. Calculate its mean and standard deviation, and locate the points with an average distance greater than a certain threshold as outliers and remove them.

[0050] 2.3 Point cloud voxelization

[0051] Adopt the method of voxelized grid for downsampling. Create a three-dimensional voxel grid for the input original point cloud, and use the centroid to replace other points in the voxel grid within the voxel grid to reduce the number of point clouds while ensuring the basic features of the point cloud.

[0052] 3. Point cloud registration

[0053] Select one point cloud data as the reference point cloud and register the remaining point cloud data with the reference point cloud. The registration method uses the Iterative Closest Point (ICP) algorithm or its improved versions, such as the ICP algorithm based on feature point matching, the ICP algorithm combined with global optimization, or the registration algorithm based on deep learning. These improved methods can improve the registration accuracy and efficiency and solve the problem that the traditional ICP algorithm is prone to falling into local optimality in complex scenarios.

[0054] 3.1 Coarse registration

[0055] Coarse registration is performed using the Super4PCS algorithm. Four coplanar points are selected from the source point cloud to form a point basis E, and its ratio and distance are calculated. In the target point cloud, spheres are drawn with each point as the center and the calculated distance as the radius, obtaining two point pair sets. The intersection points and angles between the point pairs are calculated according to the affine invariance for the matching of the four-point basis on the same plane. The transformation matrix of all four-point sets corresponding to the point basis E in the target point cloud is calculated, and the transformation matrix with the highest registration accuracy is selected for global transformation.

[0056] 3.2 Fine registration

[0057] Fine registration is performed using the point-to-plane ICP algorithm. The tangent plane of the target point cloud is calculated, and the nearest tangent plane from the points in the source point cloud to the target point cloud is found to form corresponding point pairs. The normal vector angle threshold strategy is used to remove the wrong point pairs, the target error function is constructed, and the least squares method is used to fit the optimal model to obtain the optimal transformation matrix to minimize the sum of squares of the distances from the source points to the corresponding target point tangent planes.

[0058] 4. Data processing

[0059] 4.1 Denoising processing

[0060] The registered point cloud data is subjected to denoising processing, and Gaussian filtering, median filtering or statistics-based filtering methods are used to remove the noise points. This step can effectively reduce the noise interference in the data and improve the accuracy of subsequent processing.

[0061] 4.2 Filling processing

[0062] The denoised point cloud data is subjected to filling processing, and radial basis function interpolation, nearest neighbor interpolation or machine learning-based interpolation methods are used to fill the missing regions. The filling processing can repair the holes and missing parts in the data and make the point cloud data more complete.

[0063] 4.3 Smoothing processing

[0064] The filled point cloud data is subjected to smoothing processing, and the moving average method, surface fitting or curvature-based smoothing algorithm is used to smooth the point cloud. The smoothing processing can remove the sharp edges and irregular parts in the data and make the point cloud data smoother and more natural.

[0065] 5. 3D model reconstruction

[0066] Based on the processed point cloud data set, a 3D model is generated using the triangular mesh reconstruction algorithm. Commonly used triangular mesh reconstruction algorithms include the Poisson reconstruction algorithm, Alpha shape algorithm and spherical harmonics method. These algorithms can generate high-quality 3D models according to the geometric and topological information of the point cloud data.

[0067] 6. Texture mapping

[0068] Perform texture mapping on the generated 3D model, and map the texture data collected by a color camera or a multispectral camera to the surface of the 3D model through UV coordinate mapping or a method based on feature point alignment. Texture mapping can provide a realistic visual effect for the 3D model, making it more vivid and visually appealing.

[0069] 7. Model Optimization

[0070] Perform optimization processing on the generated 3D model, including model simplification, detail enhancement, or lighting adjustment. Model simplification can reduce the complexity of the model and improve the rendering efficiency of the model; detail enhancement can highlight the detailed parts of the model and make it more refined; lighting adjustment can improve the lighting effect of the model and make it more in line with the actual scene.

[0071] 8. Application Fields

[0072] The present invention is applicable to fields such as large mechanical components, building structures, cultural relic protection, etc., and is particularly applicable to large components with a size greater than 1 meter. In these fields, 3D reconstruction technology can be used for digital modeling, structural analysis, cultural relic protection, etc. of components, and has broad application prospects.

[0073] 9. 3D Reconstruction System

[0074] The present invention also includes a 3D reconstruction system, which includes a data acquisition module, a registration module, a data processing module, a 3D model reconstruction module, a texture mapping module, and an optimization module. These modules work together to achieve a complete process from data acquisition to 3D model generation.

[0075] 9.1 Data Acquisition Module

[0076] The data acquisition module is used to collect point cloud data of the target large component from different perspectives. This module can use devices such as a laser scanner, a depth camera, or an RGB-D camera to ensure that the collected data has high precision and high resolution.

[0077] 9.2 Registration Module

[0078] The registration module is used to register the collected point cloud data with the reference point cloud. This module uses the ICP algorithm or its improved version to ensure the precise alignment of point cloud data from different perspectives in 3D space.

[0079] 9.3 Data Processing Module

[0080] The data processing module is used to perform denoising, filling, and smoothing processing on the registered point cloud data. This module uses a variety of filtering and interpolation methods to improve the quality and integrity of the point cloud data.

[0081] 9.4 3D Model Reconstruction Module

[0082] The 3D model reconstruction module is used to generate a 3D model based on the processed point cloud dataset. This module adopts a triangular mesh reconstruction algorithm to generate a high-quality 3D model.

[0083] 9.5 Texture mapping module

[0084] The texture mapping module is used to map texture data onto the surface of the 3D model to generate the final 3D reconstruction model. This module adopts methods such as UV coordinate mapping or feature point alignment-based methods to ensure the precise alignment of the texture data with the 3D model.

[0085] 9.6 Optimization module

[0086] The optimization module is used to perform optimization processing on the generated 3D model. This module can perform model simplification, detail enhancement, or lighting adjustment to improve the rendering efficiency and visual effect of the model.

[0087] Through the above specific implementation manners, the present invention can efficiently complete the 3D reconstruction of large components, generate high-quality 3D models, and meet the application requirements of different fields.

[0088] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A three-dimensional reconstruction method for large components based on multi-viewpoint cloud registration, characterized in that Including the following steps: Step 1: Collect point cloud data of the target large component from different perspectives. Each perspective covers different regions of the component, and there is an overlap between adjacent regions, with the overlap degree being 20%-50%; Step 2: Select one point cloud data as the reference point cloud, and register the remaining point cloud data with the reference point cloud. The registration method includes the Iterative Closest Point (ICP) algorithm or its improved version; Step 3: Denoise the registered point cloud data, and use Gaussian filtering, median filtering or statistics-based filtering methods to remove noise points; Step 4: Fill the denoised point cloud data, and use radial basis function interpolation, nearest neighbor interpolation or machine learning-based interpolation methods to fill the missing regions; Step 5: Smooth the filled point cloud data, and use the moving average method, surface fitting or curvature-based smoothing algorithm to smooth the point cloud; Step 6: Based on the processed point cloud data set, use the triangular mesh reconstruction algorithm to generate a three-dimensional model. The triangular mesh reconstruction algorithm includes the Poisson reconstruction algorithm, Alpha shape algorithm or spherical harmonics method; Step 7: Perform texture mapping on the three-dimensional model, and map the texture data collected by a color camera or a multispectral camera to the surface of the three-dimensional model through UV coordinate mapping or feature point alignment-based method to generate the final three-dimensional reconstruction model.

2. The three-dimensional reconstruction method of large components based on multi-viewpoint cloud registration according to claim 1, wherein: The acquisition device includes a laser scanner, a depth camera or an RGB-D camera.

3. The three-dimensional reconstruction method of large components based on multi-viewpoint cloud registration according to claim 2, characterized in that: The improved version of the registration method includes the ICP algorithm based on feature point matching, the ICP algorithm combined with global optimization or the registration algorithm based on deep learning.

4. The three-dimensional reconstruction method of large components based on multi-viewpoint cloud registration according to claim 3, characterized in that: In Step 7, the texture data is collected by a color camera or a multispectral camera, and the texture data is mapped to the surface of the three-dimensional model through UV coordinate mapping or feature point alignment-based method.

5. The three-dimensional reconstruction method of large components based on multi-viewpoint cloud registration according to claim 4, characterized in that: The method is applicable to fields such as large mechanical components, building structures, cultural relics protection, etc., and is particularly applicable to large components with a size greater than 1 meter.

6. The three-dimensional reconstruction method of large components based on multi-viewpoint cloud registration according to claim 5, characterized in that: It also includes a three-dimensional reconstruction system, and the three-dimensional reconstruction system includes: Data acquisition module: used to collect point cloud data of the target large component from different perspectives; Registration module: used to register the collected point cloud data with the reference point cloud; Data processing module: used to denoise, fill and smooth the registered point cloud data; Three-dimensional model reconstruction module: used to generate a three-dimensional model based on the processed point cloud data set; Texture mapping module: used to map the texture data to the surface of the three-dimensional model to generate the final three-dimensional reconstruction model.

7. The three-dimensional reconstruction method of large components based on multi-viewpoint cloud registration according to claim 6, characterized in that: The three-dimensional reconstruction system also includes an optimization module, which is used to perform optimization processing on the generated three-dimensional model. The optimization processing includes model simplification, detail enhancement or lighting adjustment.

Images