Flexible three-dimensional point cloud splicing method based on structured light projection

Abstract

The invention relates to a flexible three-dimensional point cloud splicing method based on structured light projection, and the method comprises the steps: setting the initial position and parameters of equipment, and adjusting the focal length and exposure time of a camera; selecting a grating pattern, and adjusting the brightness and the projection angle of the projector; a multi-view collection path is planned, and equipment clock synchronization is ensured; performing preliminary testing, and optimizing equipment setting to ensure an optimal acquisition effect; moving the equipment according to a preset path, gradually collecting scene data, and recording the position and posture of the equipment; transmitting and storing the acquired data in real time; generating an initial point cloud and carrying out noise filtering and sparse processing; point cloud registration is carried out by adopting a beam adjustment method, and errors between visual angles are eliminated; and fusing the multi-view point clouds to generate a complete three-dimensional model, and performing quality inspection on the three-dimensional model. The three-dimensional point cloud splicing method provided by the invention is efficient and flexible, is particularly suitable for high-precision three-dimensional data acquisition and modeling of a large complex scene, and has a remarkable precision advantage and operation convenience.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional reconstruction, and relates to a flexible three-dimensional point cloud stitching method based on structured light projection. Background Art

[0002] Three-dimensional reconstruction technology has been widely used in recent years, from industrial inspection to cultural heritage protection, all relying on high-precision three-dimensional models. However, existing three-dimensional reconstruction technologies have some limitations. Especially when dealing with large-sized objects or complex scenes, they face challenges of insufficient accuracy and poor flexibility.

[0003] Traditional three-dimensional reconstruction methods mainly include technologies such as laser scanning, stereo vision, and structured light projection. Among them, laser scanners are widely used due to their high precision and long-distance measurement capabilities. However, their equipment cost is high, operation is complex, and efficiency is low when dealing with large areas and multi-level structures. Stereo vision technology uses binocular or multi-camera systems to calculate depth information through parallax. Although the cost is low and the adaptability is good, the accuracy depends on the baseline distance of the camera and the image resolution, and it performs poorly on complex surfaces and low-texture areas.

[0004] As an active optical measurement method, structured light projection technology projects a grating pattern with a known structure onto the object surface and combines the camera to capture the deformed image of the pattern, enabling the calculation of high-precision three-dimensional point cloud data. Compared with other methods, structured light projection has significant advantages in terms of data acquisition speed and accuracy. However, traditional structured light projection systems are often fixed in one position and cannot flexibly cover large-scale scenes, which becomes a bottleneck when scanning large-sized objects. In addition, in the process of multi-viewpoint cloud stitching, how to effectively eliminate the cumulative error and ensure global consistency is also an urgent problem to be solved. Summary of the Invention

[0005] The technical problem solved by the present invention is: overcoming the deficiencies of the prior art, and proposing a flexible three-dimensional point cloud stitching method based on structured light projection, using bundle adjustment method for high-precision registration of point cloud data, and at the same time realizing flexible movement of the projector through a three-axis motion mechanism to solve the three-dimensional reconstruction problem of large-sized objects or complex scenes.

[0006] The technical solution of the present invention is: a flexible three-dimensional point cloud stitching method based on structured light projection, including the following steps:

[0007] Install a structured light projector and a camera, set the initial position and movement range of the three-axis motion mechanism, and adjust the focal length and exposure time of the camera;

[0008] Select the grating pattern to be projected onto the object surface, adjust the frequency and contrast of the projection pattern, set the brightness and projection angle of the structured light projector, and ensure the visibility of the projection pattern in the whole scene;

[0009] Preset multi-view acquisition paths, plan the moving trajectory of the projector, and ensure the clock synchronization between the projector and the camera;

[0010] According to the preset acquisition path, control the three-axis motion mechanism to gradually move the position of the projector. At each predetermined position, the projector projects a grating pattern onto the surface of the object, and the camera synchronously captures the deformation of the pattern, and records the position and attitude information of the projector and the camera during each acquisition;

[0011] Use a decoding algorithm to process the image data captured by the camera, generate the preliminary point cloud for each view, and preprocess the generated point cloud data;

[0012] Use bundle adjustment to register the multi-view point clouds, align the point clouds of each view to the same coordinate system, calculate the initial poses of the point clouds of each view and adjust the pose parameters, and iteratively reduce the cumulative error of the registration of the point clouds of each view;

[0013] Perform fusion processing on the registered multi-view point clouds, eliminate redundant points and errors, and then generate a complete three-dimensional model through a surface reconstruction algorithm to complete the three-dimensional point cloud stitching.

[0014] Furthermore, the number of projectors is not less than 2, and the number of cameras is not less than 6.

[0015] Furthermore, the grating pattern adopts a sine wave or cosine wave pattern. The grating pattern encodes three-dimensional information through the projection of multiple frames of different patterns, and uses the brightness or phase change in different regions of a single pattern to encode depth information; the period of the grating pattern is the distance between two adjacent regions with the same brightness, and the phase change of the grating sine wave or cosine wave pattern is carried out according to the period during projection;

[0016] Adopt multi-frequency separation technology, project grating patterns with different frequencies in sequence to achieve a larger measurement range and higher accuracy. The brightness and period changes of each frame of the pattern are captured by the camera and used for subsequent three-dimensional coordinate calculation.

[0017] Furthermore, multiple projectors are divided into two groups to project grating patterns crosswise.

[0018] Furthermore, before image acquisition according to the preset acquisition path, it also includes the step of preliminary image acquisition test: start the projector and the camera, check the quality of the acquired images, and judge whether there is overexposure or underexposure; according to the test results, adjust the brightness of the projector, the exposure settings and focal length of the camera to ensure the best acquisition effect.

[0019] Furthermore, the preprocessing of the generated point cloud data includes noise filtering and point cloud sparsification.

[0020] Further, the registration of multi-view point clouds using bundle adjustment is specifically as follows:

[0021] Taking the coordinate system of one of the cameras as a reference, align the point clouds of each view to the same coordinate system;

[0022] Extract features from the three-dimensional point cloud data collected from different views, including normal vectors and surface curvatures;

[0023] Based on the extracted features, calculate the initial poses of the point clouds of each view using a feature-matching alignment method;

[0024] Adopt an iterative optimization algorithm, and gradually adjust the pose parameters between the point clouds of each view, including the rotation matrix and translation vector, by minimizing the distance or other similarity measures between the point clouds until the convergence condition is reached;

[0025] Output the three-dimensional point cloud in the same coordinate system after registration.

[0026] Further, the iterative optimization method includes the following steps:

[0027] 810. Set the initial values of the internal and external parameters of the camera and the initial values of the three-dimensional point coordinates projected by the projector onto the object surface;

[0028] 820. Calculate the projection error of all three-dimensional points in each camera image; the projection error refers to the difference between the true projection position of the three-dimensional point on the camera image and the predicted or estimated projection position. For a three-dimensional point P and a camera, the true projection position is (u, v), and the predicted projection position obtained through three-dimensional reconstruction is (\(\hat{u},\hat{v}\)), then the projection error is defined as the Euclidean distance between these two two-dimensional projection points:

[0029] e = \(\sqrt{(u - \hat{u})^2}\) 2 + \(\sqrt{(v - \hat{v})^2}\) 2

[0030] 830. Construct a global optimization problem with the goal of minimizing the sum of the projection errors in all views, and use the non-linear least squares method to solve the global optimization problem;

[0031] 840. Repeat steps 810 - 830, iteratively update the internal and external parameters of the camera and the three-dimensional point coordinates, and gradually reduce the projection error until the sum of the projection errors converges to a local optimal solution;

[0032] 850. Obtain the internal and external parameters of the camera and the three-dimensional point coordinates Pj corresponding to the local optimal solution, as well as the rotation matrix Ri and translation vector Ti after iterative optimization, and use the formula Pj″ = Ri·Pj + Ti to obtain the final point Pj″ in the global coordinate system, realizing the registration of multi-view point clouds.

[0033] Further, the surface reconstruction algorithm includes any one of Poisson surface reconstruction (PSR) or Moving Least Squares (MLS).

[0034] Further, an optical mask or the power of the projector is adjusted to optimize the projection effect.

[0035] The beneficial effects of the present invention compared with the prior art are as follows:

[0036] (1) The present invention performs point cloud registration by bundle adjustment method, significantly improving the stitching accuracy of multi-view point cloud data. Especially when dealing with large-sized objects, it can effectively reduce the cumulative error generated during the stitching of large-sized object point clouds, ensuring the high precision of the final model.

[0037] (2) By using a three-axis motion mechanism, the present invention can not only adjust the position of the projector, but also adapt to objects with different shapes and surface characteristics by changing the projection angle and focal length, improving the comprehensiveness of scanning and the ability to capture details. Moreover, the three-axis motion mechanism provides flexible motion control, adapting to different scene requirements, and can achieve large-range scanning coverage without affecting the measurement accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flowchart of a flexible three-dimensional point cloud stitching method based on structured light projection according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0039] The present invention will be further described below in conjunction with embodiments.

[0040] Embodiment 1

[0041] As Figure 1 shown, a flexible three-dimensional point cloud stitching method based on structured light projection proposed in this embodiment includes the following steps:

[0042] S1. Scene preparation and initial settings:

[0043] According to the size and shape of the scene, select a suitable position to install the structured light projector and the camera, determine the initial position of the three-axis motion mechanism, and set the motion range of the three-axis motion mechanism to ensure that the entire target area can be covered. Adjust the focal length and exposure time of the camera to optimize the image quality. Among them, the three-axis motion mechanism is used to move the position of the projector, and the installation position of the camera is fixed.

[0044] S2. Projection parameter settings:

[0045] Select the grating pattern to be projected onto the object surface, and adjust the frequency and contrast of the projection pattern according to the scene characteristics. Set the brightness and projection angle of the projector to ensure the visibility of the projection pattern throughout the scene.

[0046] The projector projects a sine / cosine wave pattern, which has a smooth brightness change and is suitable for high-precision measurement. To ensure the uniqueness of each pixel, encoding techniques are usually adopted for the grating pattern. Three-dimensional information is encoded by projecting multiple frames of different patterns. Depth information is encoded by using the brightness or phase change in different regions of a single pattern. The period of the grating pattern refers to the distance between two adjacent regions with the same brightness. During projection, the phase of the grating sine / cosine wave pattern changes according to the period. At the same time, multi-frequency separation technology is adopted, and grating patterns with different frequencies (i.e., different bar spacings) are projected in sequence to achieve a larger measurement range and higher precision. The brightness and period changes of each frame of the pattern are captured by the camera and used for subsequent three-dimensional coordinate calculation.

[0047] S3. Preparation for multi-view data acquisition:

[0048] Preset multi-view acquisition paths, plan the moving trajectory of the projector, and ensure the clock synchronization between the projector and the camera so that the images of each view can be accurately matched during subsequent data processing.

[0049] S4. Preliminary acquisition and adjustment:

[0050] Start the projector and the camera, and conduct preliminary image acquisition tests: Check the quality of the acquired images, especially pay attention to whether there is overexposure or underexposure. According to the test results, finely adjust the brightness of the projector, the exposure settings and the focal length of the camera to ensure the best acquisition effect.

[0051] S5. Formal data acquisition:

[0052] According to the preset acquisition path, control the three-axis motion mechanism to gradually move the position of the projector. At each predetermined position, the projector projects a grating pattern onto the object surface, and the camera synchronously captures the deformation of the pattern, recording the position and attitude information of the projector and the camera during each acquisition.

[0053] In the present invention, to flexibly cover a large area scene, the number of projectors is not less than 2, and the number of cameras is not less than 6. In this embodiment, 6 projectors and 6 cameras are taken as an example. The working principle of the projection process is described as follows:

[0054] The projection system globally captures the object surface through 6 cameras. At the same time, the 6 projectors are divided into two groups (1, 3, 5 and 2, 4, 6) to cross-project structured light stripes, ensuring that the stripes cover all regions of the object from different perspectives. Subsequently, the three-axis motion mechanism is activated, enabling each projector to adjust its position, so that the projected structured light stripes completely wrap and adapt to the complex shape of the object surface. This dynamic adjustment process ensures the uniform distribution of the structured light on the entire object surface, providing complete and accurate data for subsequent three-dimensional point cloud capture and stitching.

[0055] S6. Data Transmission and Storage:

[0056] Transmit the collected image data in real time to the control system and perform preliminary storage. Ensure the integrity and security of the data, and adopt redundant storage and backup mechanisms to prevent data loss.

[0057] S7. Point Cloud Generation and Preprocessing:

[0058] Use a decoding algorithm to process the image data collected by the camera, generate a preliminary point cloud for each perspective, and preprocess the generated point cloud data, including noise filtering and point cloud sparsification, to improve the data quality.

[0059] S8. Multi - perspective Point Cloud Registration:

[0060] Use bundle adjustment to perform multi - perspective point cloud registration, align the point clouds of each perspective to the same coordinate system, calculate the initial pose of the point clouds of each perspective and adjust the pose parameters, and iteratively reduce the cumulative error of the multi - perspective point cloud registration.

[0061] The coordinate system is defined as follows:

[0062] This coordinate system refers to the global three - dimensional space coordinate framework jointly referred to by all point cloud data after registration by bundle adjustment, usually based on the coordinate system of one of the sensors (such as Camera 1).

[0063] The use of bundle adjustment for multi - perspective point cloud registration is specifically as follows:

[0064] First, align the point clouds of each perspective to the same coordinate system, and extract features (such as normal vectors, surface curvatures, etc.) from the three - dimensional point cloud data collected from different perspectives; second, calculate the initial pose of the point clouds of each perspective (such as rough alignment based on feature matching) according to the extracted features to provide a starting point for subsequent optimization; then, adopt an iterative optimization method (such as an improved ICP algorithm), by minimizing the distance or other similarity measures between point clouds, gradually adjust the pose parameters (such as rotation matrix and translation vector) between the point clouds of each perspective until the convergence condition is reached; finally, output the three - dimensional point cloud in the same coordinate system after registration. This method is efficient and robust, and is especially suitable for scenes with large - scale or non - rigid transformations.

[0065] Among them, the iterative optimization method specifically includes the following steps:

[0066] 810. Set the initial values of the internal and external camera parameters and the initial values of the three - dimensional point coordinates projected by the projector onto the object surface. Among them, the initial values of the internal and external camera parameters are obtained through pose estimation by camera system calibration.

[0067] 820. Calculate the projection error of all three - dimensional points in each camera image.

[0068] The projection error refers to the difference between the true projection position of a 3D point on the camera image and the predicted or estimated projection position. Specifically, for a 3D point P and a camera, its true projection position is (u, v), while the predicted projection position obtained through 3D reconstruction is (\(\hat{u}\), \(\hat{v}\)), and the projection error is usually defined as the Euclidean distance between these two 2D projection points:

[0069] e = \(\sqrt{(u - \hat{u})^2}\) 2 + \(\sqrt{(v - \hat{v})^2}\) 2

[0070] 830. Construct a global optimization problem with the goal of minimizing the sum of projection errors in all viewpoints, and use the non - linear least - squares method to solve the global optimization problem.

[0071] 840. Repeat steps 810 - 830, iteratively update the internal and external camera parameters and the 3D point coordinates, gradually reduce the projection error until the sum of projection errors converges to a local optimal solution.

[0072] 850. Obtain the internal and external camera parameters and the 3D point coordinates \(P_j\) corresponding to the local optimal solution, as well as the rotation matrix \(R_i\) and translation vector \(T_i\) after iterative optimization. Use the formula \(P_j'' = R_i\cdot P_j+T_i\) to obtain the final points \(P_j''\) in the global coordinate system, and achieve the registration of multi - view point clouds.

[0073] S9. Point cloud fusion and reconstruction:

[0074] Fuse the registered multi - view point clouds, eliminate redundant points and errors, and then generate a complete 3D model through a surface reconstruction algorithm to ensure the smoothness and detail accuracy of the 3D model surface, and complete the 3D point cloud stitching.

[0075] Among them, the surface reconstruction algorithm can adopt Poisson surface reconstruction (PSR) or moving least squares (MLS).

[0076] S10. Data verification and quality inspection:

[0077] Conduct quality inspection on the generated 3D model to verify the accuracy and integrity of the model. If problems are found, re - analyze the data of relevant steps and make necessary corrections and supplementary acquisitions.

[0078] In the specific implementation, the following matters need to be particularly noted:

[0079] Lighting conditions: Since structured light projection is sensitive to lighting conditions, strong ambient light interference should be avoided as much as possible during the acquisition process. A light - proof cover can be used or the projector power can be adjusted to optimize the projection effect.

[0080] Motion precision control: The precision of the three-axis motion mechanism directly affects the registration effect of point cloud data. Therefore, during the motion process, the stability and accuracy of the motion mechanism should be ensured to avoid vibration and external interference.

[0081] Data processing efficiency: Due to the large amount of data in large-scale scenes, efficient data processing algorithms and hardware support are required during point cloud generation and registration to accelerate the calculation process.

[0082] In summary, the flexible three-dimensional point cloud stitching method of the present invention has the following advantages:

[0083] 1. Wide coverage:

[0084] When a traditional fixed-position structured light projection system scans a large-area object, it is necessary to manually adjust the device position multiple times, which is cumbersome and prone to data discontinuity problems. By introducing a three-axis motion mechanism, the present invention realizes the automatic adjustment of the projector and camera, and can continuously scan the surface of large-sized objects without frequent repositioning.

[0085] 2. High-precision stitching:

[0086] In large-scale scenes, the stitching of multi-view point clouds usually faces the challenge of cumulative errors. Although the traditional ICP (Iterative Closest Point) algorithm can perform point cloud registration, its effect in eliminating cumulative errors is limited. The present invention adopts the bundle adjustment method to reduce the cumulative errors generated during the point cloud stitching process through global optimization, ensuring the high precision of the final model.

[0087] 3. Flexible adaptability:

[0088] Due to the complexity and diversity of large-sized objects, the system needs to have a high degree of adaptability. The three-axis motion mechanism of the present invention can not only adjust the positions of the projector and camera, but also adapt to objects with different shapes and surface characteristics by changing the projection angle and focal length, improving the comprehensiveness of scanning and the ability to capture details.

[0089] Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solution of the present invention using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and decorations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention all belong to the protection scope of the technical solution of the present invention.

[0090] The content not detailedly described in the specification of the present invention belongs to the well-known technology of those skilled in the art.

Claims

1. A flexible three-dimensional point cloud stitching method based on structured light projection, characterized in that, Including the following steps: Install a structured light projector and a camera, set the initial position and movement range of the three-axis motion mechanism, and adjust the focal length and exposure time of the camera; Select a grating pattern for projection onto the object surface, adjust the frequency and contrast of the projection pattern, set the brightness and projection angle of the structured light projector, and ensure the visibility of the projection pattern throughout the scene; Preset multi-view acquisition paths, plan the movement trajectory of the projector, and ensure the clock synchronization between the projector and the camera; According to the preset acquisition path, control the three-axis motion mechanism to gradually move the position of the projector. At each predetermined position, the projector projects the grating pattern onto the object surface, and the camera synchronously captures the deformation of the pattern, recording the position and pose information of the projector and the camera during each acquisition; Use a decoding algorithm to process the image data collected by the camera, generate a preliminary point cloud for each view, and preprocess the generated point cloud data; Use bundle adjustment to register the multi-view point clouds, align the point clouds of each view to the same coordinate system, calculate the initial pose of the point clouds of each view, and adjust the pose parameters, iteratively reducing the cumulative error of the registration of the point clouds of each view; Perform fusion processing on the registered multi-view point clouds, eliminate redundant points and errors, and then generate a complete three-dimensional model through a surface reconstruction algorithm to complete the three-dimensional point cloud stitching.

2. A flexible three-dimensional point cloud stitching method based on structured light projection according to claim 1, characterized in that The number of projectors is not less than 2, and the number of cameras is not less than 6.

3. A flexible three-dimensional point cloud stitching method based on structured light projection according to claim 2, characterized in that The grating pattern uses a sine wave or cosine wave pattern. The grating pattern encodes three-dimensional information through the projection of multiple frames of different patterns, and uses the brightness or phase change in different regions of a single pattern to encode depth information; the period of the grating pattern is the distance between two adjacent regions with the same brightness, and the phase change of the grating sine wave or cosine wave pattern is carried out according to the period during projection; Adopt multi-frequency separation technology, project grating patterns with different frequencies in sequence to achieve a larger measurement range and higher accuracy. The brightness and period changes of each frame of the pattern are captured by the camera and used for subsequent three-dimensional coordinate calculation.

4. A flexible three-dimensional point cloud stitching method based on structured light projection according to claim 2, characterized in that Multiple projectors are divided into two groups to project grating patterns crosswise.

5. A flexible three-dimensional point cloud stitching method based on structured light projection according to claim 1, characterized in that, Before image acquisition according to the preset acquisition path, it also includes the step of preliminary image acquisition test: start the projector and the camera, check the quality of the acquired images, and judge whether there is overexposure or underexposure; According to the test results, adjust the brightness of the projector, the exposure settings and focal length of the camera to ensure the best acquisition effect.

6. A flexible three-dimensional point cloud stitching method based on structured light projection according to claim 1, characterized in that, The preprocessing of the generated point cloud data includes noise filtering and point cloud sparsification.

7. A flexible three-dimensional point cloud stitching method based on structured light projection according to claim 1, characterized in that, The use of bundle adjustment for multi-view point cloud registration is specifically as follows: Taking the coordinate system of one of the cameras as a reference, align the point clouds of each view to the same coordinate system; Extract features from the three-dimensional point cloud data collected from different views, including normal vectors and surface curvatures; Based on the extracted features, calculate the initial pose of the point clouds of each view using a feature-matching alignment method; Adopt an iterative optimization algorithm, and gradually adjust the pose parameters between the point clouds of each view, including the rotation matrix and translation vector, by minimizing the distance or other similarity measures between the point clouds until the convergence condition is reached; Output the three-dimensional point cloud in the same coordinate system after registration.

8. A flexible three-dimensional point cloud stitching method based on structured light projection according to claim 7, characterized in that The iterative optimization method includes the following steps:

810. Set the initial values of the internal and external parameters of the camera and the initial values of the three-dimensional point coordinates projected by the projector onto the object surface; 820. Calculate the projection error of all three-dimensional points in each camera image; the projection error refers to the difference between the true projection position and the predicted or estimated projection position of the three-dimensional point in the camera image. For a three-dimensional point P and a camera, the true projection position is (u, v), and the predicted projection position obtained through three-dimensional reconstruction is (û, v̂), then the projection error is defined as the Euclidean distance between these two two-dimensional projection points: e = (u - û) 2 + (v - v̂) 2 830. Construct a global optimization problem with the goal of minimizing the sum of the projection errors from all perspectives, and use the non-linear least squares method to solve the global optimization problem; 840. Repeat steps 810 - 830, iteratively update the internal and external parameters of the camera and the three-dimensional point coordinates, gradually reduce the projection error until the sum of the projection errors converges to a local optimal solution; 850. Obtain the internal and external parameters of the camera and the three-dimensional point coordinates Pj corresponding to the local optimal solution, as well as the rotation matrix Ri and translation vector Ti after iterative optimization. Use the formula Pj″ = Ri · Pj + Ti to obtain the final points Pj″ in the global coordinate system, and achieve the registration of multi-viewpoint clouds.

9. A flexible three-dimensional point cloud stitching method based on structured light projection according to claim 1, characterized in that The surface reconstruction algorithm includes any one of Poisson surface reconstruction PSR or local fitting smooth surface MLS.

10. A flexible three-dimensional point cloud stitching method based on structured light projection according to claim 1, characterized in that Optimize the projection effect by using a light shield or adjusting the power of the projector.

Images