Three-dimensional reconstruction method and system based on monocular camera and line laser

Through a three-dimensional reconstruction method based on a monocular camera and a line laser, combined with surface reconstruction algorithm and automated control, the problems of three-dimensional reconstruction accuracy and cost in the existing technology are solved, and fast, stable and high-precision three-dimensional reconstruction is achieved, suitable for multiple application areas.

CN118918265APending Publication Date: 2024-11-08WUHAN UNIV +1

Patent Information

Application Number
CN202411152360.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In the existing three-dimensional reconstruction technology, monocular vision methods are difficult to accurately estimate depth, binocular vision methods are difficult to accurately match camera pictures, and the equipment cost of active laser scanning is high, and there is a lack of high-precision and low-cost three-dimensional reconstruction solutions.

Method used

Three-dimensional reconstruction method based on monocular cameras and line lasers is adopted, and three-dimensional reconstruction is achieved through camera calibration, light plane calibration, laser bar center extraction and point cloud registration, combined with surface reconstruction algorithm. The system includes a camera, line laser, calibration checkerboard, guide rail, rotary turntable and main control components, and uses a Raspberry Pi for automated control to reduce hardware costs.

Benefits of technology

It realizes fast, stable and high-precision three-dimensional reconstruction in complex environments, reduces hardware costs, is cost-effective, is suitable for three-dimensional scanning of small objects in multiple angles, and is highly versatile. It is suitable for scientific research, education, and industrial manufacturing and other fields.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918265A_ABST
    Figure CN118918265A_ABST
Patent Text Reader

Abstract

The invention discloses a three-dimensional reconstruction method and system based on a monocular camera and a line laser. The method comprises the following steps: calibrating the camera to obtain camera parameters; light plane calibration is carried out, and a light plane equation is obtained by combining camera parameters; the method comprises the following steps: scanning a to-be-reconstructed target by adopting a line laser, shooting by adopting a camera to obtain a plurality of laser images, obtaining pixel coordinates of a plurality of laser strip centers from the plurality of laser images, and obtaining three-dimensional point clouds of the plurality of laser strip centers based on a light plane equation to form a three-dimensional point cloud image under a current visual angle; rotating the to-be-reconstructed target, performing scanning at different viewing angles by using the line laser to obtain three-dimensional point cloud pictures of the to-be-reconstructed target under different viewing angles, and performing point cloud registration to obtain a complete three-dimensional point cloud picture of the to-be-reconstructed target; and performing curved surface reconstruction on the to-be-reconstructed target based on the complete three-dimensional point cloud picture of the to-be-reconstructed target. According to the invention, rapid, stable, high-precision and intelligent three-dimensional reconstruction can be realized in various complex environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and in particular to a three-dimensional reconstruction method and system based on a monocular camera and a line laser. Background Art

[0002] 3D reconstruction technology is a technology that uses specific devices and methods to accurately measure the 3D information of the measured object or scene, and uses this information to construct the 3D contour of the object, and then restore the 3D structure of its surface. This technology occupies a pivotal position in the field of computer vision, and its application fields are wide and far-reaching, playing an indispensable role in industrial product design, medical testing, cultural relics protection and other fields.

[0003] The methods of 3D reconstruction can be roughly divided into two categories: passive and active. The passive monocular vision method uses only a single camera as the acquisition device, which has the advantages of low cost and easy deployment, but it has an inherent problem, that is, a single image may correspond to countless real physical world scenes, so it is difficult to estimate the depth from the image and then realize 3D reconstruction. The passive binocular vision method uses the two corrected images obtained by the left and right cameras to find the matching points of the left and right images, and then restore the 3D information of the environment according to the geometric principle. However, the difficulty of this method lies in the matching of the left and right camera images. Inaccurate matching will affect the imaging effect of the final algorithm. The active method has a laser scanning method, that is, using a laser radar to scan the object and obtain the 3D information of the object. The laser radar calculates the distance by emitting a laser beam and measuring the time it takes to reflect back, thereby constructing a 3D model of the object. However, the equipment cost used in this method is relatively high. Therefore, it is necessary to provide a high-precision, low-cost 3D reconstruction solution. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the first object of the present invention is to provide a 3D reconstruction method based on a monocular camera and a line laser, which can more economically achieve fast, stable, high-precision and intelligent 3D reconstruction in various complex environments.

[0005] A second object of the present invention is to provide a three-dimensional reconstruction system based on a monocular camera and a line laser.

[0006] To achieve the above object, the present invention is implemented through the following technical solutions:

[0007] A three-dimensional reconstruction method based on a monocular camera and a line laser, comprising:

[0008] Using a camera to shoot a calibration chessboard to obtain a calibration chessboard image, and calibrating the camera based on the calibration chessboard image to obtain camera parameters;

[0009] Calibrate the light plane by using a camera, a line laser and a calibration chessboard, and obtain a light plane equation in combination with the camera parameters;

[0010] The line laser is used to scan the target to be reconstructed, and the camera is used to shoot to obtain a plurality of laser images, pixel coordinates of the centers of a plurality of laser light strips are obtained from the plurality of laser images, and a three-dimensional point cloud of the centers of the plurality of laser light strips is obtained based on the light plane equation to form a three-dimensional point cloud image of the scanned target to be reconstructed at the current viewing angle;

[0011] Rotate the target to be reconstructed, use the line laser to scan at different viewing angles to obtain a three-dimensional point cloud image of the target to be reconstructed at different viewing angles, and perform point cloud registration to obtain a complete three-dimensional point cloud image of the target to be reconstructed;

[0012] Based on the complete three-dimensional point cloud image of the target to be reconstructed, a surface reconstruction algorithm is used to perform surface reconstruction on the target to be reconstructed, so as to achieve three-dimensional reconstruction of the target to be reconstructed.

[0013] Preferably, a calibration chessboard is photographed with a camera to obtain a calibration chessboard image, and the camera is calibrated based on the calibration chessboard image to obtain camera parameters, including:

[0014] The camera is used to photograph the calibration chessboard in different positions and postures to obtain images of the calibration chessboard in different positions and postures;

[0015] Corner point detection is performed on each chessboard image, and each corner point detected in each chessboard image is further detected to obtain pixel coordinates of each sub-pixel corner point. Camera calibration is performed based on the pixel coordinates of each sub-pixel corner point to obtain the camera parameters, which include camera intrinsic parameters, camera extrinsic parameters and distortion parameters.

[0016] Preferably, the light plane is calibrated by using a camera, a line laser and a calibration chessboard, and the light plane equation is obtained in combination with the camera parameters, including:

[0017] Processing the calibrated chessboard image to obtain a chessboard coordinate system;

[0018] The calibration chessboard is illuminated by the line laser, and a laser image is obtained by photographing with the camera, the laser image is preprocessed to obtain a laser light stripe area, the center of the laser light stripe is extracted from the laser light stripe area by an improved grayscale centroid method, and the spatial coordinates of the center of the laser light stripe in the chessboard coordinate system are obtained;

[0019] The spatial coordinates of the center of the laser light strip in the chessboard coordinate system are converted by using the camera parameters to obtain the three-dimensional coordinates of the center of the laser light strip in the world coordinate system;

[0020] Adjusting the positions of the camera and the line laser, irradiating and photographing the calibration chessboard at different positions, so as to obtain the centers of the multiple laser light strips corresponding to the different positions, and obtaining the three-dimensional coordinates of the centers of the multiple laser light strips corresponding to the different positions in the world coordinate system;

[0021] The light plane equation is obtained by fitting the three-dimensional coordinates of the centers of the multiple laser light strips in the world coordinate system.

[0022] Preferably, the preprocessing step includes distortion correction, Gaussian filtering and threshold segmentation.

[0023] Preferably, the distortion correction step includes:

[0024] The distortion correction function is called, the laser image, the camera internal parameters and the distortion parameters are input, and the distortion-corrected image is obtained by processing with the distortion correction function.

[0025] Preferably, the Gaussian filtering step includes:

[0026] The Gaussian smoothing function is called, and the distortion-corrected image, the Gaussian convolution kernel size and the standard deviation value are input, and the laser image after Gaussian filtering is obtained by processing with the Gaussian smoothing function.

[0027] Preferably, the step of threshold segmentation includes:

[0028] Set a grayscale threshold;

[0029] In the laser image after Gaussian filtering, the pixel points whose grayscale values ​​are greater than the grayscale threshold are set as masks, and the grayscale values ​​of the pixel points outside the mask are set to zero, so as to extract the laser light strip area.

[0030] Preferably, the laser light stripe center is extracted from the laser light stripe region using an improved grayscale centroid method, comprising:

[0031] Scanning the first row of the laser image after threshold segmentation to obtain the grayscale maximum value of the pixels in the first row, and multiplying the grayscale maximum value by a preset coefficient to obtain a preset threshold value;

[0032] Obtain a light spot area from the first row, and if the grayscale value of the light spot area is greater than the preset threshold, extract the centroid of the light spot area, traverse all the light spot areas in the first row to determine all the light spot areas whose grayscale values ​​are greater than the preset threshold, and extract the centroid of each light spot area;

[0033] Select the maximum center of gravity from multiple center of gravity values;

[0034] Scan each row in turn to obtain the maximum center of gravity of each row, and fit the pixel points at the position of the maximum center of gravity of each row to obtain the center of the laser light strip.

[0035] Preferably, point cloud registration to obtain a complete three-dimensional point cloud image of the target to be reconstructed includes:

[0036] Select at least four feature points from two 3D point cloud images at different viewing angles for point cloud registration;

[0037] A pair of point clouds in two three-dimensional point cloud images under different viewing angles are selected, and a transformation matrix that minimizes the distance between the pair of point clouds is iteratively obtained. The point cloud in a three-dimensional point cloud image is transformed by the transformation matrix to achieve precise point cloud registration.

[0038] To achieve the above-mentioned object, the second aspect of the present invention provides a three-dimensional reconstruction system based on a monocular camera and a line laser, which is applied to the three-dimensional reconstruction method based on a monocular camera and a line laser described above, wherein the system includes a camera, a line laser, a calibration chessboard, a guide rail, a rotating turntable and a main control component;

[0039] The camera and the line laser are arranged on the guide rail, and the camera, the line laser and the guide rail are connected to the main control assembly;

[0040] The rotating turntable is used to place the target to be reconstructed;

[0041] The calibration chessboard is used to implement camera calibration and light plane calibration;

[0042] The main control component includes a Raspberry Pi component and a terminal computer. The Raspberry Pi component is used to drive the guide rail so that the line laser and the camera can illuminate and photograph the calibration chessboard or the target to be reconstructed at different positions, and can be used to obtain a three-dimensional point cloud image of the target to be reconstructed after scanning at different viewing angles; the terminal computer is used to realize point cloud registration and surface reconstruction of the target to be reconstructed.

[0043] The present invention has at least the following technical effects:

[0044] 1. The present invention provides a 3D reconstruction solution based on a monocular camera and a line laser. This solution not only has the advantages of a traditional laser triangulation system, but also achieves high-precision system construction and calibration without initial spatial position requirements through a chessboard calibration method. The positions of the camera, line laser and measurement platform do not need to be accurately determined in advance, so the system has a high degree of freedom in construction.

[0045] 2. The present invention uses a mobile camera and a line laser to perform three-dimensional reconstruction, which can be beneficial to protecting the object to be reconstructed. In addition, due to the use of high-precision camera calibration and high-performance laser light strip center extraction algorithm, the linear scanning of the present invention has high accuracy and meets the needs of multi-angle three-dimensional reconstruction of small objects at medium and short distances. In addition, the basic measurement principle of the present invention can be applied to a variety of measurement modes and measurement ranges. For example, when indoor three-dimensional scanning is required, it is only necessary to adjust the line laser and the lens, and install the line laser and the camera on a rotating platform for rotational measurement. The measurement method is the same as the method of this application, and then the point cloud can be restored in angle. Therefore, the present invention also has strong versatility.

[0046] 3. The improved grayscale centroid method proposed in the present invention is implemented on the Raspberry Pi. The algorithm has good robustness, high extraction accuracy for uneven light spots, high accuracy in measuring the contour of objects, and low logic resource occupancy rate of the Raspberry Pi, which improves operating efficiency and can meet industrial needs.

[0047] 4. As a low-cost single-board computer, Raspberry Pi not only greatly reduces the hardware cost but also is easy to integrate compared to expensive professional 3D scanners. At the same time, the combination of monocular camera and line laser is relatively economical, making the whole system have a high cost performance. The present invention uses Raspberry Pi to control the movement of the camera and line laser, realizes the automatic operation of the control system, and has the ability to achieve fast, stable and intelligent 3D reconstruction in various complex environments. The system has a high degree of automation, precise scanning control, and can scan and reconstruct objects from multiple angles. In addition, this solution has the advantages of high cost-effectiveness, good flexibility, strong scalability and outstanding innovation. These advantages make the solution have broad application prospects in scientific research, education, industrial manufacturing and other fields.

[0048] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 The present invention is a flowchart of a three-dimensional reconstruction method based on a monocular camera and a line laser according to an embodiment of the present invention.

[0050] Figure 2 This is a flow chart of an improved grayscale centroid method according to an embodiment of the present invention.

[0051] Figure 3 FIG. 4 is a schematic diagram of the center of a laser light strip according to an embodiment of the present invention.

[0052] Figure 4 4 is a light plane point cloud image before fitting according to an embodiment of the present invention.

[0053] Figure 5This is a fitted light plane point cloud image according to an embodiment of the present invention.

[0054] Figure 6 Schematic diagram of a line laser triangulation model according to an embodiment of the present invention.

[0055] Figure 7 3D point cloud images of the target to be reconstructed at different viewing angles according to an embodiment of the present invention.

[0056] Figure 8 This is a complete three-dimensional point cloud image of the target to be reconstructed according to an embodiment of the present invention.

[0057] Fig. 9 This is a target model diagram to be reconstructed according to an embodiment of the present invention.

[0058] Fig.10 This is a schematic diagram of a target surface to be reconstructed after reconstruction according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The present embodiment is described in detail below, and examples of the embodiment are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0060] The following describes the three-dimensional reconstruction method and system based on a monocular camera and a line laser of this embodiment with reference to the accompanying drawings.

[0061] Figure 1 FIG. 1 is a flow chart of a three-dimensional reconstruction method based on a monocular camera and a line laser according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0062] Step S1: photographing a calibration chessboard with a camera to obtain a calibration chessboard image, and calibrating the camera based on the calibration chessboard image to obtain camera parameters.

[0063] In one embodiment of the present invention, a calibration chessboard is photographed by a camera to obtain a calibration chessboard image, and a camera is calibrated based on the calibration chessboard image to obtain camera parameters, including:

[0064] A camera is used to shoot a calibration chessboard in different poses to obtain calibration chessboard images in different poses; corner point detection is performed on each chessboard image, and each corner point detected in each chessboard image is further detected to obtain the pixel coordinates of each sub-pixel corner point; camera calibration is performed based on the pixel coordinates of each sub-pixel corner point to obtain camera parameters, which include camera intrinsic parameters, camera extrinsic parameters and distortion parameters.

[0065] In this embodiment, a calibration plate image, i.e., a calibration chessboard image, may be captured first. The calibration chessboard image is a black and white grid image. Specifically, the calibration chessboard is placed in front of the camera, and the calibration chessboard is moved to different positions. During the acquisition process, the aperture and focal length are kept unchanged, and the calibration chessboard is made to occupy as much as possible of the field of view in the image. Multiple images are captured at different angles and orientations, and the entire calibration chessboard area is required to be covered. At the same time, the calibration chessboard is ensured to be as large as possible within the camera field of view.

[0066] Furthermore, the corner points are detected. The corner points (the intersection positions of the black and white squares on the chessboard) of the multiple calibration chessboard images obtained are detected, and then the sub-pixel corner point information of each detected corner point is further extracted to calculate its pixel coordinates in the image coordinate system. Then, the corner point information of the multiple calibration chessboard images taken is calibrated, the internal parameters, parallel vectors, rotation vectors, etc. of the camera are calculated, and the calibration results are saved for subsequent image preprocessing and light plane calibration.

[0067] Specifically, take multiple sets of calibration chessboard images, obtain the correspondence between the pixel coordinates of the calibration chessboard corners and the calibration chessboard coordinates from the calibration chessboard images, and then iteratively solve the homography matrix to decompose the camera's internal parameters, distortion parameters, and external parameters relative to the calibration chessboard to achieve camera calibration. This program mainly uses Opencv (a cross-platform computer vision library) to achieve this function. First, use the cv2.findChessboardCorners function in the Opencv library to find the calibration chessboard obtained by the camera and return the coordinates of each corner of the calibration chessboard, then use the cv2.cornerSubPix function to find the sub-pixel corners of the calibration chessboard, and finally use the cv2.calibrateCamera function to calibrate the camera parameters.

[0068] Step S2: Calibrate the light plane using the camera, line laser and calibration chessboard, and obtain the light plane equation in combination with the camera parameters.

[0069] After the camera calibration is completed, the intrinsic and extrinsic matrix can be obtained, but due to the existence of the scale factor, the two-dimensional pixel coordinates cannot be restored to three-dimensional coordinates. Therefore, a constraint condition, namely the light plane equation, needs to be added to obtain the final three-dimensional coordinates.

[0070] In one embodiment of the present invention, the light plane is calibrated by using a camera, a line laser and a calibration chessboard, and the light plane equation is obtained by combining the camera parameters, including:

[0071] S21: Process the calibration chessboard image to obtain a chessboard coordinate system.

[0072] Specifically, a clear photo of the calibration chessboard is taken with a camera, and the pixel coordinates of all corner points of the calibration chessboard are extracted using the chessboard corner point extraction function of Opencv. The posture of the calibration chessboard is calculated through the PnP (estimated camera pose function) solving function of Opencv, that is, the rotation vector and translation vector of the coordinate system centered on the origin of the calibration chessboard relative to the camera coordinate system are obtained, so as to obtain the coordinate system parameters of the calibration chessboard position. After obtaining the coordinate system parameters of the calibration chessboard position, the coordinate system of the calibration chessboard is extracted.

[0073] S22: Use a line laser to illuminate the calibration chessboard, and obtain a laser image by photographing it with a camera, preprocess the laser image to obtain a laser light stripe area, use an improved grayscale centroid method to extract the laser light stripe center from the laser light stripe area, and obtain the spatial coordinates of the laser light stripe center in the chessboard coordinate system.

[0074] In this embodiment, the calibration chessboard can be kept at the current position, and the line laser is projected onto the surface of the calibration chessboard. The laser image is obtained by photographing with a camera, and then the laser image is preprocessed to obtain the laser light strip area, and the center of the laser light strip is extracted from the laser light strip area using an improved grayscale centroid method. For the chessboard coordinate system, the chessboard surface is equivalent to the plane Z=0, and Z represents the Z axis of the coordinate system. Since the parameters of the chessboard coordinate system have been obtained, the spatial coordinates of the center of the laser light strip in the chessboard coordinate system can be obtained at this time.

[0075] In one embodiment of the present invention, the preprocessing step includes distortion correction, Gaussian filtering and threshold segmentation.

[0076] The step of distortion correction includes: calling the distortion correction function, inputting the laser image, the camera internal parameters and the distortion parameters, and obtaining the distortion-corrected image through the distortion correction function. The step of Gaussian filtering includes: calling the Gaussian smoothing function, inputting the distortion-corrected image, the Gaussian convolution kernel size and the standard deviation value, and obtaining the Gaussian-filtered laser image through the Gaussian smoothing function. The step of threshold segmentation includes: setting a grayscale threshold; in the laser image after Gaussian filtering, setting the pixel points with grayscale values ​​greater than the grayscale threshold as a mask, and setting the grayscale values ​​of the pixels outside the mask to zero to extract the laser light strip area.

[0077] The distortion correction process is specifically as follows: Correcting the distortion of the laser image is a prerequisite for accurately obtaining the coordinates of the three-dimensional point cloud. The distortion correction algorithm used in the present invention is the undistort function that comes with Opencv, that is, the distortion correction function. As long as the laser image and the distortion parameters in the camera are input, the image after distortion correction can be obtained. The internal processing flow of this function mainly has two steps. The first step is to establish a mapping relationship between the undistorted pixel coordinate system and the distorted pixel coordinate system according to the distortion parameters in the camera. The second step is to use the bilinear interpolation method to restore the distorted laser image to an undistorted image, thereby completing the distortion correction.

[0078] The Gaussian filtering process is as follows: the GaussianBlur function in Opencv, i.e., the Gaussian smoothing function, is used to implement Gaussian filtering. As long as the distortion-corrected image, Gaussian convolution kernel size, and standard deviation value are input, the calculation can be performed. This algorithm has little effect on the accuracy of laser light strip center extraction, and has the advantage of reducing the influence of uneven laser distribution and line laser background noise. After Gaussian filtering, the background noise in the image is controlled, and the brightness distribution of the laser light strip becomes more uniform.

[0079] The specific process of threshold segmentation is as follows: In order to extract the laser light strip area separately, this embodiment adopts a threshold-based mask segmentation method. Based on the fact that 95% of the energy of the Gaussian section of the light strip is concentrated within a set range, a fixed value of 76 is selected as the grayscale threshold, and the pixel points with grayscale values ​​greater than the grayscale threshold are set as masks, and the grayscale values ​​of the pixels outside the mask are set to zero, thereby realizing the segmentation of the laser light strip area and then extracting the laser light strip area.

[0080] In one embodiment of the present invention, the center of the laser light stripe is extracted from the laser light stripe region using an improved grayscale centroid method, including:

[0081] like Figure 2 As shown, the first row of the laser image after threshold segmentation is scanned to obtain the grayscale maximum value of the first row of pixels, and a preset threshold is obtained by multiplying the grayscale maximum value by a preset coefficient of 0.7; a spot area is obtained from the first row, and if the grayscale value of the spot area is greater than the preset threshold, the center of gravity of the spot area is extracted, and all the spot areas in the first row are traversed to determine all the spot areas with grayscale values ​​greater than the preset threshold, and the center of gravity of each spot area is extracted; the maximum center of gravity is selected from multiple center of gravity values ​​as the center of gravity of the spot of the row; each row is scanned in turn to obtain the maximum center of gravity of each row, and the pixel points at the position of the maximum center of gravity of each row are fitted and output to obtain the center of the laser light strip, that is, Figure 3 Blue laser light strips in the

[0082] It should be noted that, in this embodiment, Numba (an open source just-in-time compiler) can be used to accelerate the laser light stripe center extraction algorithm.

[0083] S23: The spatial coordinates of the center of the laser light strip in the chessboard coordinate system are converted by using camera parameters to obtain the three-dimensional coordinates of the center of the laser light strip in the world coordinate system.

[0084] For a certain point in space, its coordinate value in the world coordinate system and the coordinate value in the camera coordinate system satisfy:

[0085]

[0086] Among them, M1 is a 4×4 transformation matrix about the world coordinate system, (X W ,Y W ,Z W ) represents the world coordinates based on the plane of the calibrated chessboard, (X C ,Y C ,Z C ) is the camera coordinate with the optical center of the camera as the origin, and the other three axes are parallel to the horizontal boundary, vertical boundary of the camera plane and the camera optical axis respectively.

[0087] At the same time, the coordinate values ​​of the chessboard coordinate system and the camera coordinate system satisfy:

[0088]

[0089] Among them, M2 is a 4×4 transformation matrix about the chessboard coordinate system, (X B ,Y B ,Z B ) are the three coordinate values ​​in the chessboard coordinate system.

[0090] Therefore, the coordinate value of the point in the chessboard coordinate system and the coordinate value in the world coordinate system satisfy:

[0091]

[0092] Since the world coordinate system parameters and chessboard coordinate system parameters have been obtained through coordinate system extraction, M1 and M2 are both known, and the three-dimensional coordinates of the center of the laser light strip in the world coordinate system can be solved by the above formula.

[0093] S24: adjusting the positions of the camera and the line laser, irradiating and photographing the calibration chessboard at different positions, so as to obtain the centers of the multiple laser light strips corresponding to the different positions, and obtain the three-dimensional coordinates of the centers of the multiple laser light strips corresponding to the different positions in the world coordinate system.

[0094] Through the above steps, the world coordinate system point cloud of the center of a laser light strip in the light plane can be extracted. Then, the chessboard position is moved multiple times or the position of the camera and line laser is adjusted to scan and obtain the centers of multiple laser light strips, and obtain the three-dimensional coordinates of the centers of multiple laser light strips in the world coordinate system, that is, to obtain the point cloud of the centers of multiple laser light strips in the light plane.

[0095] S25: Fitting the three-dimensional coordinates of the centers of the multiple laser light strips in the world coordinate system to obtain a light plane equation.

[0096] In this embodiment, after obtaining the three-dimensional coordinates of the centers of the multiple laser light strips, that is, obtaining the light plane point cloud, the light plane equation can be obtained by least squares fitting. In this embodiment, the light plane point cloud map before fitting and the light plane point cloud map after fitting are respectively as follows: Figure 4 and Figure 5 shown.

[0097] Step S3: Use a line laser to scan the target to be reconstructed, and use a camera to capture multiple laser images, obtain the pixel coordinates of the centers of multiple laser light strips from the multiple laser images, and obtain a three-dimensional point cloud of the centers of the multiple laser light strips based on the light plane equation to form a three-dimensional point cloud image of the target to be reconstructed after scanning at the current viewing angle.

[0098] The object to be reconstructed is placed on a rotating platform, such as Figure 6 As shown, turn on the line laser to illuminate the target to be reconstructed, then run the "point cloud processing.py" program, set the single-step value of the stepper motor driver (driving the guide rails for placing the line laser and the camera), scan the target to be reconstructed from one side to the other side at the current viewing angle, obtain the pixel coordinates of the centers of multiple laser light strips, and obtain the three-dimensional point cloud of the centers of multiple laser light strips based on the light plane equation, thereby forming a three-dimensional point cloud image of the target to be reconstructed after scanning at the current viewing angle.

[0099] Step S4: Rotate the target to be reconstructed, use a line laser to scan at different angles to obtain a three-dimensional point cloud image of the target to be reconstructed at different angles, and perform point cloud registration to obtain a complete three-dimensional point cloud image of the target to be reconstructed.

[0100] In this embodiment, the rotating platform is controlled at several special angles, and the above steps are repeated to obtain the three-dimensional point cloud images of the target to be reconstructed at different viewing angles. Figure 7 shown.

[0101] In one embodiment of the present invention, point cloud registration to obtain a complete three-dimensional point cloud image of the target to be reconstructed includes:

[0102] At least four feature points are selected from two three-dimensional point cloud images under different viewing angles for point cloud registration; a pair of point clouds are selected from two three-dimensional point cloud images under different viewing angles, and a transformation matrix that minimizes the distance between a pair of point clouds is iteratively obtained, and the point cloud in a three-dimensional point cloud image is transformed through the transformation matrix to achieve precise point cloud registration.

[0103] Specifically, MeshLab (3D geometry processing software) can be used to align the point cloud. First, a manual rough alignment is performed, and then a point cloud fine alignment is performed.

[0104] Manual rough registration is to select at least 4 feature points from two 3D point cloud images under different viewing angles for point cloud registration, and then manually translate and rotate the point cloud images to splice the images. Furthermore, the distance between two point clouds, i.e. a pair of point clouds, is continuously iterated to minimize to calculate the transformation matrix, so as to achieve high-precision registration of point clouds.

[0105] The two registered 3D point cloud images are merged into a new point cloud image, and the above steps are repeated in sequence until all point clouds of the target surface to be reconstructed are registered, and a complete 3D point cloud image of the target to be reconstructed is obtained, such as Figure 8 shown.

[0106] Step S5: Based on the complete three-dimensional point cloud image of the target to be reconstructed, a surface reconstruction algorithm is used to perform surface reconstruction on the target to be reconstructed, so as to achieve three-dimensional reconstruction of the target to be reconstructed.

[0107] In this embodiment, relevant parameters can be set and the normal vector of the 3D point cloud image can be calculated, and the surface reconstruction algorithm can be used to perform surface reconstruction on the complete 3D point cloud image. On this basis, the reconstruction result can be improved by deleting outliers, removing overlapping data, optimizing surface details, etc., wherein the original image of the target model to be reconstructed is as follows: Fig. 9 As shown, after the target surface is reconstructed, Fig.10 It should be noted that this step is a prior art and will not be described in detail in this embodiment.

[0108] Furthermore, the present invention also provides a three-dimensional reconstruction system based on a monocular camera and a line laser, which is applied to the above-mentioned three-dimensional reconstruction method based on a monocular camera and a line laser. The system includes a camera, a line laser, a calibration chessboard, a guide rail, a rotating turntable and a main control component.

[0109] Among them, the camera and the line laser are arranged on the guide rail, and the camera, the line laser, the guide rail are connected to the main control component; the rotating turntable is used to place the target to be reconstructed; the calibration chessboard is used to realize camera calibration and light plane calibration; the main control component includes a Raspberry Pi component and a terminal computer, and the Raspberry Pi component is used to drive the guide rail so that the line laser and the camera can illuminate and shoot the calibration chessboard or the target to be reconstructed at different positions, and can be used to obtain a three-dimensional point cloud image of the target to be reconstructed after scanning at different viewing angles; the terminal computer is used to realize point cloud registration and surface reconstruction of the target to be reconstructed.

[0110] In this embodiment, the measurement component of the system consists of three parts: a camera, a line laser, and a calibration chessboard. The motion component of the system includes two parts: a guide rail and a rotating turntable. In order to send pulse signals to the stepper motor driver and call the camera, the system uses a Raspberry Pi 4B computer with GPIO (general input and output port) signal input capability and computing performance, that is, the Raspberry Pi component, and uses a laptop as a terminal computer to realize point cloud registration and surface reconstruction of the target to be reconstructed.

[0111] In summary, the present invention not only greatly reduces the hardware cost but also is easy to integrate. At the same time, the combination of the monocular camera and the line laser is relatively economical, making the entire system have a high cost-effectiveness ratio. The present invention uses Raspberry Pi to control the movement of the camera and the line laser, realizes the automatic operation of the control system, and has the ability to achieve fast, stable, and intelligent three-dimensional reconstruction in various complex environments. In addition, the system has a high degree of automation, precise scanning control, and can scan and reconstruct objects from multiple angles. In addition, the present solution has the advantages of high cost-effectiveness, good flexibility, strong scalability, and outstanding innovation. These advantages make the solution have broad application prospects in scientific research, education, industrial manufacturing and other fields.

[0112] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0113] Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.

Claims

1. A three-dimensional reconstruction method based on a monocular camera and a line laser, characterized in that: include: Using a camera to shoot a calibration chessboard to obtain a calibration chessboard image, and calibrating the camera based on the calibration chessboard image to obtain camera parameters; Calibrate the light plane by using a camera, a line laser and a calibration chessboard, and obtain a light plane equation in combination with the camera parameters; The line laser is used to scan the target to be reconstructed, and the camera is used to shoot to obtain a plurality of laser images, pixel coordinates of the centers of a plurality of laser light strips are obtained from the plurality of laser images, and a three-dimensional point cloud of the centers of the plurality of laser light strips is obtained based on the light plane equation to form a three-dimensional point cloud image of the scanned target to be reconstructed at the current viewing angle; Rotate the target to be reconstructed, use the line laser to scan at different viewing angles to obtain a three-dimensional point cloud image of the target to be reconstructed at different viewing angles, and perform point cloud registration to obtain a complete three-dimensional point cloud image of the target to be reconstructed; Based on the complete three-dimensional point cloud image of the target to be reconstructed, a surface reconstruction algorithm is used to perform surface reconstruction on the target to be reconstructed, so as to achieve three-dimensional reconstruction of the target to be reconstructed.

2. The three-dimensional reconstruction method based on a monocular camera and a line laser as claimed in claim 1, characterized in that: The calibration chessboard is photographed by a camera to obtain a calibration chessboard image, and the camera is calibrated based on the calibration chessboard image to obtain camera parameters, including: The camera is used to photograph the calibration chessboard in different positions and postures to obtain images of the calibration chessboard in different positions and postures; Corner point detection is performed on each chessboard image, and each corner point detected in each chessboard image is further detected to obtain pixel coordinates of each sub-pixel corner point. Camera calibration is performed based on the pixel coordinates of each sub-pixel corner point to obtain the camera parameters, which include camera intrinsic parameters, camera extrinsic parameters and distortion parameters.

3. The three-dimensional reconstruction method based on a monocular camera and a line laser as claimed in claim 2, characterized in that: The light plane is calibrated by using a camera, a line laser and a calibration chessboard, and the light plane equation is obtained by combining the camera parameters, including: Processing the calibrated chessboard image to obtain a chessboard coordinate system; The calibration chessboard is illuminated by the line laser, and a laser image is obtained by photographing with the camera, the laser image is preprocessed to obtain a laser light stripe area, the center of the laser light stripe is extracted from the laser light stripe area by an improved grayscale centroid method, and the spatial coordinates of the center of the laser light stripe in the chessboard coordinate system are obtained; The spatial coordinates of the center of the laser light strip in the chessboard coordinate system are converted by using the camera parameters to obtain the three-dimensional coordinates of the center of the laser light strip in the world coordinate system; Adjusting the positions of the camera and the line laser, irradiating and photographing the calibration chessboard at different positions, so as to obtain the centers of the multiple laser light strips corresponding to the different positions, and obtaining the three-dimensional coordinates of the centers of the multiple laser light strips corresponding to the different positions in the world coordinate system; The light plane equation is obtained by fitting the three-dimensional coordinates of the centers of the multiple laser light strips in the world coordinate system.

4. The three-dimensional reconstruction method based on a monocular camera and a line laser as claimed in claim 3, characterized in that: The preprocessing steps include distortion correction, Gaussian filtering and threshold segmentation.

5. The three-dimensional reconstruction method based on a monocular camera and a line laser as claimed in claim 1, characterized in that: The distortion correction step comprises: The distortion correction function is called, the laser image, the camera internal parameters and the distortion parameters are input, and the distortion-corrected image is obtained by processing with the distortion correction function.

6. The three-dimensional reconstruction method based on a monocular camera and a line laser as claimed in claim 5, characterized in that: The Gaussian filtering step comprises: The Gaussian smoothing function is called, and the distortion-corrected image, the Gaussian convolution kernel size and the standard deviation value are input, and the laser image after Gaussian filtering is obtained by processing with the Gaussian smoothing function.

7. The three-dimensional reconstruction method based on a monocular camera and a line laser as claimed in claim 6, characterized in that: The step of threshold segmentation comprises: Set a grayscale threshold; In the laser image after Gaussian filtering, the pixel points whose grayscale values ​​are greater than the grayscale threshold are set as masks, and the grayscale values ​​of the pixel points outside the mask are set to zero, so as to extract the laser light strip area.

8. The three-dimensional reconstruction method based on a monocular camera and a line laser as claimed in claim 7, characterized in that: The center of the laser light stripe is extracted from the laser light stripe region by using an improved grayscale centroid method, including: Scanning the first row of the laser image after threshold segmentation to obtain the grayscale maximum value of the pixels in the first row, and multiplying the grayscale maximum value by a preset coefficient to obtain a preset threshold value; Obtain a light spot area from the first row, and if the grayscale value of the light spot area is greater than the preset threshold, extract the centroid of the light spot area, traverse all the light spot areas in the first row to determine all the light spot areas whose grayscale values ​​are greater than the preset threshold, and extract the centroid of each light spot area; Select the maximum center of gravity from multiple center of gravity values; Scan each row in turn to obtain the maximum center of gravity of each row, and fit the pixel points at the position of the maximum center of gravity of each row to obtain the center of the laser light strip.

9. The three-dimensional reconstruction method based on a monocular camera and a line laser as claimed in claim 1, characterized in that: Point cloud registration is performed to obtain a complete three-dimensional point cloud image of the target to be reconstructed, including: Select at least four feature points from two 3D point cloud images at different viewing angles for point cloud registration; A pair of point clouds in two three-dimensional point cloud images under different viewing angles are selected, and a transformation matrix that minimizes the distance between the pair of point clouds is iteratively obtained. The point cloud in a three-dimensional point cloud image is transformed by the transformation matrix to achieve precise point cloud registration.

10. A three-dimensional reconstruction system based on a monocular camera and a line laser, characterized in that: The method for three-dimensional reconstruction based on a monocular camera and a line laser as described in any one of claims 1 to 9, wherein the system comprises a camera, a line laser, a calibration chessboard, a guide rail, a rotating turntable and a main control component; The camera and the line laser are arranged on the guide rail, and the camera, the line laser and the guide rail are connected to the main control assembly; The rotating turntable is used to place the target to be reconstructed; The calibration chessboard is used to implement camera calibration and light plane calibration; The main control component includes a Raspberry Pi component and a terminal computer. The Raspberry Pi component is used to drive the guide rail so that the line laser and the camera can illuminate and photograph the calibration chessboard or the target to be reconstructed at different positions, and can be used to obtain a three-dimensional point cloud image of the target to be reconstructed after scanning at different viewing angles; the terminal computer is used to realize point cloud registration and surface reconstruction of the target to be reconstructed.

Citation Information

Patent Citations

  • System and method for three-dimensional reconstruction of commodity

    CN109118582A

  • High-precision size detection system and method for three-dimensional workpiece

    CN110763136A

  • Three-dimensional image reconstruction method and system based on monocular structured light and rotating platform

    CN113936099A

  • High-precision line laser three-dimensional reconstruction calibration method

    CN115187676A

  • Underwater target dense three-dimensional reconstruction system and method

    CN118135121A

Cited By

  • Three-dimensional power transmission channel reconstruction method and device combined with linear laser

    CN119762682A

  • Method and device for three-dimensional reconstruction of power transmission lines using cross laser

    CN119762683A

  • Method and device for three-dimensional reconstruction of power transmission lines using cross lasers

    CN119762683B

  • Road corrugated plate integrity detection method based on vehicle-mounted three-dimensional camera

    CN120747098A

  • A highway corrugated plate integrity detection method based on a vehicle-mounted three-dimensional camera

    CN120747098B