A method for generating a three-dimensional color point cloud of an indoor space

Abstract

The application discloses a kind of indoor space three-dimensional color point cloud generation method, utilize the three-dimensional laser SLAM and panoramic video data of indoor collection to generate indoor scene three-dimensional color point cloud, realization process includes indoor space handheld SLAM system data acquisition, indoor space three-dimensional point cloud generation, panoramic position and posture synchronous generation, single frame panoramic and three-dimensional point cloud registration, generate color point cloud.The application is aimed at the generation process of indoor space three-dimensional color point cloud, generates indoor scene point cloud by laser SLAM, and utilizes the pose information of SLAM trajectory to carry out point cloud and panoramic image registration parameter calculation, realizes indoor scene point cloud automatic coloring.This mode solves the problem of lacking initial image pose in point cloud and panoramic image fusion, reduces calculation data volume, improves registration efficiency, provides a new scheme for generating indoor space three-dimensional color point cloud in optoelectronic surveying and mapping field.

Description

Technical Field

[0001] This invention relates to the field of optoelectronic surveying and mapping technology, specifically to a method for generating three-dimensional color point clouds in indoor spaces. Background Technology

[0002] Compared to traditional indoor static laser scanning methods, handheld mobile laser scanning devices can simultaneously and rapidly generate indoor scene point clouds, and can utilize an additional onboard panoramic camera to enrich the color information of the indoor point clouds. Laser SLAM technology is designed for high-precision mapping and localization of indoor scenes. During mobile scanning, it performs its own localization based on sensor positions and the map, and simultaneously builds incremental maps based on its own localization, improving positioning accuracy in the absence of GPS signals. With the maturity of the technology and the reduction in the cost of laser sensors, multi-sensor handheld mobile scanning devices based on laser SLAM will become a more effective tool for indoor scene mapping. The advantages of handheld devices—lightweight, fast, and providing richer data—will continue to emerge, showing broad application prospects in indoor and other small-scene mapping fields.

[0003] Generating 3D color point clouds for indoor spaces requires addressing the issue of adding scene color information to the point clouds. Since lasers primarily use the invisible infrared light band, point clouds acquired by lidar do not inherently possess color information. Indoor optical images captured by various cameras, however, possess RGB color information. The primary challenge in generating 3D color point clouds for indoor spaces is how to assign the color information of objects in these images to the corresponding scene's point clouds. There are generally two methods for generating 3D color point clouds in indoor spaces. One method involves using the camera system built into the scanning equipment to simultaneously acquire images during point cloud acquisition. The images and point clouds are then fused according to the calibration parameters of the camera and LiDAR at the time of manufacture to generate a color point cloud. This method increases the cost of the scanning equipment and is not conducive to widespread application. The other method involves simultaneous acquisition of images by the camera while the LiDAR scans the point cloud. The point cloud and image are directly registered using corresponding features, or the point cloud is converted into a depth map and fused with the image. Alternatively, a densely matched point cloud generated from the image can be fused with the acquired point cloud. Regardless of whether it's "point cloud-image," "depth map-image," or "point cloud-dense point cloud," this method of directly fusing data based on corresponding features is computationally intensive and inefficient, and lacks initial image pose information. Furthermore, feature-based fusion requires a high degree of visual accuracy between the image and the point cloud, making global registration difficult and affecting the accuracy of point cloud coloring. Summary of the Invention

[0004] The purpose of this invention is to provide a method for generating three-dimensional colored point clouds in indoor spaces. This method utilizes three-dimensional laser SLAM data and panoramic video data collected in the indoor space to locate and pose the SLAM point clouds, fusing them to generate indoor space scene point clouds. Frames are extracted from the panoramic video to output panoramic images. Furthermore, based on the time of the indoor space panoramic images, the SLAM pose output is interpolated to the position and orientation of the corresponding panoramic images. Local registration parameters are calculated between the point clouds within a certain range and the panoramic images based on the panoramic pose. Finally, the point clouds within the corresponding range are colored according to the registration parameters and the panoramic images, thus traversing the process to generate a complete indoor scene colored point cloud.

[0005] This invention provides a method for generating a three-dimensional color point cloud of an indoor space. It utilizes the SLAM method to generate an indoor point cloud, and then uses optimized registration parameters and panoramic imagery for batch coloring of the scene point cloud, thereby generating a color point cloud of the indoor scene. The implementation process includes the following steps:

[0006] Step 1: Data acquisition of the handheld SLAM system in indoor space, including setting the parameters of the LiDAR and panoramic camera integrated in the handheld system, and acquiring point cloud and panoramic video data of the indoor space scene.

[0007] Step 2, 3D point cloud generation of indoor space, including using SLAM positioning and pose determination method to fuse and generate scene point cloud and its trajectory, and performing cropping, downsampling and noise reduction processing on scene point cloud;

[0008] Step 3: Synchronous generation of panoramic position and pose, including setting panoramic video frame extraction parameters, generating panoramic images, and using SLAM trajectory interpolation to generate panoramic image pose.

[0009] Step 4, registration of single-frame panorama and 3D point cloud, including point cloud cropping of single-frame panorama range, registration of single panoramic image and point cloud based on line features, calculation of extrinsic parameters of the first frame image, and batch optimization of panoramic pose based on extrinsic parameter correction.

[0010] Step 5: Generate color point cloud, including point cloud segmentation based on panoramic center spacing, point cloud batch coloring based on registration parameters, and color point cloud optimization.

[0011] Furthermore, step 1 includes the following sub-steps,

[0012] Step 1.1: Set the parameters of the handheld SLAM device acquisition system, including the resolution of the 3D LiDAR and the acquisition frequency of the camera;

[0013] Step 1.2: Use a handheld SLAM device to collect 3D laser point cloud data and panoramic video data of the indoor space scene.

[0014] Furthermore, step 2 includes the following sub-steps,

[0015] Step 2.1: Use the FAST_LIO method to perform SLAM positioning and pose determination on the three-dimensional lidar point cloud data, fuse and generate the three-dimensional point cloud of the indoor space scene, and output the trajectory data of the lidar, including time, position, and attitude information;

[0016] Step 2.2: Crop the three-dimensional lidar point cloud to remove the point cloud outside the indoor space scene range;

[0017] Step 2.3: Use the voxel grid filtering method to downsample the scene point cloud, reduce the point cloud density to reduce the data volume, and improve the data processing efficiency;

[0018] Step 2.4: Use the moving least squares method to smooth the three-dimensional lidar point cloud and remove the noise points in the scene.

[0019] Moreover, Step 3 includes the following sub-steps:

[0020] Step 3.1: Set the frame extraction parameters for the panoramic video, including the frame rate for generating the panoramic image, the resolution size, and the timestamp of the starting image;

[0021] Step 3.2: Generate a series of panoramic images according to the frame extraction parameters of the panoramic video and output the timestamp information of each image;

[0022] Step 3.3: According to the panoramic image time and the SLAM trajectory pose, output the lidar position and pose when the panoramic image corresponding to the interpolated timestamp is acquired, that is, the external parameters of the lidar in the acquisition coordinate system at the corresponding moment. The interpolation formula is as follows:

[0023]

[0024] In formula (1), T is the panoramic image acquisition time, T1 and T2 are the time intervals on the laser SLAM trajectory closest to T, that is, T1 < T < T2, F(T1) and F(T2) are the laser trajectory poses corresponding to T1 and T2 respectively, and F(T) is the pose interpolated at time T.

[0025] Moreover, Step 4 includes the following sub-steps:

[0026] Step 4.1: Take a part of the scene point cloud within the relevant spatial range according to the pose of the first-frame lidar obtained in Step 3.3;

[0027] Step 4.2: Use the homologous line features in the panoramic image and the point cloud for registration, use the external parameters in Step 3.3 as the initial registration parameters for iterative convergence, and calculate the correction value of the initial registration parameters after convergence;

[0028] Step 4.3: Calculate the registration parameters between the panoramic image and the point cloud using the correction values, and output the extrinsic parameters of the camera when the panoramic image was acquired.

[0029] Step 4.4: Using the correction values ​​and the LiDAR pose obtained in step 3.3, calculate the camera's extrinsic parameters when acquiring each panoramic image.

[0030] Furthermore, step 5 includes the following sub-steps:

[0031] Step 5.1: Segment the point cloud according to the scene range contained in each panoramic image and the external parameters of the panoramic image, and associate each segment of the point cloud with the panoramic image.

[0032] Step 5.2: Read the extrinsic parameters acquired during the acquisition of the first panoramic image and use them as registration parameters to color the first segment of the point cloud. The coloring formula is as follows:

[0033] Color(C) = Color(P) (2)

[0034] In formula (2), C is the coordinates (X,Y,Z) of any point cloud, P is the coordinates (Row,Col) of the previous pixel in the panoramic image corresponding to the point cloud, and Color(C) and Color(P) are the colors added to the point cloud and the colors of the corresponding pixels themselves, respectively; let the pose parameters of a panoramic image be (X,Y,Z). S ,Y S Z S ,θ, Given that the width and height of the image are r and c respectively, the correspondence between point cloud coordinates (X, Y, Z) and pixel coordinates (Row, Col) can be transformed using the following three equations:

[0035]

[0036]

[0037]

[0038] In formula (3), (x,y,z) are the point cloud coordinates after normalizing the origin of the coordinate system to the center of the sphere, and R θ , R Ψ The rotation matrices are respectively for the three attitude angles, as shown in formulas (6), (7), and (8):

[0039]

[0040]

[0041]

[0042] Step 5.3: Repeat step 5.2 to traverse all panoramic images and segmented point clouds and complete the coloring of all segmented point clouds;

[0043] Step 5.4: Check for deviations in point cloud shading, adjust the pose and recolor until the point cloud shading is basically without deviation, and obtain the indoor scene color point cloud.

[0044] The present invention has the following positive effects:

[0045] 1) This invention proposes a method and technical process for generating three-dimensional color point clouds in indoor spaces. The method generates point clouds of indoor scenes through laser SLAM and uses SLAM trajectory information to calculate the registration parameters between the point clouds and panoramic images. This method provides initial values ​​for the registration of point clouds and panoramic images, reduces the amount of computational data, improves registration efficiency, and realizes automatic coloring of point clouds in indoor scenes.

[0046] 2) This invention performs registration of panoramic and point clouds by interpolating the position and attitude of the lidar when the panoramic image is acquired based on the panoramic image time and SLAM pose output. This enables data fusion of any panoramic camera and lidar without product parameters, and improves the portability of mobile acquisition devices for color point clouds.

[0047] This invention applies 3D laser SLAM to the registration of scene point clouds and panoramic data, enabling the generation of colored point clouds for indoor spatial scenes. This improves the richness of point cloud information in indoor scenes and the portability of acquisition equipment, automatically completing point cloud coloring. It has broad application prospects in fields such as indoor 3D mapping, interior renovation design, and post-disaster reconstruction. Attached Figure Description

[0048] Figure 1 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation

[0049] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0050] To achieve the above objectives, this invention provides a method for generating 3D colored point clouds of indoor spaces. The method utilizes SLAM to generate indoor point clouds and employs optimized registration parameters and panoramic imagery for batch coloring of the scene point clouds, thereby generating colored point clouds of indoor scenes. This process includes using 3D laser SLAM data and panoramic video data acquired in the indoor space to locate and pose the SLAM point clouds, fusing them to generate indoor space scene point clouds, extracting frames from the panoramic video to output panoramic images, further interpolating the SLAM pose output according to the time of the indoor space panoramic images to determine the position and orientation of the corresponding panoramic images, calculating local registration parameters between the point clouds within a certain range based on the panoramic pose and the panoramic images, and finally coloring the corresponding range of point clouds according to the registration parameters and the panoramic images. This process iterates through the points to generate complete colored point clouds of indoor scenes.

[0051] See Figure 1 The embodiment proposes a method for generating a three-dimensional color point cloud of an indoor space, which specifically includes the following steps:

[0052] Step 1) Data acquisition using the handheld SLAM system in indoor space. This step mainly involves setting the acquisition parameters for the LiDAR and panoramic camera used for indoor space scene data acquisition, and using the handheld device to acquire laser point cloud and panoramic video data.

[0053] Further, preferred step 1 includes the following sub-steps,

[0054] Step 1.1: Set the parameters of the handheld SLAM device acquisition system, including the resolution of the 3D LiDAR and the acquisition frequency of the camera;

[0055] Step 1.2: Use a handheld SLAM device to collect 3D laser point cloud data and panoramic video data of the indoor space scene.

[0056] In practice, the following implementation method is recommended:

[0057] First, the acquisition parameters of the handheld SLAM system are set, including the acquisition parameters of the LiDAR and panoramic camera. The feature metrics used in this process are:

[0058] 1) The frame rate of the panoramic video is no less than 30Hz.

[0059] 2) The frame rate of the 3D laser point cloud is not less than 50Hz.

[0060] Secondly, the 3D LiDAR and camera acquisition system are assembled, and the point cloud is output in chronological order.

[0061] Finally, using a pre-configured handheld SLAM system, scene data of a closed indoor room was collected, starting from the initial position, circling the room once, and returning to the starting point. The collected data included time-stamped point clouds and panoramic video from the camera.

[0062] Step 2) Generation of 3D point cloud in indoor space. This step mainly involves positioning the LiDAR sensor based on the laser point cloud collected from the indoor room, fusing the point clouds of each time stamp to generate a complete indoor scene point cloud in the collection coordinate system, and performing processing such as cropping, downsampling, and noise reduction on the point cloud to lay the foundation for the fusion of point cloud and panoramic data.

[0063] The preferred embodiment utilizes the SLAM positioning and orientation method to fuse and generate scene point clouds and their trajectories, and performs cropping, downsampling and noise reduction processing on the scene point clouds.

[0064] Further, preferred step 2 includes the following sub-steps,

[0065] Step 2.1: Use the FAST_LIO method to perform SLAM positioning and pose determination on the 3D laser point cloud data, fuse them to generate a 3D point cloud of the indoor space scene, and output the trajectory data of the lidar, including time, position and attitude information.

[0066] Step 2.2: Trim the 3D laser point cloud to remove point clouds outside the indoor space scene area;

[0067] Step 2.3: Use voxel grid filtering to downsample the scene point cloud, reduce the point cloud density to reduce the amount of data and improve data processing efficiency;

[0068] Step 2.4: The moving least squares method is used to smooth the 3D laser point cloud and remove noise points in the scene.

[0069] In practice, the following implementation method is recommended:

[0070] First, an acquisition coordinate system is established using the initial acquisition position as the origin for the 3D laser point cloud data acquired in chronological order. Then, frame-by-frame registration and optimization of the point cloud are performed using the laser SLAM method to generate a complete indoor scene point cloud with timestamps, coordinates, and intensity information in the acquisition coordinate system, but excluding RGB color information. The feature metrics used in this process include:

[0071] 1) The laser SLAM uses the FAST_LIO method, which is based on iterative Kalman filtering to achieve registration, balancing efficiency and robustness.

[0072] 2) The output laser trajectory timestamp uses Unix time, and the frame rate is 50 frames per second.

[0073] Secondly, it supports users to select and delete non-indoor point clouds outside of indoor scenes, such as point clouds outside windows and doors.

[0074] Secondly, the point cloud of the indoor scene is downsampled to reduce the amount of data and improve data processing efficiency. The feature metrics used in this process include:

[0075] 1) Downsampling uses a voxel grid filtering method, with the voxel grid centroid as a simplification of the voxel laser point cloud.

[0076] 2) The size of the voxel mesh is taken as the maximum side length of the bounding box of the UAV.

[0077] Finally, the 3D laser point cloud is smoothed using the moving least squares method to further remove some outlier noise points isolated from the main body of the point cloud.

[0078] Step 3), generating panoramic position and pose synchronously, including setting panoramic video frame extraction parameters, generating panoramic images, and generating panoramic image poses by interpolating the SLAM trajectory;

[0079] In the embodiment, this step extracts frames from the collected panoramic video to generate multiple panoramic images of the indoor scene with timestamps, interpolates the laser trajectory obtained in step 2) according to the timestamps of the panoramic images, and the interpolation method is shown in formula (1) to obtain the lidar pose corresponding to the timestamp, providing initial external parameters for the point cloud and panoramic fusion.

[0080] Furthermore, preferably, step 3 includes the following sub-steps.

[0081] Step 3.1, setting the frame extraction parameters of the panoramic video, including the frame rate for generating panoramic images, the resolution size, and the timestamp of the starting image;

[0082] Step 3.2, generating a series of panoramic images according to the frame extraction parameters of the panoramic video and outputting the timestamp information of each image;

[0083] Step 3.3, according to the panoramic image time and the SLAM trajectory pose, output the lidar position and pose when obtaining the interpolated panoramic image corresponding to the timestamp, that is, the external parameters of the lidar in the acquisition coordinate system at the corresponding moment. The interpolation formula is as follows:

[0084]

[0085] In formula (1), T is the time when the panoramic image is obtained, T1 and T2 are the time intervals closest to T on the laser SLAM trajectory, that is, T1 < T < T2, F(T1) and F(T2) are the laser trajectory poses corresponding to T1 and T2 respectively, and F(T) is the pose interpolated at time T.

[0086] Suggestions for specific implementation are as follows:

[0087] First, according to the size and complexity of the indoor scene, set reasonable frame extraction parameters. The characteristic indicators used in this process are:

[0088] 1) The resolution of the panoramic image is set to 3040×6080.

[0089] 2) The extraction frequency is set to one frame every two seconds, and the starting time is the camera system time at the start of the acquisition.

[0090] Secondly, extract frames from the panoramic video according to the frame extraction parameters to obtain panoramic images with timestamps.

[0091] Finally, based on the timestamp of each panoramic image frame, the pose corresponding to the nearest timestamp range is searched in the laser trajectory, and these poses are interpolated according to the timestamp to obtain the laser pose corresponding to the timestamp of each panoramic image frame.

[0092] Step 4) Registration of single-frame panorama and 3D point cloud, including point cloud cropping of single-frame panorama range, registration of single panoramic image and point cloud based on line features, calculation of extrinsic parameters of the first frame image, and batch optimization of panoramic pose based on extrinsic parameter correction.

[0093] In this embodiment, this step is based on the panoramic pose obtained by interpolating the LiDAR SLAM trajectory. Point clouds within the corresponding range of the panoramic image are extracted, and local registration of the panoramic image and the laser point cloud is performed using the line feature registration method. The correction value of the panoramic pose is calculated, and the correction value is used to correct the laser pose of all panoramic images to obtain the camera pose of the panoramic image in the acquisition coordinate system, providing accurate parameters for point cloud coloring.

[0094] Furthermore, preferred step 4 includes the following sub-steps:

[0095] Step 4.1: Take a portion of the scene point cloud within a certain spatial range based on the first frame of lidar pose obtained in step 3.3;

[0096] Step 4.2: Use the corresponding line features in the panoramic image and point cloud for registration, use the extrinsic parameters in step 3.3 as the initial registration parameters for iterative convergence, and calculate the correction value of the initial registration parameters after convergence.

[0097] Step 4.3: Calculate the registration parameters between the panoramic image and the point cloud using the correction values, and output the extrinsic parameters of the camera when the panoramic image was acquired.

[0098] Step 4.4: Using the correction values ​​and the LiDAR pose obtained in step 3.3, calculate the camera's extrinsic parameters when acquiring each panoramic image.

[0099] In practice, the following implementation method is recommended:

[0100] First, processing begins with the first panoramic image. Based on the range of the scene contained in the panoramic image and the distance between the corresponding panoramic laser pose and the poses of other panoramic images, the point cloud range for coloring the panoramic image is set. The scene point cloud is then segmented, with each segment corresponding to a panoramic image.

[0101] Secondly, using the corresponding line features on the panoramic image and the segmented point cloud as the registration basis, the laser pose corresponding to the panoramic image is used as the initial pose value for the panoramic camera for registration. This establishes the correspondence between the 3D coordinates of the starting point cloud and the 2D plane coordinates of the panoramic image. The X, Y, and Z coordinates and the three attitude angles are adjusted until the registration converges, and the correction value of the panoramic image pose is output. The feature indicators used in this process include:

[0102] 1) Select at least three corresponding line features from panoramic images and point clouds.

[0103] 2) The basis for the completion of pose correction is the convergence of the registration error equation.

[0104] Finally, the laser pose of each panoramic image is corrected using the panoramic image pose correction value to obtain the camera pose of the panoramic image. Since the installation positions of the panoramic camera and LiDAR on the acquisition equipment are relatively fixed, the correction value should be applicable to the pose value calculation of all panoramic images from LiDAR to camera arm.

[0105] Step 5) Generate indoor scene color point cloud, including point cloud segmentation based on panoramic center spacing, point cloud batch coloring based on registration parameters, and color point cloud optimization.

[0106] In this embodiment, the step uses the camera pose after panoramic image correction as a parameter to color the segmented point cloud corresponding to each panoramic image. The process is repeated for all panoramic images until the point cloud of each segment is colored. If there is a deviation in the coloring, the pose is fine-tuned to achieve the generation of colored point clouds for indoor space scenes.

[0107] Furthermore, preferred step 5 includes the following sub-steps:

[0108] Step 5.1: Segment the point cloud according to the scene range contained in each panoramic image and the external parameters of the panoramic image, and associate each segment of the point cloud with the panoramic image.

[0109] Step 5.2: Read the extrinsic parameters from the first panoramic image acquisition and color the first point cloud segment according to these registration parameters. The coloring formula is as follows:

[0110] Color(C) = Color(P) (10)

[0111] In formula (2), C is the coordinates (X,Y,Z) of any point cloud, P is the coordinates (Row,Col) of the previous pixel in the panoramic image corresponding to the point cloud, and Color(C) and Color(P) are the colors added to the point cloud and the colors of the corresponding pixels themselves, respectively; let the pose parameters of a panoramic image be (X,Y,Z). S ,Y S Z S ,θ, If the width and height of the image are r and c respectively, then the correspondence between the point cloud coordinates (X,Y,Z) and the pixel coordinates (Row,Col) can be transformed using the following three equations:

[0112]

[0113]

[0114]

[0115] In formula (3), (x,y,z) are the point cloud coordinates after normalizing the origin of the coordinate system to the center of the sphere, and R θ , R Ψ The rotation matrices are respectively for the three attitude angles, as shown in formulas (6), (7), and (8):

[0116]

[0117]

[0118]

[0119] Step 5.3: Repeat step 5.2 to traverse all panoramic images and segmented point clouds and complete the coloring of all segmented point clouds;

[0120] Step 5.4: Check for deviations in point cloud shading, adjust the pose and recolor until the point cloud shading is basically without deviation, and obtain the indoor scene color point cloud.

[0121] In specific implementation, the following implementation method is recommended: First, take the X, Y, and Z coordinates in the panoramic camera pose as the camera center, and the three pose angles represent the orientation of the panoramic camera's coordinate axes. Transform the point cloud coordinates into the corresponding two-dimensional coordinates on the panoramic image, as shown in formulas (3), (4), and (5).

[0122] Secondly, the RGB color value of the pixel at the two-dimensional coordinate of the panoramic image is assigned to all the three-dimensional point clouds corresponding to that pixel coordinate, so that each segment of the point cloud has color information, thus completing the coloring, as shown in formula (2). The feature indicators used in this process are:

[0123] 1) At the connection points of each segment of the point cloud, the same color parts of the same object should belong to the same color band.

[0124] 2) Criteria for judging whether scene point cloud shading is successful: Whether the overall shading deviation of the scene point cloud is obvious. If each segment of the point cloud has points with obvious color errors, the correction value may not be applicable to pose correction of all panoramic images, indicating that there is an error in steps 1) to 4), and you should go back to check.

[0125] Finally, check if there are any deviations in the scene point cloud coloring, such as at corners or where objects are occluded. Fine-tune the pose of the deviations and recolor them to obtain the colored point cloud of the indoor space scene.

[0126] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0127] In some possible embodiments, an indoor space three-dimensional color point cloud generation system is provided, including a processor and a memory. The memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a method for radiometric calibration of UAV spectral images using geometric distortion parameters as described above.

[0128] In some possible embodiments, an indoor space three-dimensional color point cloud generation system is provided, including a readable storage medium on which a computer program is stored. When the computer program is executed, it implements a method for radiometric calibration of UAV spectral images using geometric distortion parameters as described above.

[0129] The above description is merely a preferred embodiment of the present invention. The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains can make various modifications or additions to the described specific embodiments or use similar methods to replace them, but without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A method for generating a three-dimensional colored point cloud in an indoor space, characterized in that: Generate indoor point clouds using the SLAM method, and perform batch coloring of scene point clouds by optimizing registration parameters and panoramic images, thereby generating indoor scene colored point clouds; The implementation process includes the following steps, Step 1, Data collection of a handheld SLAM system in an indoor space, including setting parameters for the lidar and panoramic camera integrated in the handheld system, and collecting point cloud and panoramic video data for the indoor space scene; Step 2, Generation of three-dimensional point clouds in an indoor space, including generating scene point clouds and their trajectories by fusing using the SLAM positioning and pose estimation method, and performing cropping, downsampling, and denoising processing on the scene point clouds; Step 3, Synchronous generation of panoramic position and pose, including setting panoramic video frame extraction parameters, generating panoramic images, and generating panoramic image poses by interpolating using the SLAM trajectory; The implementation process includes the following sub-steps, Step 3.1, Set the frame extraction parameters of the panoramic video, including the frame rate for generating panoramic images, the resolution size, and the timestamp of the starting image; Step 3.2, Generate a series of panoramic images according to the frame extraction parameters of the panoramic video, and output the timestamp information of each image; Step 3.3, According to the panoramic image time and the SLAM trajectory pose, output the position and pose of the lidar when the panoramic image corresponding to the interpolated timestamp is acquired, that is, the external parameters of the lidar in the acquisition coordinate system at the corresponding moment. The interpolation formula is as follows: In formula (1), T is the time when the panoramic image is acquired, T1 and T2 are the time intervals on the laser SLAM trajectory closest to T, that is, T1 < T < T2, F(T1) and F(T2) are the laser trajectory poses corresponding to T1 and T2 respectively, and F(T) is the pose interpolated at time T; Step 4, Registration of a single-frame panoramic image and three-dimensional point clouds, including cropping of point clouds within the range of a single-frame panoramic image, registration of a single panoramic image and point clouds based on line features, calculation of the external parameters of the first-frame image, and batch optimization of panoramic poses based on external parameter correction values; The implementation process includes the following sub-steps, Step 4.1, Take partial scene point clouds within the relevant spatial range according to the pose of the first-frame lidar obtained in Step 3.3; Step 4.2, Use the homologous line features in the panoramic image and point clouds for registration, take the external parameters in Step 3.3 as the initial registration parameters for iterative convergence, and calculate the correction value of the initial registration parameters after convergence; Step 4.3, Calculate the registration parameters of the panoramic image and point clouds using the correction value, and output the external parameters of the camera when the panoramic image of this frame is acquired; Step 4.4, Calculate the external parameters of the camera when each panoramic image is acquired using the correction value and the lidar pose obtained in Step 3.3; Step 5, Generate color point clouds, including point cloud segmentation according to the panoramic center spacing, batch coloring of point clouds based on registration parameters, and optimization of colored point clouds.

2. The method for generating three-dimensional color point clouds in indoor space according to claim 1, characterized in that: Step 1 includes the following sub-steps, Step 1.1, Set the parameters of the handheld SLAM device acquisition system, including the resolution of the three-dimensional lidar and the acquisition frequency of the camera; Step 1.2, Use the handheld SLAM device to collect three-dimensional laser point cloud data and panoramic video data of the indoor space scene.

3. The method for generating a three-dimensional color point cloud in an indoor space according to claim 1, characterized in that: Step 2 includes the following sub-steps, Step 2.1: Use the FAST_LIO method to perform SLAM positioning and pose determination on the 3D laser point cloud data, fuse them to generate a 3D point cloud of the indoor space scene, and output the trajectory data of the lidar, including time, position and attitude information. Step 2.2: Trim the 3D laser point cloud to remove point clouds outside the indoor space scene area; Step 2.3: Use voxel grid filtering to downsample the scene point cloud, reduce the point cloud density to reduce the amount of data and improve data processing efficiency; Step 2.4: The moving least squares method is used to smooth the 3D laser point cloud and remove noise points in the scene.

4. The method for generating a three-dimensional color point cloud of an indoor space according to claim 1, 2, or 3, characterized in that: Step 5 includes the following sub-steps: Step 5.1: Segment the point cloud according to the scene range contained in each panoramic image and the external parameters of the panoramic image, and associate each segment of the point cloud with the panoramic image. Step 5.2: Read the extrinsic parameters acquired during the acquisition of the first panoramic image and use them as registration parameters to color the first segment of the point cloud. The coloring formula is as follows: In formula (2), C is the coordinates (X,Y,Z) of any point cloud, P is the coordinates (Row,Col) of the previous pixel in the panoramic image corresponding to the point cloud, and Color(C) and Color(P) are the colors added to the point cloud and the colors of the corresponding pixels themselves, respectively; let the pose parameters of a panoramic image be (X,Y,Z). S ,Y S Z S , If the width and height of the image are r and c respectively, then the correspondence between the point cloud coordinates (X,Y,Z) and the pixel coordinates (Row,Col) can be transformed using the following three equations: In formula (3), (x, y, z) are the point cloud coordinates after normalizing the origin of the coordinate system to the center of the sphere. , , The rotation matrices are respectively for the three attitude angles, as shown in formulas (6), (7), and (8): Step 5.3: Repeat step 5.2 to traverse all panoramic images and segmented point clouds and complete the coloring of all segmented point clouds; Step 5.4: Check for deviations in point cloud shading, adjust the pose and recolor until the point cloud shading is basically without deviation, and obtain the indoor scene color point cloud.

Images