Three-dimensional color modeling method and system based on multi-sensor fusion

Abstract

The invention discloses a three-dimensional color modeling method and system based on multi-sensor fusion, and the method comprises the steps: obtaining monocular vision pose information, an RGB image and radar three-dimensional point cloud information, enabling the monocular vision pose information to be used for correcting a radar pose, and generating corrected point cloud information; the corrected point cloud information is screened and then projected to a specific plane; proper feature points are extracted from the projection image and the RGB image, and according to the corresponding relation between the feature points of the projection image and the RGB image, the plane point cloud of the projection image and the pixel coordinate plane of the RGB image are registered; performing fusion modeling on the point cloud and the image data by using an MRF model; according to the method, the radar pose is corrected by using visual pose information, the accumulated error influence of a system is reduced, the corrected point cloud is projected to a plane, and feature points of a projection image and a color image are extracted to establish an MRF model for describing the relationship between a pixel plane and the corresponding point cloud, so that a point cloud image which is uniformly distributed in space and contains color information is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional modeling, and in particular to a three-dimensional color modeling method and system based on multi-sensor fusion. Background Art

[0002] LiDAR has many advantages. Compared with infrared cameras and visible light cameras, it is not easily affected by changing factors such as ambient temperature and illumination, and can quickly obtain stable information about the target to be measured. However, it still lacks specific texture and color information of the environment to be measured.

[0003] The three-dimensional space reconstruction method based on visual image sensors can utilize relatively low-cost visual sensors to achieve three-dimensional space reconstruction of the target scene, and contains specific texture and color information, but lacks depth information of objects in space.

[0004] Therefore, integrating LiDAR and image sensors to make up for their respective shortcomings and improve the performance of the overall system is an important area of ​​research and exploration. However, how to quickly and accurately align and fuse the point cloud data obtained by LiDAR with the RGB images collected by visible light cameras is a difficult point. Summary of the invention

[0005] The main technical problem solved by the present invention is to provide a three-dimensional color modeling method and system based on multi-sensor fusion, to fuse laser sensors and visual sensors, to calibrate the laser sensors using visual sensors, and to fuse the corrected point cloud information with image information, thereby compensating for the defects of each sensor and improving the overall performance of the system.

[0006] In order to solve the above technical problems, a technical solution adopted by the present invention is to provide a three-dimensional color modeling method based on multi-sensor fusion, the steps of which are:

[0007] Step 1: Build a multi-sensor detection platform consisting of laser radar and visual sensors;

[0008] Step 2: Obtain three-dimensional point cloud information of the scene environment based on the laser radar;

[0009] Step 3: Obtain RGB image and visual pose information based on the visual sensor;

[0010] Step 4: Using the extended Kalman filter method, the monocular vision posture information is used to correct the radar posture and generate the corrected point cloud information;

[0011] Step 5: Screen and process the corrected point cloud information, remove outliers and noise points, calculate the unit angle through statistical analysis principles, and project the corrected point cloud information onto a specific plane;

[0012] Step 6: Extract appropriate feature points from the projection image and the RGB image, obtain the corresponding relationship between the feature points based on the RANSAC algorithm, minimize the root mean square error, and align the plane point cloud of the projection image with the pixel coordinate plane of the RGB image;

[0013] Step 7: Use the MRF model to fuse the point cloud and image data, generate texture information with color, and complete the modeling operation after image depth restoration.

[0014] Furthermore, in the step one, the laser radar and the visual sensor are fixedly connected by a mechanical structure so that the focal length is constant, thereby ensuring that the transformation equation between the point cloud coordinate system and the physical coordinate system of the visual sensor is constant and does not change due to the movement of the detection platform.

[0015] Furthermore, in step 3, the process of obtaining visual posture information is as follows:

[0016] After basic processing of the image data, data tracking is completed through feature extraction and matching, and then the camera motion trajectory is estimated using the principle of epipolar geometry, and feature point depth recovery is achieved through triangulation.

[0017] Furthermore, in step 3, the process of implementing depth recovery of feature points by triangulation is as follows:

[0018] 1) Down-sample the reconstructed point cloud data using the point cloud threshold constrained voxel centroid neighboring point algorithm to simplify the data while retaining the local features of the original data;

[0019] 2) Smooth the simplified point cloud through statistical filtering and Gaussian filtering, remove the noise points of the point cloud, and obtain smooth data;

[0020] 3) For any point in the point cloud, use the octree to search the neighborhood space of the point, quickly obtain the neighborhood point cloud of the point, and use the MLS algorithm to calculate the normal vector of each point cloud;

[0021] 4) Project the point cloud onto a two-dimensional plane through the normal vector, perform plane triangulation using the Del aunay space region growing algorithm, construct a point cloud sample triangle, and determine a growth edge. Take the point with the smallest cosine value of the angle between the growth edge and the projection point on the plane as the extension point, and expand the triangle from the inside to the outside until all the point cloud data are traversed to form a complete triangulated mesh.

[0022] Furthermore, in step four, the process of using monocular vision posture information to correct radar posture is as follows: using laser posture as predicted value and visual posture as measured value, obtaining predicted value at moment k through input at moment k-1, and then correcting predicted value with observed value at moment k, estimating optimal state at moment k, continuously performing iterative calculation, and deducing state and error at current moment according to state at previous moment, and finally obtaining fusion posture of laser and vision.

[0023] Furthermore, in step 5, the process of screening the corrected point cloud information is as follows:

[0024] 1) Perform voxel filtering on cloud points without changing the geometric shape of the point cloud model to reduce the number of point clouds, and then use statistical filtering to remove sparsely distributed point clouds that are far away from surrounding data points and have a small number;

[0025] 2) Construct a grid map, set the height difference threshold of the point cloud in each grid in the vertical direction of the ground for screening, and then obtain the restricted range of the extracted ground points to achieve pre-screening of the ground point cloud;

[0026] 3) Use the RANSAC algorithm to perform plane fitting on the pre-screened point cloud set to achieve precise screening of ground points.

[0027] Furthermore, in the step five, a one-to-one mapping method is used to project the corrected 3D point cloud into the 2D pixel space.

[0028] Furthermore, in step six, the process of aligning the planar point cloud of the projection image with the pixel coordinate plane of the RGB image is as follows: combining the intrinsic parameter matrix of the visual sensor, using the checkerboard joint calibration method to calibrate the lidar and the visual sensor, and according to the same-name feature point pairs between the projection image and the RGB image, through the coordinate transformation matrix obtained through the initial calibration, the point cloud data collected from the same perspective is associated with the corresponding RGB values.

[0029] Furthermore, in step seven, bilinear interpolation is performed on the normalized constants in the MRF model to set variable values, and the model parameters are estimated and updated by the maximum expectation algorithm (EM).

[0030] In order to solve the above technical problems, another technical solution adopted by the present invention is to provide a three-dimensional color modeling system based on multi-sensor fusion, characterized in that it includes a laser acquisition module, a visual acquisition module, a correction module, a registration module and a model building module, wherein:

[0031] Laser acquisition module: obtains the three-dimensional point cloud information of the scene environment based on the laser radar, and sends the point cloud information to the correction module;

[0032] Visual acquisition module: obtains RGB images and visual posture information based on the visual sensor, sends the RGB images to the registration module, and sends the visual posture information to the correction module;

[0033] Correction module: The extended Kalman filter method is used to correct the received radar posture using the received monocular vision posture information, and generate the corrected point cloud information;

[0034] Registration module: Screen and process the corrected point cloud information, remove outliers and noise points, calculate the unit angle through statistical analysis principles, and project the corrected point cloud information to a specific plane; extract appropriate feature points from the projected image and the RGB image, obtain the corresponding relationship between the feature points between the two based on the RANSAC algorithm, minimize the root mean square error, and register the plane point cloud of the projected image with the pixel coordinate plane of the RGB image;

[0035] Model building module: Use the MRF model to fuse point cloud and image data, generate texture information with color, and complete the modeling operation after image depth restoration.

[0036] The beneficial effects of the present invention are:

[0037] 1. The present invention fixes a laser sensor and a visual sensor on the same platform to achieve information acquisition by multiple sensors at the same viewing angle at the same time, and uses the posture information obtained by the visual sensor to correct the posture of the radar, thereby reducing the impact of the system's cumulative error. At the same time, the corrected point cloud is projected onto a specific plane, and the feature points of the projected image and the color image obtained by the visual sensor are extracted to establish an MRF model that describes the relationship between the pixel plane and the corresponding point cloud, thereby obtaining a spatially uniformly distributed point cloud image containing color information, thereby achieving color construction of a three-dimensional dense map of multi-sensor fusion.

[0038] 2. The point cloud data reconstructed by monocular vision is sparse and lacks local detail features. Therefore, the point cloud threshold is used to constrain the downsampling of the neighboring points of the voxel centroid to simplify the point cloud and improve the efficiency of the algorithm. Noise points are removed by statistical filtering and Gaussian filtering to obtain smooth data. Then, in order to obtain a more accurate normal vector, the point cloud normal vector is estimated based on the octree-optimized moving least squares method to reduce spatial complexity.

[0039] 3. In the present invention, during the scanning process, not only the surrounding environment and obstacle information are scanned, but also the ground is scanned. Therefore, the amount of calculation is large and the noise is obvious after the position fusion of the laser sensor and the visual sensor. It is achieved by constructing a grid map and setting the height difference threshold of the point cloud in each grid in the vertical direction of the ground for screening, thereby obtaining the restricted range of the extracted ground points and realizing the pre-screening of the ground point cloud. Finally, the RANSAC algorithm is used to perform plane fitting on the pre-screened point cloud set to achieve precise screening of ground points. Ground point cloud filtering can reduce the amount of calculation and improve the subsequent obstacle detection accuracy.

[0040] In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only four of the drawings in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 It is a flowchart of the application method;

[0043] Figure 2 This is the statistical chart of the standard surface 1 results before and after the MRF model optimization;

[0044] Figure 3 This is the statistical chart of the standard surface 2 results before and after the MRF model optimization;

[0045] Figure 4 It is a structural block diagram of the system of this application. DETAILED DESCRIPTION

[0046] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. Preferred embodiments of the present invention are provided in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in this specification. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0047] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more related listed items.

[0048] See also Figure 1-3 , Figure 1 The present invention shows a three-dimensional color modeling method based on multi-sensor fusion, the steps of which are:

[0049] Step 1: Build a multi-sensor detection platform consisting of laser radar and visual sensors;

[0050] Step 2: Obtain three-dimensional point cloud information of the scene environment based on the laser radar;

[0051] Step 3: Obtain RGB image and visual pose information based on the visual sensor;

[0052] Step 4: Using the extended Kalman filter method, the monocular vision posture information is used to correct the radar posture and generate the corrected point cloud information;

[0053] Step 5: Screen and process the corrected point cloud information, remove outliers and noise points, calculate the unit angle through statistical analysis principles, and project the corrected point cloud information onto a specific plane;

[0054] Step 6: Extract appropriate feature points from the projection image and the RGB image, obtain the corresponding relationship between the feature points based on the RANSAC algorithm, minimize the root mean square error, and align the plane point cloud of the projection image with the pixel coordinate plane of the RGB image;

[0055] Step 7: Use the MRF model to fuse the point cloud and image data, generate texture information with color, and complete the modeling operation after image depth restoration.

[0056] In the step S1, the laser radar and the visual sensor are fixedly connected by a mechanical structure to keep the focal length constant, thereby ensuring that the transformation equation between the point cloud coordinate system and the physical coordinate system of the visual sensor is constant and does not change due to the movement of the detection platform; a laser radar and a visual sensor are used, preferably a visible light camera or a CCD industrial camera, so the conversion between the coordinates of the two can be achieved through the positional relationship between the visual sensor and the laser radar.

[0057] In step S3, the process of obtaining visual posture information is as follows:

[0058] After basic processing of the image data, data tracking is completed through feature extraction and matching, and then the camera motion trajectory is estimated using the principle of epipolar geometry, and feature point depth recovery is achieved through triangulation.

[0059] The ORB algorithm is used to extract features in the environment; the FLANN algorithm is used for feature matching.

[0060] In step S3, the process of implementing feature point depth recovery by triangulation is as follows:

[0061] 1) Down-sample the reconstructed point cloud data using the point cloud threshold constrained voxel centroid neighboring point algorithm to simplify the data while retaining the local features of the original data;

[0062] 2) Smooth the simplified point cloud through statistical filtering and Gaussian filtering, remove the noise points of the point cloud, and obtain smooth data;

[0063] 3) For any point in the point cloud, use the octree to search the neighborhood space of the point, quickly obtain the neighborhood point cloud of the point, and use the MLS algorithm to calculate the normal vector of each point cloud;

[0064] 4) Project the point cloud onto a two-dimensional plane through the normal vector, perform plane triangulation using the Del aunay space region growing algorithm, construct a point cloud sample triangle, and determine a growth edge. Take the point with the smallest cosine value of the angle between the growth edge and the projection point on the plane as the extension point, and expand the triangle from the inside to the outside until all the point cloud data are traversed to form a complete triangulated mesh.

[0065] In step S4, the process of using the monocular vision posture information to correct the radar posture is as follows: using the laser posture as the predicted value and the visual posture as the measured value, obtaining the predicted value at time k through the input at time k-1, and then correcting the predicted value with the observed value at time k, estimating the optimal state at time k, continuously performing iterative calculations, and deducing the state and error at the current moment based on the state at the previous moment, and finally obtaining the fusion posture of laser and vision.

[0066] In step S5, the process of screening the corrected point cloud information is as follows:

[0067] 1) Perform voxel filtering on cloud points without changing the geometric shape of the point cloud model to reduce the number of point clouds, and then use statistical filtering to remove sparsely distributed point clouds that are far away from surrounding data points and have a small number;

[0068] 2) Construct a grid map, set the height difference threshold of the point cloud in each grid in the vertical direction of the ground for screening, and then obtain the restricted range of the extracted ground points to achieve pre-screening of the ground point cloud;

[0069] 3) Use the RANSAC algorithm to perform plane fitting on the pre-screened point cloud set to achieve precise screening of ground points.

[0070] Among them, a one-to-one mapping method is used to project the corrected 3D point cloud into the 2D pixel space.

[0071] In step S6, the process of aligning the planar point cloud of the projection image with the pixel coordinate plane of the RGB image is as follows: combining the intrinsic parameter matrix of the visual sensor, the laser radar and the visual sensor are calibrated using the checkerboard joint calibration method, and according to the same-name feature point pairs between the projection image and the RGB image, the coordinate transformation matrix obtained through the initial calibration is used to associate the point cloud data collected from the same perspective with the corresponding RGB values.

[0072] The feature points of the image are extracted based on Harris's corner point extraction algorithm. Specifically, a window of appropriate size is selected in the image and moved. The presence of corner points is determined based on the degree of change in the eigenvalue of the image in the window. When the eigenvalue changes drastically, it indicates that the corner point exists; when the eigenvalue changes slowly or fluctuates within a certain range, it indicates that the corner point does not exist.

[0073] In step S7, bilinear interpolation is performed on the normalized constants in the MRF model to set variable values, and the model parameters are estimated and updated by the maximum expectation algorithm (EM).

[0074] To build an MRF model, you need to determine the neighborhood range and define the potential function, and reduce the amount of model calculation through the posterior distribution, as shown below:

[0075]

[0076] Where Z is the canonical function (normalization constant), X is the image pixel information, and K is the depth information after the 3D point cloud is converted to a plane;

[0077] The potential function is defined as follows:

[0078]

[0079] Where L is the set of points in the neighborhood of i, k is a constant weight placed on the depth measurement, and Ψ is the weighted quadratic difference between adjacent nodes;

[0080] The prior energy function of deep smoothing can be expressed by the following formula:

[0081]

[0082] Where N (i) is the set of adjacent nodes, Φ is the weighted value of the distance between adjacent nodes;

[0083] Auxiliary nodes I and J are also key factors, which can provide the association between the various graphic layers of MRF. ij They all correspond to the deterministic function of the relationship between two adjacent depth image nodes, and the calculation results are as follows:

[0084] J ij=c×I ij (4)

[0085]

[0086] In the formula, c is a constant, I ij It is the determination function corresponding to the relationship between two adjacent pixel nodes, and σ is the basis for judging the relationship between adjacent points.

[0087] Therefore, the depth map recovery algorithm is relatively simple. It only needs to perform bilinear interpolation on the normalization constant Z to set the Y value, and estimate and update the model parameters through the EM algorithm.

[0088] The following is a test verification of the solution proposed in this application:

[0089] The RE05 laser radar provided by Ocular Robotics and the PikeF-100B / C industrial camera of AlliedVision are used. They are fixedly connected through a mechanical structure to ensure that the relative position relationship between the two devices is certain. This fixed connection method ensures that the relationship between the basic coordinate system of the point cloud data and the physical coordinate system of the camera is determined and does not change due to the movement of the experimental platform.

[0090] The regular hexahedron formed by standard planes is used as the sampling target. The offsets of the data points on the standard planes collected before and after model optimization to the fitting plane are compared. The point cloud offset is used to evaluate the depth map optimized by the model, and the point cloud offset is used to evaluate the quality of the model.

[0091] Depend on Figure 2 and Figure 3 It can be seen that after the MRF model optimization processing, the offset of the point cloud is sharply reduced. Considering the accuracy of the laser radar and the accuracy of the standard plane, it is believed that when the offset of the point exceeds 0.01m, the collected data is invalid. Tables 1 and 2 are the statistical results of the offset of the points on two planes of the regular hexahedron. After calculation, the data accuracy was improved by 16.16% and 12.85% respectively after the MRF model was optimized. By collecting data from multiple standard planes, it is calculated that the MRF model optimization can improve the accuracy by about 15%.

[0092] Table 1 Results of standard surface 1

[0093] Offset / m Without MRF Using MRF 1 0 to 0.001 0.0449 0.0651 2 -0.001 to 0 0.0445 0.0617 3 0.001 to 0.005 0.1556 0.2222 4 -0.00 to -0.001 0.1487 0.2138 5 0.005 to 0.01 0.1600 0.1571 6 -0.01 to -0.005 0.1502 0.1455 7 0.01 to 0.02 0.1526 0.0705 8 -0.02 to -0.01 0.1124 0.0574 9 Greater than 0.02 0.0182 0.0043 10 Less than -0.02 0.0128 0.0022

[0094] Table 2 Results of standard surface 2

[0095]

[0096]

[0097] See also Figure 4 , Figure 4 This application provides another technical solution adopted by the present invention to solve the above technical problems. The technical solution is: to provide a three-dimensional color modeling system based on multi-sensor fusion, characterized in that it includes a laser acquisition module 41, a visual acquisition module 42, a correction module 43, a registration module 44 and a model building module 45, wherein:

[0098] Laser acquisition module 41: acquires three-dimensional point cloud information of the scene environment according to the laser radar, and sends the point cloud information to the correction module;

[0099] Visual acquisition module 42: acquires RGB images and visual posture information according to the visual sensor, sends the RGB images to the registration module, and sends the visual posture information to the correction module;

[0100] Correction module 43: adopts the extended Kalman filter method to use the received monocular vision posture information to correct the received radar posture and generate corrected point cloud information;

[0101] Registration module 44: Screening and processing the corrected point cloud information, removing outliers and noise points, and calculating the unit angle through statistical analysis principles, and projecting the corrected point cloud information to a specific plane; extracting appropriate feature points from the projected image and the RGB image, obtaining the corresponding relationship between the feature points between the two according to the RANSAC algorithm, minimizing the root mean square error, and registering the plane point cloud of the projected image with the pixel coordinate plane of the RGB image;

[0102] Model building module 45: Use the MRF model to fuse the point cloud and image data, generate texture information with color, and complete the modeling operation after image depth restoration.

[0103] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, this application will not go into details.

[0104] The above are only embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A three-dimensional color modeling method based on multi-sensor fusion, the steps of which are: Step 1: Build a multi-sensor detection platform consisting of laser radar and visual sensors; Step 2: Obtain three-dimensional point cloud information of the scene environment based on the laser radar; Step 3: Obtain RGB image and visual pose information based on the visual sensor; Step 4: Using the extended Kalman filter method, the monocular vision posture information is used to correct the radar posture and generate the corrected point cloud information; Step 5: Screen and process the corrected point cloud information, remove outliers and noise points, calculate the unit angle through statistical analysis principles, and project the corrected point cloud information onto a specific plane; Step 6: Extract appropriate feature points from the projection image and the RGB image, obtain the corresponding relationship between the feature points based on the RANSAC algorithm, minimize the root mean square error, and align the plane point cloud of the projection image with the pixel coordinate plane of the RGB image; Step 7: Use the MRF model to fuse the point cloud and image data, generate texture information with color, and complete the modeling operation after image depth restoration.

2. The three-dimensional color modeling method based on multi-sensor fusion according to claim 1, characterized in that: In the step 1, the laser radar and the visual sensor are fixedly connected by a mechanical structure to keep the focal length constant, thereby ensuring that the transformation equation between the point cloud coordinate system and the physical coordinate system of the visual sensor is constant and does not change due to the movement of the detection platform.

3. The three-dimensional color modeling method based on multi-sensor fusion according to claim 1, characterized in that: In step 3, the process of obtaining visual posture information is as follows: After basic processing of the image data, data tracking is completed through feature extraction and matching, and then the camera motion trajectory is estimated using the principle of epipolar geometry, and feature point depth recovery is achieved through triangulation.

4. The three-dimensional color modeling method based on multi-sensor fusion according to claim 3 is characterized in that: In step 3, the process of implementing feature point depth recovery by triangulation is as follows: 1) Down-sample the reconstructed point cloud data using the point cloud threshold constrained voxel centroid neighboring point algorithm to simplify the data while retaining the local features of the original data; 2) Smooth the simplified point cloud by statistical filtering and Gaussian filtering, remove the noise points of the point cloud, and obtain smooth data; 3) For any point in the point cloud, use the octree to search the neighborhood space of the point, quickly obtain the neighborhood point cloud of the point, and use the MLS algorithm to calculate the normal vector of each point cloud; 4) Project the point cloud onto a two-dimensional plane through the normal vector, triangulate the plane using the Delaunay space region growing algorithm, construct a point cloud sample triangle, and determine a growth edge. Take the point with the smallest cosine value of the angle between the growth edge and the projection point on the plane as the extension point, and expand the triangle from the inside to the outside until all the point cloud data are traversed to form a complete triangulated mesh.

5. The three-dimensional color modeling method based on multi-sensor fusion according to claim 1, characterized in that: In step 4, the process of using monocular vision posture information to correct radar posture is as follows: using laser posture as predicted value and visual posture as measured value, obtaining predicted value at moment k through input at moment k-1, and then correcting predicted value with observed value at moment k, estimating optimal state at moment k, continuously performing iterative calculation, and deducing state and error at current moment according to state at previous moment, and finally obtaining fusion posture of laser and vision.

6. The three-dimensional color modeling method based on multi-sensor fusion according to claim 1, characterized in that: In step 5, the process of screening the corrected point cloud information is as follows: 1) Perform voxel filtering on cloud points without changing the geometric shape of the point cloud model to reduce the number of point clouds, and then use statistical filtering to remove sparsely distributed point clouds that are far away from surrounding data points and have a small number; 2) Construct a grid map, set the height difference threshold of the point cloud in each grid in the vertical direction of the ground for screening, and then obtain the restricted range of the extracted ground points to achieve pre-screening of the ground point cloud; 3) Use the RANSAC algorithm to perform plane fitting on the pre-screened point cloud set to achieve precise screening of ground points.

7. The three-dimensional color modeling method based on multi-sensor fusion according to claim 1, characterized in that: In the step five, a one-to-one mapping method is used to project the corrected 3D point cloud into the 2D pixel space.

8. The three-dimensional color modeling method based on multi-sensor fusion according to claim 1, characterized in that: In step six, the process of aligning the planar point cloud of the projection image with the pixel coordinate plane of the RGB image is as follows: combining the intrinsic parameter matrix of the visual sensor, using the checkerboard joint calibration method to calibrate the lidar and the visual sensor, according to the same-name feature point pairs between the projection image and the RGB image, through the coordinate transformation matrix obtained by the initial calibration, the point cloud data collected from the same perspective is associated with the corresponding RGB values.

9. The three-dimensional color modeling method based on multi-sensor fusion according to claim 1, characterized in that: In the step 7, bilinear interpolation is performed on the normalized constants in the MRF model to set variable values, and the model parameters are estimated and updated by the maximum expectation algorithm.

10. A three-dimensional color modeling system based on multi-sensor fusion, characterized in that: It includes laser acquisition module, visual acquisition module, correction module, registration module and model building module, among which: Laser acquisition module: obtains the three-dimensional point cloud information of the scene environment based on the laser radar, and sends the point cloud information to the correction module; Visual acquisition module: obtains RGB images and visual posture information based on the visual sensor, sends the RGB images to the registration module, and sends the visual posture information to the correction module; Correction module: The extended Kalman filter method is used to correct the received radar posture using the received monocular vision posture information, and generate the corrected point cloud information; Registration module: Screen and process the corrected point cloud information, remove outliers and noise points, calculate the unit angle through statistical analysis principles, and project the corrected point cloud information to a specific plane; extract appropriate feature points from the projected image and the RGB image, obtain the corresponding relationship between the feature points between the two based on the RANSAC algorithm, minimize the root mean square error, and register the plane point cloud of the projected image with the pixel coordinate plane of the RGB image; Model building module: Use the MRF model to fuse point cloud and image data, generate texture information with color, and complete the modeling operation after image depth restoration.

Images