Multi-source data high-precision fusion method based on deep learning

Abstract

The invention discloses a multi-source data high-precision fusion method based on deep learning. According to the invention, the step of epipolar line calculation can ensure the geometric consistency of the point cloud data. An epipolar line is a straight line connecting one point in a three-dimensional space and a corresponding projection point of the epipolar line on an image, so that accurate geometric constraint is provided for subsequent point cloud processing by correct calculation of the epipolar line. The calculation of epipolar line parameters provides directional information for denoising and smoothing of the point cloud. In the de-noising process, filtering can be carried out along the epipolar line direction, so that abnormal points can be identified and removed more effectively, and meanwhile, useful structure information is reserved. By calculating the epipolar line in advance, the calculation amount in the subsequent processing step can be reduced. In the point cloud registration and fusion process, epipolar line information can be utilized to quickly position corresponding points, so that the convergence speed of the algorithm is accelerated. Therefore, more accurate basic data is provided for subsequent three-dimensional reconstruction.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-source data fusion, and specifically is a multi-source data high-precision fusion method based on deep learning. Background Art

[0002] With the rapid development of computer vision, machine learning, and deep learning, 3D modeling technology has been widely used in virtual reality (VR), augmented reality (AR), and game development. Traditional 3D modeling methods usually rely on manual creation or acquisition of point cloud data through scanning equipment, and then manual processing to build a 3D model. However, this method has problems such as low efficiency, high cost, and difficulty in large-scale scene modeling. In recent years, researchers have begun to explore automated 3D modeling technology using multi-source data. These multi-source data include photo sequences from different perspectives, lidar point clouds, and other sensor data. By effectively fusing these data sources, the accuracy and efficiency of 3D modeling can be significantly improved.

[0003] However, some existing technologies only consider a single data source, such as using only photo sequences or only using LiDAR point clouds for modeling. This single-source data processing method often fails to fully utilize the information carried by other types of data, resulting in the final generated 3D model being inaccurate. Summary of the invention

[0004] The purpose of the present invention is to provide a high-precision fusion method for multi-source data based on deep learning in order to solve the above-mentioned problems.

[0005] The technical solution adopted by the present invention is as follows: a high-precision fusion method of multi-source data based on deep learning, the method comprising the following steps:

[0006] S1: Input sequence images, collect and process the raw data of multiple sensors, including laser point cloud and camera images, to form a series of image data;

[0007] S2: performing relative orientation element and feature point calculation, for each sensor acquired data, respectively performing relative orientation element calculation and feature point extraction to establish a preliminary spatial geometric relationship;

[0008] S3: perform epipolar line calculation, and calculate the parameters of epipolar line according to the relative orientation elements for subsequent point cloud denoising and smoothing processing;

[0009] S4: performing point cloud denoising and smoothing, denoising and smoothing the laser point cloud data to improve the accuracy of the data;

[0010] S5: performing point cloud smoothing, further smoothing the point cloud data to reduce noise interference;

[0011] S6: coordinate conversion is performed on the data of different sensors so that they are in the same coordinate system to facilitate subsequent fusion processing;

[0012] S7: Combining the point cloud fusion processing algorithm of the improved SAC-IA algorithm and the improved ICP algorithm, the improved random sampling consistency iterative closest point algorithm and the improved iterative closest point algorithm are used to fuse the point cloud data to further improve the accuracy and integrity of the data;

[0013] S8: Use the R-tree structure to index and organize the point cloud data to facilitate rapid search and fusion of adjacent point cloud data;

[0014] S9: Construct a 3D model of the object by performing geometric modeling on the fused point cloud data

[0015] S10: Using the texture information in the camera image, texture mapping is performed on the three-dimensional model to make the model more realistic;

[0016] S11: After completing all data processing and fusion, the final complete and high-precision 3D model is output.

[0017] In a preferred embodiment, in step S1, the drone is equipped with a high-precision lidar and a high-resolution RGB camera; the input sequence image data includes point cloud data scanned by the lidar at a rate of 100,000 points per second, and 1920x1080 resolution JPEG images captured by the camera at a rate of 30 frames per second; each data file contains timestamp information to ensure time synchronization; the lidar data is stored in LAS format, and the camera image is stored in JPEG format containing EXIF ​​metadata.

[0018] In a preferred embodiment, in step S2, the ORB-SLAM3 algorithm is used to solve the relative orientation elements of the camera; the specific parameters include: the FAST threshold of the ORB algorithm used in feature point extraction is set to 20, the pyramid level is 8 layers, and the size of the feature point descriptor is 32 bytes; when solving the relative orientation elements, the five-point algorithm is used to estimate the camera pose, and the number of RANSAC iterations is set to 1000 times, and the internal point threshold is set to 0.005.

[0019] In a preferred embodiment, in step S3, the calculation process includes:

[0020] S3-1: Camera parameter acquisition: First, you need to obtain the camera's intrinsic parameter matrix, which includes focal length, principal point coordinates, and possible distortion coefficients;

[0021] S3-2 Image pair registration: Next, it is necessary to determine the relative orientation elements between two or more camera images, which are the rotation matrix and translation vector obtained by feature matching and geometric transformation;

[0022] S3-3: Calculate epipolar lines: Using the above parameters, calculate epipolar lines for each camera image;

[0023] The collinear equation is as follows:

[0024] Where f is the focal length; (c_x, c_y) are the principal point coordinates; R = [r_1, r_2, ..., r_9] is the rotation matrix; t = [t_x, t_y, t_z] is the translation vector; X, Y, Z are the coordinates of the spatial point; x, y are the pixel coordinates on the image.

[0025] In a preferred embodiment, in step S4, denoising is performed using the StatisticalOutlierRemoval filter in the PCL library; the specific parameters are set as follows: the number of neighbor points is set to 50, and the standard deviation multiple is set to 1.0; then, downsampling and smoothing are performed using the VoxelGrid filter, and the voxel size is set to 0.01 meters.

[0026] In a preferred embodiment, in step S5, the MovingLeastSquares filter in the PCL library is used to smooth the point cloud; specific parameters include: setting the search radius to 0.03 meters to determine the size of the neighborhood affecting each point; and setting the order of the polynomial fitting to 2 to ensure that the smoothing process does not excessively change the original shape of the point cloud.

[0027] In a preferred embodiment, in step S6, the calculation process includes:

[0028] S6-1: Obtain sensor data: First, collect data from different sensors, such as point cloud data from lidar and image data from cameras;

[0029] S6-2: Determine the sensor coordinate system: Identify the original coordinate system of each sensor data;

[0030] S6-3: Sensor calibration: The internal and external parameters of each sensor are obtained through the calibration process; the internal parameters include focal length, principal point coordinates and distortion coefficient, and the external parameters include rotation matrix and translation vector;

[0031] S6-14 Coordinate transformation: Use external parameters to transform the data of each sensor into a unified reference coordinate system;

[0032] The coordinate transformation formula is:

[0033] Where: (X_{ref}, Y_{ref}, Z_{ref}) are the coordinates of the point converted to the reference coordinate system; (Xsensor, Ysensor, Zsensor) are the coordinates of the point in the original sensor coordinate system; R is the rotation matrix, which is used to describe the rotation from the sensor coordinate system to the reference coordinate system; t is the translation vector, which is used to describe the translation from the sensor coordinate system to the reference coordinate system.

[0034] In a preferred embodiment, in step S7, an improved SAC-IA algorithm is used for initial registration; parameter settings include: setting the sampling ratio to 0.05 and the distance threshold to 0.1 meters; then, an improved ICP algorithm is applied for fine registration, and the parameters include setting the maximum number of iterations to 100 and the convergence threshold to 0.001 meters;

[0035] In step S8, an R-tree is used to index the point cloud data; specific parameters include: the maximum number of entries for each R-tree node is set to 10, and the minimum number of entries for each node is set to 5; this ensures that the point cloud data is effectively organized in space and facilitates rapid retrieval.

[0036] In a preferred embodiment, in step S9, a Poisson reconstruction algorithm is used for geometric modeling; parameter settings include: the grid resolution is set to 0.005 meters to ensure that the model has a high accuracy; the neighborhood size used in normal estimation is set to 0.01 meters.

[0037] In a preferred embodiment, in step S10, the MVS texture mapping technology is used; the specific parameters include: setting the minimum parallax angle of texture mapping to 5 degrees and the maximum parallax angle to 75 degrees to ensure the quality of texture mapping; the resolution of the texture image is set to 1024x1024 pixels;

[0038] In the step S11, the three-dimensional model is exported to the OBJ format; the parameter settings include: the compression ratio of the model mesh is set to 95% to reduce the file size; the compression format of the texture mapping uses JPEG, and the compression quality is set to 90%; the exported model contains vertices, texture coordinates and face information, ensuring the integrity and availability of the model in subsequent applications.

[0039] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0040] 1. In the present invention, the step of performing epipolar calculation can ensure the geometric consistency of point cloud data. The epipolar line is a straight line connecting a point in three-dimensional space and its corresponding projection point on the image. Therefore, the correct calculation of the epipolar line provides accurate geometric constraints for subsequent point cloud processing. The calculation of the epipolar line parameters provides directional information for point cloud denoising and smoothing. In the denoising process, filtering can be performed along the epipolar line direction, which can more effectively identify and remove abnormal points while retaining useful structural information. By pre-calculating the epipolar line, the amount of calculation in subsequent processing steps can be reduced. In the process of point cloud registration and fusion, the epipolar line information can be used to quickly locate corresponding points, thereby accelerating the convergence speed of the algorithm. The epipolar line calculation helps to improve the overall quality of the data. By performing data analysis and processing in the epipolar line direction, the intrinsic structure and characteristics of the data can be better understood, thereby providing more accurate basic data for subsequent three-dimensional reconstruction. The epipolar line calculation provides additional geometric constraints for the algorithm, which helps to improve the robustness of the entire fusion method. Even in the presence of noise and errors, the algorithm can perform effective data correction and optimization through epipolar line information.

[0041] 2. In the present invention, by performing coordinate conversion on the data of different sensors, all sensor data are placed in the same coordinate system, which is the prerequisite for effective data fusion. The unified coordinate system ensures the spatial alignment between different data sources and provides a basis for subsequent fusion processing. The accuracy of coordinate conversion directly affects the accuracy of the fusion result. Through accurate coordinate conversion, the error caused by inconsistent coordinate systems can be minimized, thereby improving the accuracy of the final three-dimensional model. Coordinate conversion allows data from different sensors to interoperate under the same framework, enhancing the flexibility of the data processing process. This interoperability is particularly important for complex data fusion tasks. Once all data are in the same coordinate system, the subsequent data processing process will be greatly simplified. Steps such as registration, fusion and reconstruction can apply unified methods and algorithms without the need to adjust them separately for each sensor. Coordinate conversion promotes the integration of different sensor data, allowing the advantages of each data source to complement each other. LiDAR data can provide accurate geometric information, while camera images can provide rich texture information. The effective combination of the two depends on accurate coordinate conversion. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0043] Example:

[0044] A high-precision fusion method for multi-source data based on deep learning, the method comprising the following steps:

[0045] S1: Input sequence images, collect and process the raw data of multiple sensors, including laser point cloud, camera images, etc., to form a series of image data;

[0046] S2: performing relative orientation element and feature point calculation, for each sensor acquired data, respectively performing relative orientation element calculation and feature point extraction to establish a preliminary spatial geometric relationship;

[0047] S3: perform epipolar line calculation, and calculate the parameters of epipolar line according to the relative orientation elements for subsequent point cloud denoising and smoothing processing;

[0048] S4: performing point cloud denoising and smoothing, denoising and smoothing the laser point cloud data to improve the accuracy of the data;

[0049] S5: performing point cloud smoothing, further smoothing the point cloud data to reduce noise interference;

[0050] S6: coordinate conversion is performed on the data of different sensors so that they are in the same coordinate system to facilitate subsequent fusion processing;

[0051] S7: Combining the point cloud fusion processing algorithm of the improved SAC-IA algorithm and the improved ICP algorithm, the improved random sampling consensus (SAC) iterative closest point (IA) algorithm and the improved iterative closest point (ICP) algorithm are used to fuse the point cloud data to further improve the accuracy and integrity of the data;

[0052] S8: Use the R-tree structure to index and organize the point cloud data to facilitate rapid search and fusion of adjacent point cloud data;

[0053] S9: Construct a 3D model of the object by performing geometric modeling on the fused point cloud data

[0054] S10: Using the texture information in the camera image, texture mapping is performed on the three-dimensional model to make the model more realistic;

[0055] S11: After completing all data processing and fusion, the final complete and high-precision 3D model is output.

[0056] In step S1, the drone is equipped with a high-precision LiDAR and a high-resolution RGB camera. The input sequence image data includes point cloud data scanned by the LiDAR at a rate of 100,000 points per second and 1920x1080 resolution JPEG images captured by the camera at a rate of 30 frames per second. Each data file contains timestamp information to ensure time synchronization. The LiDAR data is stored in LAS format, while the camera images are stored in JPEG format with EXIF ​​metadata.

[0057] In step S2, the ORB-SLAM3 algorithm is used to solve the relative orientation elements of the camera. The specific parameters include: the FAST threshold of the ORB algorithm used in feature point extraction is set to 20, the pyramid level is 8 layers, and the size of the feature point descriptor is 32 bytes. When solving the relative orientation elements, the five-point algorithm is used to estimate the camera pose, and the number of RANSAC iterations is set to 1000 times, and the inlier threshold is set to 0.005.

[0058] In step S3, the calculation process includes:

[0059] S3-1: Camera parameter acquisition: First, you need to obtain the camera's intrinsic parameter matrix, which usually includes the focal length (f), principal point coordinates (c_x, c_y) and possible distortion coefficients.

[0060] S3-2 Image pair registration: Next, the relative orientation elements between two or more camera images need to be determined, which are usually the rotation matrix (R) and translation vector (t) obtained through feature matching and geometric transformation.

[0061] S3-3: Calculate epipolar lines: Using the above parameters, epipolar lines can be calculated for each camera image;

[0062] The collinear equation is as follows:

[0063] Where f is the focal length; (c_x, c_y) are the principal point coordinates; R = [r_1, r_2, ..., r_9] is the rotation matrix; t = [t_x, t_y, t_z] is the translation vector; X, Y, Z are the coordinates of the spatial point; x, y are the pixel coordinates on the image;

[0064] In step S4, the StatisticalOutlierRemoval filter in the PCL library is used for denoising. The specific parameters are set as follows: the number of neighbor points is set to 50, and the standard deviation multiple is set to 1.0. Then, the VoxelGrid filter is used for downsampling and smoothing, and the voxel size is set to 0.01 meters.

[0065] In step S5, the MovingLeastSquares filter in the PCL library is used to smooth the point cloud. The specific parameters include: setting the search radius to 0.03 meters to determine the size of the neighborhood that affects each point; and setting the order of the polynomial fitting to 2 to ensure that the smoothing process does not excessively change the original shape of the point cloud.

[0066] In step S6, the calculation process includes:

[0067] S6-1: Obtain sensor data: First, collect data from different sensors, such as point cloud data from lidar and image data from cameras.

[0068] S6-2: Determine sensor coordinate system: Identify the original coordinate system of each sensor data.

[0069] S6-3: Sensor calibration: The internal and external parameters of each sensor are obtained through the calibration process. The internal parameters include focal length, principal point coordinates and distortion coefficients, and the external parameters include rotation matrix (R) and translation vector (t).

[0070] S6-14 Coordinate transformation: Use external parameters to transform the data of each sensor into a unified reference coordinate system;

[0071] The coordinate transformation formula is:

[0072] Where: (X_{ref}, Y_{ref}, Z_{ref}) are the coordinates of the point converted to the reference coordinate system; (Xsensor, Ysensor, Zsensor) are the coordinates of the point in the original sensor coordinate system. R is the rotation matrix, which is used to describe the rotation from the sensor coordinate system to the reference coordinate system. t is the translation vector, which is used to describe the translation from the sensor coordinate system to the reference coordinate system.

[0073] In step S7, the improved SAC-IA algorithm is used for initial registration. The parameter settings include: setting the sampling ratio to 0.05 and the distance threshold to 0.1 meters. Then, the improved ICP algorithm is applied for fine registration, and the parameters include setting the maximum number of iterations to 100 and the convergence threshold to 0.001 meters;

[0074] In step S8, an R-tree is used to index the point cloud data. The specific parameters include: the maximum number of entries per R-tree node is set to 10, and the minimum number of entries per node is set to 5. This ensures that the point cloud data is effectively organized in space and is easy to retrieve quickly.

[0075] In step S9, a Poisson reconstruction algorithm is used for geometric modeling. Parameter settings include: the grid resolution is set to 0.005 meters to ensure that the model has high accuracy; the neighborhood size used in normal estimation is set to 0.01 meters.

[0076] In step S10, the MVS texture mapping technology is used. Specific parameters include: setting the minimum parallax angle of texture mapping to 5 degrees and the maximum parallax angle to 75 degrees to ensure the quality of texture mapping; and setting the resolution of the texture image to 1024x1024 pixels.

[0077] In step S11, the 3D model is exported to OBJ format. The parameter settings include: the compression ratio of the model mesh is set to 95% to reduce the file size; the compression format of the texture map uses JPEG, and the compression quality is set to 90%. The exported model contains vertex, texture coordinate and face information, ensuring the integrity and availability of the model in subsequent applications.

[0078] In the present invention, the step of performing epipolar line calculation can ensure the geometric consistency of point cloud data. The epipolar line is a straight line connecting a point in three-dimensional space and its corresponding projection point on the image. Therefore, the correct calculation of the epipolar line provides accurate geometric constraints for subsequent point cloud processing. The calculation of the epipolar line parameters provides directional information for point cloud denoising and smoothing. In the denoising process, filtering can be performed along the epipolar line direction, which can more effectively identify and remove abnormal points while retaining useful structural information. By pre-calculating the epipolar line, the amount of calculation in subsequent processing steps can be reduced. In the process of point cloud registration and fusion, the epipolar line information can be used to quickly locate corresponding points, thereby accelerating the convergence speed of the algorithm. The epipolar line calculation helps to improve the overall quality of the data. By performing data analysis and processing in the epipolar line direction, the intrinsic structure and characteristics of the data can be better understood, thereby providing more accurate basic data for subsequent three-dimensional reconstruction. The epipolar line calculation provides additional geometric constraints for the algorithm, which helps to improve the robustness of the entire fusion method. Even in the presence of noise and errors, the algorithm can perform effective data correction and optimization through the epipolar line information.

[0079] In the present invention, by performing coordinate transformation on the data of different sensors, all sensor data are placed in the same coordinate system, which is the prerequisite for effective data fusion. The unified coordinate system ensures the spatial alignment between different data sources and provides a basis for subsequent fusion processing. The accuracy of coordinate transformation directly affects the accuracy of the fusion result. Through accurate coordinate transformation, the error caused by inconsistent coordinate systems can be minimized, thereby improving the accuracy of the final three-dimensional model. Coordinate transformation allows data from different sensors to interoperate under the same framework, enhancing the flexibility of the data processing process. This interoperability is particularly important for complex data fusion tasks. Once all data are in the same coordinate system, the subsequent data processing process will be greatly simplified. Steps such as registration, fusion and reconstruction can apply unified methods and algorithms without the need to adjust them separately for each sensor. Coordinate transformation promotes the integration of different sensor data, allowing the advantages of each data source to complement each other. LiDAR data can provide accurate geometric information, while camera images can provide rich texture information. The effective combination of the two depends on accurate coordinate transformation.

[0080] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0081] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-precision fusion method for multi-source data based on deep learning, characterized by: The method comprises the following steps: S1: Input sequence images, collect and process the raw data of multiple sensors, including laser point cloud and camera images, to form a series of image data; S2: performing relative orientation element and feature point calculation, for each sensor acquired data, respectively performing relative orientation element calculation and feature point extraction to establish a preliminary spatial geometric relationship; S3: perform epipolar line calculation, and calculate the parameters of epipolar line according to the relative orientation elements for subsequent point cloud denoising and smoothing processing; S4: performing point cloud denoising and smoothing, denoising and smoothing the laser point cloud data to improve the accuracy of the data; S5: performing point cloud smoothing, further smoothing the point cloud data to reduce noise interference; S6: coordinate conversion is performed on the data of different sensors so that they are in the same coordinate system to facilitate subsequent fusion processing; S7: Combining the point cloud fusion processing algorithm of the improved SAC-IA algorithm and the improved ICP algorithm, the improved random sampling consistency iterative closest point algorithm and the improved iterative closest point algorithm are used to fuse the point cloud data to further improve the accuracy and integrity of the data; S8: Use the R-tree structure to index and organize the point cloud data to facilitate rapid search and fusion of adjacent point cloud data; S9: Construct a 3D model of the object by performing geometric modeling on the fused point cloud data S10: Using the texture information in the camera image, texture mapping is performed on the three-dimensional model to make the model more realistic; S11: After completing all data processing and fusion, the final complete and high-precision 3D model is output.

2. The high-precision fusion method of multi-source data based on deep learning as claimed in claim 1, characterized in that: In step S1, the drone is equipped with a high-precision laser radar and a high-resolution RGB camera; the input sequence image data includes point cloud data scanned by the laser radar at a rate of 100,000 points per second, and 1920x1080 resolution JPEG images captured by the camera at a rate of 30 frames per second; each data file contains timestamp information to ensure time synchronization; the laser radar data is stored in LAS format, and the camera image is stored in JPEG format containing EXIF ​​metadata.

3. The high-precision fusion method of multi-source data based on deep learning as claimed in claim 1, characterized in that: In step S2, the ORB-SLAM3 algorithm is used to solve the relative orientation elements of the camera; the specific parameters include: the FAST threshold of the ORB algorithm used in feature point extraction is set to 20, the pyramid level is 8 layers, and the size of the feature point descriptor is 32 bytes; when solving the relative orientation elements, the five-point algorithm is used to estimate the camera pose, and the number of RANSAC iterations is set to 1000 times, and the internal point threshold is set to 0.

005.

4. The high-precision fusion method of multi-source data based on deep learning as claimed in claim 1, characterized in that: In step S3, the calculation process includes: S3-1: Camera parameter acquisition: First, you need to obtain the camera's intrinsic parameter matrix, which includes focal length, principal point coordinates, and possible distortion coefficients; S3-2 Image pair registration: Next, it is necessary to determine the relative orientation elements between two or more camera images, which are the rotation matrix and translation vector obtained by feature matching and geometric transformation; S3-3: Calculate epipolar lines: Using the above parameters, calculate epipolar lines for each camera image; The collinear equation is as follows: Where f is the focal length; (c_x, c_y) are the principal point coordinates; R = [r_1, r_2, ..., r_9] is the rotation matrix; t = [t_x, t_y, t_z] is the translation vector; X, Y, Z are the coordinates of the spatial point; x, y are the pixel coordinates on the image.

5. The high-precision fusion method of multi-source data based on deep learning as claimed in claim 1, characterized in that: In step S4, the StatisticalOutlierRemoval filter in the PCL library is used for denoising; the specific parameters are set as follows: the number of neighbor points is set to 50, and the standard deviation multiple is set to 1.0; then, the VoxelGrid filter is used for downsampling and smoothing, and the voxel size is set to 0.01 meters.

6. The high-precision fusion method of multi-source data based on deep learning as claimed in claim 1, characterized in that: In step S5, the MovingLeastSquares filter in the PCL library is used to smooth the point cloud; the specific parameters include: setting the search radius to 0.03 meters to determine the size of the neighborhood that affects each point; and setting the order of the polynomial fitting to 2 to ensure that the smoothing process does not excessively change the original shape of the point cloud.

7. The high-precision fusion method of multi-source data based on deep learning as claimed in claim 1, characterized in that: In step S6, the calculation process includes: S6-1: Obtain sensor data: First, collect data from different sensors, such as point cloud data from lidar and image data from cameras; S6-2: Determine the sensor coordinate system: Identify the original coordinate system of each sensor data; S6-3: Sensor calibration: The internal and external parameters of each sensor are obtained through the calibration process; the internal parameters include focal length, principal point coordinates and distortion coefficient, and the external parameters include rotation matrix and translation vector; S6-14 Coordinate transformation: Use external parameters to transform the data of each sensor into a unified reference coordinate system; The coordinate transformation formula is: Where: (X_{ref}, Y_{ref}, Z_{ref}) are the coordinates of the point converted to the reference coordinate system; (Xsensor, Ysensor, Zsensor) are the coordinates of the point in the original sensor coordinate system; R is the rotation matrix, which is used to describe the rotation from the sensor coordinate system to the reference coordinate system; t is the translation vector, which is used to describe the translation from the sensor coordinate system to the reference coordinate system.

8. The high-precision fusion method of multi-source data based on deep learning as claimed in claim 1, characterized in that: In step S7, an improved SAC-IA algorithm is used for initial registration; The parameter settings include: setting the sampling ratio to 0.05 and the distance threshold to 0.1 m; then, applying the improved ICP algorithm for fine registration, with the parameters including the maximum number of iterations set to 100 and the convergence threshold set to 0.001 m; In step S8, an R-tree is used to index the point cloud data; specific parameters include: the maximum number of entries for each R-tree node is set to 10, and the minimum number of entries for each node is set to 5; this ensures that the point cloud data is effectively organized in space and facilitates rapid retrieval.

9. The high-precision fusion method of multi-source data based on deep learning as claimed in claim 1, characterized in that: In step S9, a Poisson reconstruction algorithm is used for geometric modeling; parameter settings include: the grid resolution is set to 0.005 meters to ensure that the model has a high accuracy; the neighborhood size used in normal estimation is set to 0.01 meters.

10. The high-precision fusion method of multi-source data based on deep learning according to claim 1, characterized in that: In the step S10, the MVS texture mapping technology is used; the specific parameters include: setting the minimum parallax angle of the texture mapping to 5 degrees and the maximum parallax angle to 75 degrees to ensure the quality of the texture mapping; the resolution of the texture image is set to 1024x1024 pixels; In the step S11, the three-dimensional model is exported to the OBJ format; the parameter settings include: the compression ratio of the model mesh is set to 95% to reduce the file size; the compression format of the texture mapping uses JPEG, and the compression quality is set to 90%; the exported model contains vertices, texture coordinates and face information, ensuring the integrity and availability of the model in subsequent applications.

Images