A real scene three-dimensional modeling method and system fusing laser point cloud and image

Abstract

The application provides a kind of fusion laser point cloud and image real scene three-dimensional modeling method and system, belong to city three-dimensional modeling technical field.The method obtains the laser point cloud data and multi-view image data of target scene by synchronous acquisition equipment, and generates space-time synchronization parameters by joint calibration of laser radar and camera.Using dynamic weight registration algorithm, according to the point cloud density distribution and image texture complexity, the registration weight is dynamically adjusted, and the multi-level registration of laser point cloud and image data is realized.The registered data is used to construct three-dimensional geometric model and generate seamless texture map, and the missing area of the model is repaired by semantic driven hole, and finally the complete three-dimensional model is output.The application adopts the above-mentioned fusion laser point cloud and image real scene three-dimensional modeling method and system, solves the limitation of single data source modeling, realizes high-precision, high-integrity and real-time updating of real scene three-dimensional modeling.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of urban three-dimensional modeling, in particular to a real scene three-dimensional modeling method and system fusing laser point cloud and image. BACKGROUND

[0002] In the field of real scene three-dimensional modeling, laser radar and multi-view image are two core data sources, but the limitations of single data source make it difficult to achieve high precision and high integrity of modeling effect. Laser point cloud can provide millimeter-level geometric information, but it lacks surface texture details, resulting in a single visual presentation of the reconstructed model, and in complex scenes, data loss or noise may occur due to occlusion or multiple reflections. While multi-view images can capture rich texture and color information, their geometric reconstruction accuracy depends on stereo matching algorithms, which are easily affected by changes in lighting and weak texture areas. In dynamic scenes, there are also motion blur or ghosting phenomena caused by time synchronization deviation. In addition, multi-source data fusion faces many challenges, such as traditional registration methods relying on geometric feature alignment, not fully considering the complementarity of point cloud density and image texture, resulting in significant registration errors in occluded areas; existing fusion algorithms rely on geometric interpolation in data missing areas, lack semantic constraints, and the repair results often have geometric distortion or texture breakage; large-scale point cloud and image fusion calculations usually use offline batch processing mode, making it difficult to meet the real-time modeling needs of autonomous driving, robot navigation and other scenarios.

[0003] The prior art has deficiencies in data complementarity mining, semantic information application, and real-time assurance. For example, three-dimensional reconstruction methods based on laser radar can generate high-precision geometric models, but data loss problems still exist in occluded areas and complex scenes. Three-dimensional reconstruction methods based on images have large depth estimation errors in weak texture areas, and the calculation time increases significantly with the growth of data volume. In terms of multi-source data fusion, although existing methods attempt to register through calibration parameters and feature matching, the spatio-temporal synchronization problem in dynamic scenes has not been effectively solved, and the lack of semantic information guidance leads to deviations in the repair results from the actual scene. SUMMARY

[0004] The purpose of the present application is to provide a real scene three-dimensional modeling method and system fusing laser point cloud and image, which solves the limitations of single data source modeling and realizes high-precision, high-integrity, and real-time updating of real scene three-dimensional modeling.

[0005] To achieve the above purpose, the present application provides a real scene three-dimensional modeling method fusing laser point cloud and image, comprising the following steps:

[0006] acquiring laser point cloud data and multi-view image data of the target scene through a synchronous acquisition device;

[0007] jointly calibrating the lidar and the camera to generate spatiotemporal synchronization parameters;

[0008] based on a dynamic weight registration algorithm, using the spatiotemporal synchronization parameters generated by the joint calibration, performing multi-level registration of the laser point cloud and the image data, wherein the registration weight is dynamically adjusted according to the point cloud density distribution and the image texture complexity;

[0009] fusing the registered data to construct a three-dimensional geometric model and generate a seamless texture map;

[0010] performing semantic-driven hole filling on the missing areas of the constructed three-dimensional geometric model to output a complete three-dimensional model.

[0011] Preferably, the joint calibration of the lidar and the camera to generate spatiotemporal synchronization parameters includes:

[0012] synchronously acquiring the spatial pose relationship of the lidar and the camera using a checkerboard calibration board;

[0013] controlling the exposure time of the lidar and the camera through a time synchronization module, with a time error less than 0.1 ms;

[0014] storing the calibration results as spatiotemporal synchronization parameters, including a spatial transformation matrix and a timestamp offset.

[0015] Preferably, the dynamic weight registration algorithm includes:

[0016] in the coarse registration stage, projecting the point cloud to the image coordinate system using the spatiotemporal synchronization parameters obtained by the joint calibration;

[0017] in the fine registration stage, dividing high-confidence areas and low-confidence areas according to the superimposed results of the generated point cloud density distribution map and the image texture complexity heat map;

[0018] in the low-confidence areas, preferentially relying on laser point cloud data registration, and in the high-confidence areas, preferentially relying on image feature matching, iteratively updating the registration parameters through a nonlinear optimization algorithm to obtain the registered point cloud and image data.

[0019] Preferably, the generation method of the point cloud density distribution map and the image texture complexity heat map is:

[0020] using a point cloud density calculation module to count the number of local point clouds in units of voxels to generate a density distribution map;

[0021] using an image texture analysis module to calculate the gradient amplitude variance of each pixel point to generate a texture complexity heat map;

[0022] superimposing the generated density distribution map and texture complexity heat map after spatial alignment to generate a dynamic weight distribution atlas.

[0023] Preferably, the construction of the three-dimensional geometric model comprises:

[0024] Based on the registered laser point cloud data, an initial mesh model is generated by a Poisson reconstruction algorithm;

[0025] Based on the registered multi-view image data, a dense point cloud is generated by a stereo matching algorithm to supplement the missing areas of the laser point cloud;

[0026] When fusing the two types of point clouds, curvature consistency detection is performed on the overlapping area, and after removing outliers, an optimized geometric model is generated.

[0027] Preferably, the curvature consistency detection comprises:

[0028] The difference in local curvature between the laser point cloud and the image-generated point cloud is calculated;

[0029] If the difference exceeds a preset threshold, the area is determined to be an outlier and is removed;

[0030] The removed point cloud is subjected to mesh reconstruction to ensure that the surface of the geometric model is continuous and smooth, and an optimized geometric model is obtained.

[0031] Preferably, the generation of seamless texture mapping comprises:

[0032] In the texture mapping stage, according to the correspondence between the vertices of the obtained geometric model and the image pixels, the best visibility of the image view is selected;

[0033] In the texture fusion stage, the color difference between images is eliminated by a cross-modal feature alignment network, and a multi-resolution fusion algorithm is used to eliminate the seams;

[0034] In the texture optimization stage, the image texture is subjected to illumination consistency correction according to the reflectivity information of the laser point cloud, and the texture mapping of the complete three-dimensional model is obtained.

[0035] Preferably, the cross-modal feature alignment network is trained in the following way:

[0036] Input the reflectivity features of the laser point cloud and the RGB features of the image;

[0037] Cross-modal shared features are extracted through an attention mechanism;

[0038] The aligned texture mapping matrix is output, which is used in the texture mapping and fusion stages to minimize the spatial offset error between the vertices of the geometric model and the image features.

[0039] Preferably, the semantic-driven hole filling comprises:

[0040] In the semantic segmentation stage, the registered point cloud and image are cross-modal joint segmented to extract scene semantic labels;

[0041] In the completion stage, a local three-dimensional prior model is constructed according to the semantic labels, and a generative adversarial network is used to generate the geometry and texture of the missing area;

[0042] In the verification stage, the curvature and color distribution consistency of the completed area and the adjacent area are compared, and if they do not meet the requirements, the completed area is regenerated until a complete and accurate three-dimensional model is obtained.

[0043] Preferably, the method further comprises incremental model updating:

[0044] In a dynamic scene, newly added point cloud and image data are received in real time;

[0045] In the local registration stage, only the model area within the influence range of the newly added data is registered using a dynamic weight registration method;

[0046] In the model updating stage, the local geometry and texture are updated through GPU parallel computing to ensure that the overall model updating delay is less than 1 second, thereby maintaining the real-time and accuracy of the three-dimensional model.

[0047] The application also provides a real scene three-dimensional modeling system for implementing the above method, comprising:

[0048] Hardware unit: mobile acquisition equipment integrating laser radar, multi-view camera and synchronous trigger module, used for acquiring laser point cloud data and multi-view image data;

[0049] Dynamic registration unit: performing dynamic weight distribution and registration parameter optimization of point cloud density and image texture to provide accurate registration data for subsequent fusion modeling;

[0050] Fusion modeling unit: calling a cross-modal alignment network to realize collaborative generation of geometry and texture, and constructing a complete three-dimensional geometric model and texture map;

[0051] Semantic repair unit: realizing hole completion based on scene semantic labels and GAN network to ensure the integrity of the three-dimensional model.

[0052] Preferably, the system further comprises an incremental updating unit: realizing real-time updating of the model through a local computing pipeline to ensure the timeliness of the three-dimensional model in a dynamic scene.

[0053] Therefore, the application adopts the above-mentioned real scene three-dimensional modeling method and system for fusing laser point cloud and image, and has the following beneficial technical effects:

[0054] (1) High-precision data fusion: By jointly calibrating the laser radar and camera, accurate spatiotemporal synchronization parameters are generated, and a dynamic weight registration algorithm is used to dynamically adjust the registration weight according to the point cloud density and image texture complexity, effectively improving the registration accuracy of laser point cloud and image data, reducing the registration error in the occluded area, and making the geometric accuracy and texture details of the three-dimensional model more accurate.

[0055] (2) Model integrity enhancement: Introduce semantic-driven hole repair technology to intelligently complete the missing areas of the three-dimensional model. Cross-modal joint segmentation is used to extract scene semantic labels, and a generative adversarial network (GAN) is used to generate the geometry and texture of the missing area to ensure the semantic consistency of the repair result with the surrounding scene, significantly improving the integrity and authenticity of the model.

[0056] (3) Real-time and efficiency improvement: Design an incremental model update architecture to support local data update and GPU parallel acceleration. In dynamic scenes, only the model area affected by the newly added data is registered and updated, significantly reducing the overall model update delay, meeting the real-time requirements of applications such as autonomous driving and robot navigation.

[0057] (4) Robustness and adaptability enhancement: Through cross-modal feature alignment network and curvature consistency detection technology, the color difference and geometric offset error between images are effectively eliminated, improving the quality and consistency of texture mapping. At the same time, this method has strong adaptability to complex environments (such as vegetation-covered areas and weak texture areas) and dynamic scenes, and can generate stable and reliable three-dimensional models. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 is a flowchart of the real three-dimensional modeling method of fusing laser point cloud and image of the present application;

[0059] Figure 2 is a joint calibration flowchart;

[0060] Figure 3 is a dynamic weight registration algorithm flowchart;

[0061] Figure 4 is a three-dimensional geometric model construction flowchart;

[0062] Figure 5 is a seamless texture mapping generation flowchart;

[0063] Figure 6 is a semantic-driven hole repair flowchart. DETAILED DESCRIPTION

[0064] The technical solutions of the present application are further described below through the drawings and examples.

[0065] Unless otherwise defined, technical terms or scientific terms used in the present application shall have the ordinary meaning as understood by a person having ordinary skill in the art to which the present application pertains.

[0066] Embodiment one

[0067] As shown in the flow chart of a real scene three-dimensional modeling method fusing laser point cloud and image of the present application, comprising the following steps: Figure 1

[0068] Step S1, acquiring laser point cloud data and multi-view image data of the target scene through a synchronous acquisition device.

[0069] (1) Hardware configuration:

[0070] Laser radar: Velodyne VLP-16 is adopted, horizontal field of view 360°, vertical field of view 30°, scanning frequency 10Hz, point cloud density ≥ 160,000 points / s.

[0071] Multi-view camera: 6 Sony IMX477 global shutter cameras, resolution 20 million pixels, frame rate 30fps, triggered by FPGA hardware synchronization, time synchronization error <0.1ms.

[0072] Synchronous control unit: based on Xilinx Zynq-7000 FPGA to realize hardware level synchronization of laser radar and camera.

[0073] (2) Data preprocessing:

[0074] Point cloud denoising: DBSCAN algorithm based on density is adopted to remove outliers, the formula is

[0075] ;

[0076] Wherein, represents the number of points in the neighborhood of point , ;

[0077] Image enhancement: histogram equalization and radial distortion correction are performed on each frame of image, and the distortion correction model is:

[0078] ;

[0079] Wherein, represents the corrected normalized pixel horizontal coordinate, represents the original normalized pixel horizontal coordinate, represents the radial distance from the pixel point to the optical center, , represents the radial distortion coefficient,​​​ Indicates the radial distance from the pixel to the optical center. In this embodiment, , .

[0080] Step S2: Jointly calibrate the laser radar and camera to generate spatiotemporal synchronization parameters (such as Figure 2 shown).

[0081] (1) Place a checkerboard calibration plate (grid size 30 mm × 30 mm) in the common field of view of the lidar and camera, and collect 10 sets of point cloud and image data.

[0082] Calculate the transformation matrix from the lidar coordinate system to the camera coordinate system , by minimizing the reprojection error:

[0083] ;

[0084] in, express Homogeneous transformation matrix, including rotation and translation parameters, Represents the camera projection function, which maps three-dimensional points to a two-dimensional pixel plane. Represents the three-dimensional coordinates of the corner point in the laser radar coordinate system, Represents the pixel coordinates of the corner points in the camera coordinate system, Indicates the total number of corner points of the calibration plate, Indicates the index of the corner point of the calibration plate.

[0085] (2) Send synchronous pulse signals through FPGA to record the trigger timestamps of the lidar and camera and , calculate the time offset:

[0086] ;

[0087] in, Indicates the time synchronization error.

[0088] (3) Store the calibration results as spatiotemporal synchronization parameters, including the spatial transformation matrix and timestamp offset:

[0089] .

[0090] Step S3, based on dynamic weight registration algorithm (such as Figure 3 As shown in Figure 3), the spatiotemporal synchronization parameters generated by the joint calibration are used to perform multi-level registration of the laser point cloud and the image data, where the registration weights are dynamically adjusted according to the point cloud density distribution and the image texture complexity.

[0091] (1) In the coarse registration stage, the spatio-temporal synchronization parameters obtained by joint calibration are used to project the point cloud into the image coordinate system.

[0092] ;

[0093] wherein, represents the point projected into the camera coordinate system, represents the original laser point cloud.

[0094] (2) In the fine registration stage, according to the superimposed results of the generated point cloud density distribution map and the image texture complexity heat map, high confidence areas and low confidence areas are divided.

[0095] 2.1, point cloud density calculation: divide the space into voxels (edge length ), count the number of points in each voxel , and generate a density distribution map:

[0096] ;

[0097] wherein, represents the point cloud density at coordinate .

[0098] 2.2, image texture complexity calculation:

[0099] ;

[0100] ;

[0101] wherein, represents the gradient amplitude at pixel , , represents the gradient of the image in the horizontal and vertical directions, represents the texture complexity at pixel , represents the local window radius, and in this embodiment, .

[0102] 2.3, superimpose the density map and the texture heat map to generate a weight distribution.

[0103] (3) In the low confidence area, preferentially rely on laser point cloud data registration, and in the high confidence area, preferentially rely on image feature matching, and through a nonlinear optimization algorithm, iteratively update the registration parameters to obtain the registered point cloud and image data.

[0104] Step S4, fuse the registered data, construct a three-dimensional geometric model and generate a seamless texture map.

[0105] (1) as Figure 4As shown, the construction of the three-dimensional geometric model includes:

[0106] Based on the registered laser point cloud data, an initial mesh model is generated by a Poisson reconstruction algorithm;

[0107] ;

[0108] wherein, represents the Laplacian operator, represents the gradient operator, represents the point cloud normal vector field.

[0109] (2) Based on the registered multi-view image data, a dense point cloud is generated by a stereo matching algorithm , supplementing the missing area of the laser point cloud;

[0110] (3) When fusing the two types of point clouds, curvature consistency detection is performed on the overlapping area, and after removing outliers, an optimized geometric model is generated.

[0111] Curvature consistency detection includes:

[0112] Calculate the difference in local curvature between the laser point cloud and the image-generated point cloud;

[0113] ;

[0114] wherein, represents the local curvature difference, represents the local curvature of the laser point cloud, represents the local curvature of the MVS point cloud.

[0115] If the difference exceeds a preset threshold, the area is determined to be an outlier and is removed;

[0116] Grid reconstruction is performed on the removed point cloud to ensure that the surface of the geometric model is continuous and smooth, and an optimized geometric model is obtained.

[0117] (4) As shown, Figure 5 the generation of seamless texture mapping includes:

[0118] 4.1 In the texture mapping stage, according to the correspondence between the vertices of the obtained geometric model and the image pixels, the best visibility of the image view is selected;

[0119] 4.2 In the texture fusion stage, the color difference between images is eliminated by a cross-modal feature alignment network, and a multi-resolution fusion algorithm is used to eliminate the seams;

[0120] 4.2.1 The cross-modal feature alignment network is trained in the following way:

[0121] Input laser point cloud reflectivity features Image RGB features ;

[0122] Cross-modal shared features are extracted by attention mechanism.

[0123] The network structure includes:

[0124] Branch 1 (point cloud feature extraction): geometric features are extracted using PointNet ;

[0125] Branch 2 (image feature extraction): texture features are extracted using ResNet-18 .

[0126] The feature fusion layer aligns the geometric and texture features through attention mechanism.

[0127] The output is an aligned texture mapping matrix, which is used in the texture mapping and fusion stage to minimize the spatial offset error between the geometric model vertices and image features.

[0128] 4.2.2, construct Laplacian pyramid, hierarchical fusion of texture details of different scales:

[0129] Gaussian pyramid: generate low-resolution versions;

[0130] Laplacian pyramid: calculate high-frequency details;

[0131] Fusion rule: in each layer of the pyramid, mix multi-view textures according to the weight map output by the cross-modal network.

[0132] Poisson blending is performed on overlapping areas to minimize color gradient differences.

[0133] 4.3, in the texture optimization stage, the image texture is corrected for lighting consistency according to the reflectance information of the laser point cloud, obtaining the texture map of the complete three-dimensional model:

[0134] Establish a linear relationship between the reflectance of the point cloud and the brightness of the image:

[0135] ;

[0136] where, , are fitting parameters, is a noise term;

[0137] Adjust the image brightness according to the reflectance:

[0138] ;

[0139] where, ​represents the adjusted brightness, represents the original brightness, represents the scene average reflectance.

[0140] The histogram matching is used to unify the tone distribution of different perspectives.

[0141] Step S5, semantic-driven hole filling is performed on the missing area of the constructed three-dimensional geometric model, and a complete three-dimensional model is output.

[0142] As shown in Figure 6 The semantic-driven hole filling includes:

[0143] In the semantic segmentation stage, the registered point cloud and the image are cross-modality joint segmented, the scene semantic labels are extracted, the confidence score of semantic prediction is calculated for each point / pixel, the joint semantic confidence map is generated, and the high confidence area with a confidence score greater than 0.9 is screened.

[0144] In the completion stage, a local three-dimensional prior model is constructed according to the semantic labels, and a generative adversarial network is used to generate the geometry and texture of the missing area.

[0145] The generative adversarial network is a two-channel generative adversarial network, including a generator and a discriminator.

[0146] The generator: input the semantic label and the binary mask of the hole area, and the structure is divided into two parts. One is the geometry generation channel, which generates the completed point cloud from the input semantic label through the 3D convolution layer; the other is the texture generation channel, which generates the texture map from the semantic label through the 2D transpose convolution layer. The output is the completed point cloud and the texture.

[0147] The discriminator: input real data or generated data. Structurally, the geometry discrimination branch is based on PointNet to judge the authenticity of the point cloud; the texture discrimination branch is based on CNN to judge the authenticity of the texture.

[0148] Semantic consistency constraint.

[0149] Loss function design:

[0150] Adversarial loss: composed of two parts. One part is when the real data is input into the discriminator, the logarithmic loss of its output and the real label is calculated; the other part is when the generated data is input into the discriminator, the logarithmic loss of its output and the false label is calculated. The purpose is to make the completed data generated by the generator closer to the real data and deceive the discriminator.

[0151] Geometric reconstruction loss: calculate the mean square error between the completed point cloud and the real point cloud, which is used to measure the geometric shape difference between the completed point cloud and the real point cloud, so that the geometric shape of the completed point cloud is closer to the real situation.

[0152] Semantic consistency loss: calculate the cross-entropy loss between the true semantic label and the predicted semantic label to ensure that the semantics of the completed area are consistent with the original scene semantics.

[0153] Total loss: add the adversarial loss, geometric reconstruction loss, and semantic consistency loss after multiplying them by the corresponding weight coefficients.

[0154] The final output is the preliminary completed point cloud and texture.

[0155] In the verification phase, compare the curvature and color distribution consistency of the completed area and the adjacent area. If it does not meet the requirements, it will be regenerated until a complete and accurate three-dimensional model is obtained.

[0156] The method also includes incremental model updating:

[0157] In a dynamic scene, newly added point cloud and image data are received in real time;

[0158] In the local registration stage, only the model area within the influence range of the new data is registered using a dynamic weight registration method.

[0159] In the model updating stage, the local geometry and texture are updated through GPU parallel computing, ensuring that the overall model updating delay is less than 1 second, maintaining the real-time and accuracy of the three-dimensional model.

[0160] Embodiment two

[0161] A real scene three-dimensional modeling system that integrates laser point cloud and image includes:

[0162] Hardware unit: mobile acquisition device integrating laser radar, multi-view camera, and synchronous trigger module, used to acquire laser point cloud data and multi-view image data;

[0163] Dynamic registration unit: performs dynamic weight distribution and registration parameter optimization of point cloud density and image texture, providing accurate registration data for subsequent fusion modeling;

[0164] Fusion modeling unit: calls the cross-modal alignment network to realize the collaborative generation of geometry and texture, and constructs a complete three-dimensional geometric model and texture map;

[0165] Semantic repair unit: based on scene semantic labels and GAN network to realize hole completion, ensuring the integrity of the three-dimensional model.

[0166] The system also includes an incremental updating unit: realizes real-time updating of the model through a local computing pipeline, ensuring the timeliness of the three-dimensional model in a dynamic scene.

[0167] Therefore, the application adopts the above-mentioned fusion laser point cloud and image real scene three-dimensional modeling method and system, solves the limitation of single data source modeling, and realizes high-precision, high-integrity and real-time updating real scene three-dimensional modeling.

[0168] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit them, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that: it can still modify or equivalently replace the technical solutions of the present application, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present application.

Claims

1. A method for real-scene 3D modeling by fusing laser point cloud and image, characterized in that: The following steps are involved: Acquire laser point cloud data and multi-view image data of the target scene through synchronous acquisition equipment; Jointly calibrate the lidar and camera to generate spatiotemporal synchronization parameters; Based on the dynamic weighted registration algorithm, the laser point cloud and image data are multi-level registered using the spatiotemporal synchronization parameters generated by joint calibration. The registration weights are dynamically adjusted according to the point cloud density distribution and image texture complexity. The dynamic weighted registration algorithm includes: In the coarse registration stage, the point cloud is projected into the image coordinate system using the spatiotemporal synchronization parameters obtained by joint calibration; In the fine registration stage, the high confidence area and the low confidence area are divided according to the superposition result of the generated point cloud density distribution map and the image texture complexity heat map; In low-confidence areas, priority is given to laser point cloud data registration, and in high-confidence areas, priority is given to image feature matching. The registration parameters are iteratively updated through a nonlinear optimization algorithm to obtain the registered point cloud and image data. Fuse the registered data to build a 3D geometric model and generate seamless texture maps; The construction of the 3D geometric model includes: Based on the registered laser point cloud data, the initial mesh model is generated using the Poisson reconstruction algorithm; Based on the registered multi-view image data, a dense point cloud is generated through a stereo matching algorithm to fill in the missing areas of the laser point cloud; When fusing two types of point clouds, the curvature consistency of the overlapping area is checked, and the optimized geometric model is generated after removing outliers. Perform semantically driven hole repair on the missing areas of the constructed 3D geometric model and output a complete 3D model; Semantic-driven hole repair includes: In the semantic segmentation stage, the registered point cloud and image are cross-modally segmented to extract scene semantic labels; In the completion phase, a local 3D prior model is constructed based on the scene semantic labels, and the geometry and texture of the missing area are generated through a generative adversarial network. During the verification phase, the curvature and color distribution consistency of the completed area are compared with those of the adjacent areas. If they do not match, they are regenerated until a complete and accurate three-dimensional model is obtained.

2. The method for real-scene 3D modeling by fusing laser point cloud and image according to claim 1, characterized in that: Jointly calibrate the lidar and camera to generate spatiotemporal synchronization parameters, including: A checkerboard calibration plate is used to synchronously obtain the spatial pose relationship between the lidar and the camera; The exposure time of the lidar and camera is controlled by the time synchronization module, with a time error of less than 0.1ms; The calibration results are stored as spatiotemporal synchronization parameters, including the spatial transformation matrix and timestamp offset.

3. The method for real-scene 3D modeling by fusing laser point cloud and image according to claim 1, characterized in that: The point cloud density distribution map and image texture complexity heat map are generated as follows: Through the point cloud density calculation module, the number of local point clouds is counted in voxels to generate a density distribution map; Through the image texture analysis module, the gradient amplitude variance of each pixel is calculated to generate a texture complexity heat map; The generated density distribution map is spatially aligned with the texture complexity heat map and then superimposed to generate a dynamic weight distribution map.

4. The method for real-scene 3D modeling by fusing laser point cloud and image according to claim 1, characterized in that: Curvature consistency detection includes: Calculate the difference in local curvature between the laser point cloud and the image-generated point cloud; If the difference exceeds the preset threshold, the area is determined to be an outlier and removed; The mesh of the eliminated point cloud is reconstructed to ensure that the surface of the geometric model is continuous and smooth, and the optimized geometric model is obtained.

5. The method for real-scene 3D modeling by fusing laser point cloud and image according to claim 1, characterized in that: The generation of seamless texture maps includes: In the texture mapping stage, the image viewing angle with the best visibility is selected based on the correspondence between the geometric model vertices and the image pixels; In the texture fusion stage, the color differences between images are eliminated through a cross-modal feature alignment network, and a multi-resolution fusion algorithm is used to eliminate seams; In the texture optimization stage, the image texture is corrected for illumination consistency based on the reflectivity information of the laser point cloud to obtain the texture map of the complete three-dimensional model.

6. The method for real-scene 3D modeling by fusing laser point cloud and image according to claim 5, characterized in that: The cross-modal feature alignment network is trained in the following way: Input laser point cloud reflectivity features and image RGB features; Extract cross-modal shared features through attention mechanism; The aligned texture mapping matrix is ​​output and used in the texture mapping and fusion stages to minimize the spatial offset error between the geometric model vertices and the image features.

7. A real-scene 3D modeling system integrating laser point cloud and image, characterized in that: The method for performing real-scene 3D modeling by fusing laser point clouds and images as described in any one of claims 1 to 6 comprises: Hardware unit: A mobile acquisition device integrating a laser radar, a multi-view camera, and a synchronous trigger module, used to acquire laser point cloud data and multi-view image data; Dynamic registration unit: performs dynamic weight allocation of point cloud density and image texture and optimizes registration parameters to provide accurate registration data for subsequent fusion modeling; Fusion modeling unit: Calls the cross-modal alignment network to achieve collaborative generation of geometry and texture, and constructs a complete 3D geometric model and texture map; Semantic restoration unit: It completes holes based on scene semantic labels and GAN network to ensure the integrity of the 3D model.

Images