Live-action three-dimensional model achievement quality improvement method based on multi-source data fusion

Abstract

The invention discloses a multi-source data fused live-action three-dimensional model achievement quality improvement method, which relates to the technical field of surveying and mapping and geographic information, and comprises the steps of acquiring a large-area inclined data rough model, acquiring shielded ground features and narrow roadway model data, acquiring high-rise building fine model data, and merging and producing the data. Compared with the prior art, the method has the advantages that the problems of model cavity, distortion, deformation and the like caused by a low-altitude or narrow roadway target model photographing blind area due to fixed observation angle and limited visual angle in oblique photogrammetry are solved. The problem of large-scale damage of the tops of some ultrahigh building models caused by fixed flight height and camera shooting visual angle in urban area oblique photogrammetry is solved. The problem that the model detail part cannot be displayed in a high-fineness mode due to the fact that the resolution ratio of conventional oblique photography cannot reach the millimeter level is solved.

Description

Technical Field

[0001] The present invention relates to the field of surveying and mapping and geographic information technology, and in particular to a method for fusing multiple data sources such as drone oblique photography, ground laser radar, and ground supplementary photography to improve the quality of real-scene three-dimensional model results. Background Art

[0002] With the development of drones and oblique aerial photography equipment, and the enhancement of data processing capabilities, oblique photography and modeling technologies are playing an increasingly important role in urban development. Realistic 3D models have become an essential new type of spatial infrastructure for the construction of new smart cities. The application of real-world 3D has expanded beyond traditional urban planning to all aspects of the economy and society, and the quality requirements for model output are becoming increasingly stringent. However, due to the fixed observation angle and limited viewing angle of oblique photogrammetry, coupled with obstructions such as eaves, awnings, and narrow alleyways, oblique photography has certain blind spots, preventing full information from the ground. Modeling at the base of buildings or in blind spots can result in holes, distortions, and deformations. Furthermore, when performing oblique photography in urban areas, due to the fixed altitude and camera viewing angle, the image overlap and resolution between the tops of some extremely tall buildings and the ground can differ significantly, resulting in large-scale damage to the top of the model. This traditional 3D modeling method, limited by the fixed flight altitude and limited viewing angle, as well as the complex and changing spatial geometry of urban scenes, cannot meet the needs of higher-resolution modeling and local updates of urban scenes.

[0003] Focusing on the core demand for high-quality real-world 3D image data acquisition using drones, many domestic experts and scholars have pointed out that the key entry points for solving this problem include three aspects: the maneuverability of the drone platform, the adjustability of the payload camera's attitude angle, and the effective control of the drone and camera. For example, close-up photogrammetry, proposed by Academician Zhang Zuxun of Wuhan University, is a measurement method derived from the full integration of the advantages of drone gimbal attitude control and aerial survey high-precision positioning technology. This method can highly restore the fine structure of the surface and objects and has been successfully applied in multiple engineering projects. The core concept of optimal photogrammetry, proposed by Academician Li Qingquan of Shenzhen University, is to optimize the selection of drone aerial photography perspectives that adapt to the geometric structure of the scene object space and meet the requirements of real-world 3D reconstruction. This achieves intelligent drone aerial photography planning that minimizes viewpoint image acquisition and maximizes model reconstruction quality. This technology has been successfully applied to the real-world 3D Qingdao construction project with good results. However, these methods have high technical requirements and high costs, making them difficult to promote and popularize.

[0004] To address issues such as distortion, deformation, and voids in tilted 3D models, as well as damaged high-rise buildings, we employ integrated air-space-ground technology to acquire multi-source data. Through data fusion and 3D reconstruction, we create a detailed 3D model. This method aims to improve model quality while reducing the workload of in-house model scene retouching. It can be used to display detailed and significant features such as important buildings, sculptures, and ancient trees, providing a new approach to enhancing the 3D effects of real-world urban scenes. Summary of the Invention

[0005] The present invention aims to provide a method for improving the quality of real-scene three-dimensional model results by fusing multi-source data. This method realizes the comprehensive collection and precise fusion of model data through the comprehensive utilization of multiple data sources such as drone oblique photography, ground-based lidar, and ground supplementary photography, thereby significantly improving the accuracy and integrity of the real-scene three-dimensional model.

[0006] To solve the above technical problems, the present invention provides a technical solution: a method for improving the quality of real-scene 3D model results by fusion of multi-source data, including obtaining rough models of large-area tilted data, obtaining model data of obstructed objects and narrow alleys, obtaining detailed model data of high-rise buildings, and data merging and production:

[0007] 1) Acquisition of a rough model of large-area tilt data: A drone equipped with five cameras acquires a rough model of large-area tilt data. Based on the altitude calculation formula, the altitude is designed in the Route Planner. Using a drone equipped with tilt cameras and RTK positioning equipment, flights are conducted according to strict instrument operation specifications to acquire preliminary tilt image data. At the same time, image control points are deployed and measured.

[0008] 2) Obtaining model data for obstructed terrain and narrow alleyways: A fusion modeling approach of "close-up re-shooting + handheld LiDAR" is used. For ground areas, based on the rough model, the areas requiring re-shooting are selected, and the re-shooting route is designed.

[0009] 3) Obtaining detailed model data for high-rise buildings: Using a "cross-circling route supplementary shooting" method for fusion modeling, the features that require cross-circling supplementary shooting are delineated based on the preliminary tilt data model, and the height of the features to be circled is set, and detailed route planning is performed. After the route is planned, it can be directly imported into the drone equipment for waypoint mode operation, and the shooting mode is set to fixed-point hovering shooting;

[0010] 4) Data merging and production: The three different data sources (UAV tilt data + ground-based LiDAR data + ground-based supplementary photography data + surround-based supplementary oblique photography data) are processed in the corresponding reconstruction software, and the aerial triangulation files from different data sources are merged to produce a new 3D model.

[0011] As an improvement, after data acquisition in step 1), a rough result of a real-scene three-dimensional model is generated through aerial triangulation, multi-view image intensive matching, 3DTIN model construction, and texture association by modeling software.

[0012] As an improvement, in step 2), based on the designed route, a handheld laser radar is used to obtain the target point cloud data, and additional photos are taken at the same time. The control points are collected through RTK and total station.

[0013] As an improvement, in step 3), the features that need to be finely modeled are selected, fine route planning is performed, a cross-circling method is used, waypoint mode operation is performed, and additional oblique photography data is collected.

[0014] As an improvement, in step 4), several aerial triangulation results are merged in the Reconstruction Master software, and a TIN structure is constructed based on the merged aerial triangulation results to optimize the triangulation network.

[0015] As an improvement, the best target image is selected for texture mapping and optimization. When building a detailed three-dimensional model, the area is divided into small tiles and processed through a multi-node cluster system to complete the three-dimensional model reconstruction.

[0016] Compared with existing technologies, this invention has the following advantages: it solves the problems of model holes, distortion, and deformation in low-altitude or narrow alleyway target models caused by the fixed observation angle and limited viewing angle in oblique photogrammetry. It also solves the problem of large-scale damage to the tops of some super-tall building models caused by the fixed altitude and camera shooting angle in urban area oblique photogrammetry. It also solves the problem of conventional oblique photography that cannot display model details with high precision due to the resolution of conventional oblique photography not reaching the millimeter level. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of a method for improving the quality of real-scene three-dimensional model results by fusing multi-source data.

[0018] Figure 2 This is a schematic diagram of a method for taking supplementary photos with a ground camera at close range.

[0019] Figure 3 It is a flowchart of the fusion modeling process of multi-source data aerial triangulation results.

[0020] As shown in the figure: 1. Photo point 1, 2. Photo point 2, 3. Photo point 3, 4. Photo point 4, 5. Photo point 5, 6. Photo point 6, 7. Photo point 7, 8. Photo point 8, 9. Photo point 9, 10. Photo point 10, 11. Photo point 11, A. Main film A, B. Main film B, C. Main film C, D. Main film D, E. Main film E, F, Main film F, a. Left film, b. Right film. DETAILED DESCRIPTION

[0021] The present invention will be described in further detail below with reference to the accompanying drawings.

[0022] In practice, this invention involves 1) using a drone equipped with five cameras to obtain a rough model of large-scale oblique image data. Using an altitude calculation formula, the altitude is designed in Route Planner. Using a drone equipped with oblique cameras and RTK positioning equipment, the drone is flown according to strict instrument operating specifications to obtain preliminary oblique image data. Image control points are also placed and measured. A rough rough model of the real scene is then generated using modeling software, including aerial triangulation, multi-view image intensive matching, 3DTIN model construction, and texture correlation.

[0023] 2) A fusion modeling approach combining close-up re-shooting and handheld LiDAR is used to acquire data for obstructed terrain and narrow tunnel models. For ground areas, based on the initial model, select areas requiring re-shooting and design a re-shooting route. Following the designed route, handheld LiDAR is used to acquire target point cloud data while simultaneously taking re-shooting photos. Control points can be collected using RTK and a total station.

[0024] 3) A "cross-circling route supplementary shooting" method is used for fusion modeling to obtain detailed model data for high-rise buildings. Using five lenses for supplementary shooting would generate a large number of invalid photos, resulting in a huge workload. The circling route method, however, only requires a single lens to capture specific features, resulting in only one-fifth the data required by traditional methods. As long as the shooting plan is reasonable, single-lens photos can achieve the same modeling results with the same effect and accuracy. The circling mode uses fewer photos and captures more angles, ensuring accurate model reconstruction. When two sets of circling route paths overlap, the point cloud density in the center of the circling mode is significantly higher than that in the edge areas, resulting in better reconstruction results in the center. All angles in the circling mode face the survey area, eliminating any wasted film. Therefore, using the cross-circling route method for supplementary shooting of high-rise buildings not only requires fewer photos, saves significant office processing time, and significantly optimizes the efficiency of oblique photography, but also provides better reconstruction results in the center and overlapping areas of the cross-circling route. Based on the rough tilt data model, the features requiring cross-circling re-photography are delineated, and the altitude of the features to be circled is set, allowing for detailed route planning. The planned route can then be directly imported into the drone for waypoint mode operation, with the photography mode set to fixed-point hovering photography. Furthermore, to improve photography efficiency, multi-drone collaboration can be employed.

[0025] 4) Data merging and production: The three different data sources (drone tilt data + ground-based lidar data + ground-based supplementary photography data + surround supplementary oblique photography data) are processed in the corresponding reconstruction software, and the aerial triangulation files from different data sources are fused to produce a new 3D model. The key to fusion is the fusion and splicing of the aerial triangulation encryption points from several data sources. The quality of the fusion of aerial triangulation encryption points directly affects the refinement of the model. In the Reconstruction Master software, the results of several aerial triangulation encryptions are merged. Based on the fused aerial triangulation results, a TIN structure is constructed and the triangulation network is optimized. The best target image is selected for texture mapping and optimization. When constructing a detailed 3D model, small area tiles are divided and processed through a multi-node cluster system to complete the 3D model reconstruction.

[0026] like Figure 2 As shown in the figure, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, and 11 are photo locations. Photos A, B, C, D, E, and F are primary view photos of adjacent locations. The overlap between them is greater than 60%, which determines the spacing between photo locations. The angle α is less than 15°. The three photos at a location are the left photo (a), the main photo (D), and the right photo (b). The main photo is the photo facing the target. Photo locations are determined by the overlap between the main photos. Corner photos need to be doubled, such as at 7, 8, 9, 10, and 11 in the figure.

[0027] How this method works: This method relies on the comprehensive utilization and precise integration of multiple data sources to improve the accuracy and integrity of the real-world 3D model. The following is a detailed explanation of how this method works:

[0028] 1. Data Source Collection and Processing

[0029] Drone oblique photography: Drones equipped with five cameras capture large areas from multiple angles (including vertical and oblique), acquiring rich image data. This data is processed through aerial triangulation and multi-view image intensive matching to generate a preliminary 3D model of the real scene.

[0030] Ground-based LiDAR: For obscured terrain and narrow alleyways, ground-based LiDAR is used to scan and obtain precise point cloud data. This data can compensate for model gaps caused by obstructions in drone-based oblique photography.

[0031] Ground-based re-photography and handheld LiDAR: For areas requiring higher precision, such as high-rise buildings or complex terrain, a combination of ground-based re-photography and handheld LiDAR is used. Ground-based re-photography provides clearer texture information, while handheld LiDAR captures precise point cloud data. The combination of the two produces a more refined 3D model.

[0032] 2. Data Fusion and Optimization

[0033] Aerial triangulation file fusion: This process combines aerial triangulation files (i.e., imagery's exterior orientation and pose information) generated from various data sources, including drone oblique photography, ground-based LiDAR, and ground-based supplementary photography. This step is crucial for data fusion, ensuring consistency and accuracy across these different data sources in 3D space.

[0034] TIN Structure Construction and Optimization: Based on the fused aerotriangulated files, a triangulated irregular network (TIN) structure is constructed. This step is the foundation of 3D model construction, triangulating the point cloud data to generate a continuous 3D surface. Optimizing the TIN structure also improves model accuracy and smoothness.

[0035] Texture Mapping and Optimization: Select the optimal target image for texture mapping, applying the image's texture information to the 3D model. This step enhances the model's realism and visual quality. Texture optimization, such as denoising and contrast enhancement, can further improve model quality.

[0036] 3. 3D model reconstruction and output

[0037] Small area tile division and multi-node cluster processing: The 3D model is divided into multiple small area tiles and processed in parallel using a multi-node cluster system. This step can significantly improve the efficiency of 3D model reconstruction, especially for 3D reconstruction of large-scale scenes.

[0038] Model Output and Quality Control: After the above steps, the final 3D model is output. Before output, the model undergoes quality control, including checking its integrity, accuracy, and visual quality, to ensure that the model meets the requirements of the actual application.

[0039] In summary, the multi-source data fusion method for improving the quality of real-world 3D models works by comprehensively utilizing multiple data sources, accurately integrating information from these sources, and optimizing the 3D model's structure and texture. This method achieves high accuracy and integrity in real-world 3D models. This method has broad application prospects in fields such as urban planning, architectural design, and environmental monitoring.

[0040] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the technical features being referred to. Thus, features specified as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0041] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integral connection; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; or internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0042] In the present invention, unless otherwise expressly specified or limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features being in contact not directly but through another feature between them. Furthermore, a first feature being "above," "above," and "above" a second feature may include the first feature being directly above or obliquely above the second feature, or may simply mean that the first feature is higher in level than the second feature. A first feature being "below," "below," and "below" a second feature may include the first feature being directly below or obliquely below the second feature, or may simply mean that the first feature is lower in level than the second feature.

[0043] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. Throughout this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0044] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention without departing from the principles and purpose of the present invention.

Claims

1. A method for improving the quality of real-life 3D model results by fusion of multi-source data, including obtaining a rough model of large-area tilted data, obtaining model data of obstructed objects and narrow lanes, obtaining fine model data of high-rise buildings, and merging and producing data, characterized in that: 1) Obtaining a rough model of large-area tilt data: The UAV is equipped with 5 lenses to obtain a rough model of large-area tilt data. According to the altitude calculation formula, the altitude is designed in the route planning master. The UAV is equipped with tilt photography lenses and RTK positioning equipment. According to strict instrument operation specifications, the flight is carried out to obtain preliminary tilt image data, and the image control points are arranged and measured at the same time; 2) Obtaining obstructed ground objects and narrow lane model data: Adopting the "close-up reshoot + handheld laser radar" method to integrate modeling, for the ground area, based on the preliminary model, select the area that needs to be reshot, and design the reshoot route; 3) Obtaining fine model data of high-rise buildings: Adopting the "cross-circling route supplementary shooting" method to integrate modeling, delineating the objects that need to be cross-circling supplementary shooting based on the preliminary tilt data model, and setting the height of the objects to be circled, and conducting fine route planning. After the route planning, it can be directly imported into the drone equipment for waypoint mode operation, and the shooting mode is set to fixed-point hovering shooting; 4) Data merging and production: The three different data sources (UAV tilt data + ground lidar data + ground supplementary shooting data + surround supplementary shooting tilt photography data) are processed in the corresponding reconstruction software, and the aerial triangulation files of different data sources are integrated to produce a new 3D model.

2. The method for improving the quality of real-scene 3D model results by multi-source data fusion according to claim 1, characterized in that: Step 1) After data acquisition, the modeling software performs aerial triangulation, multi-view image intensive matching, 3DTIN model construction, and texture association to generate a rough result of the real-scene 3D model.

3. The method for improving the quality of real-scene 3D model results by multi-source data fusion according to claim 1, characterized in that: Step 2) According to the designed route, a handheld laser radar is used to obtain the target point cloud data, and photos are taken at the same time. The control points are collected through RTK and total station.

4. The method for improving the quality of real-scene 3D model results by multi-source data fusion according to claim 1, characterized in that: Step 3) Select the features that need to be finely modeled, carry out fine route planning, use the cross-surround method, perform waypoint mode operations, and supplement the collection of oblique photography data.

5. The method for improving the quality of real-scene 3D model results by multi-source data fusion according to claim 1, characterized in that: Step 4) In the Reconstruction Master software, several aerial triangulation results are combined, and based on the fused aerial triangulation results, a TIN structure is constructed to optimize the triangulation network.

6. The method for improving the quality of real-scene 3D model results by multi-source data fusion according to claim 5, characterized in that: The best target image is selected for texture mapping and optimization. When constructing a detailed 3D model, it is divided into small area tiles and processed through a multi-node cluster system to complete the 3D model reconstruction.

Images