A multi-source data fusion super high-rise building group real scene three-dimensional model construction method

Abstract

The application belongs to the technical field of three-dimensional reconstruction of super high-rise buildings, and particularly relates to a multi-source data fusion super high-rise building group real scene three-dimensional model construction method. The application generates an initial three-dimensional model through a series of processes such as aerial triangulation encryption and triangular net construction based on multi-source image data. In the process of identifying fuzzy areas and data supplementing, the areas with missing textures and structural distortion in the initial three-dimensional model can be located, the supplementing demand is determined according to the characteristics of different areas and preset threshold values, the data is supplemented by unmanned aerial vehicle oblique photography and fused with original data, the building monomer model is formed through operations such as extracting building structure contour, matching high-resolution texture data, performing texture binding and processing, and the spatial position and texture of the building monomer model are fused with the process three-dimensional model to generate a regional three-dimensional real scene model, and finally, the complete super high-rise building group real scene three-dimensional model is formed through the splicing and fusion of the three-dimensional real scene models of various areas.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional reconstruction technology of super high-rise buildings, specifically involving a method for constructing a real-scene three-dimensional model of a super high-rise building complex through multi-source data fusion. Background Technology

[0002] With the rapid development of urban construction, super high-rise building clusters are constantly emerging, characterized by their complex structures, great height, and dense distribution. Accurately constructing realistic 3D models of these super high-rise building clusters is crucial for urban management, public security, and various related research; therefore, building high-precision, highly realistic 3D models has become an urgent need.

[0003] Currently, there are numerous technical methods for constructing realistic 3D models of super high-rise building complexes. Among them, UAV oblique photogrammetry, equipped with multi-lens cameras, can acquire images from multiple angles, quickly obtaining overall image data for large areas and generating 3D models with a certain degree of realism. However, due to factors such as elevation differences, it is difficult to obtain sufficiently clear images of certain details of super high-rise buildings, such as the fine structure of ground-floor shops and the complex structure of the top of the building. This results in the model lacking texture information and having blurred details in these areas. Furthermore, there is also insufficient image data for some obscured areas, such as gaps between buildings, shaded areas, and areas under trees. Optical scanning technologies (such as SLAM) can acquire high-precision point cloud data, accurately reflecting the spatial location and shape information of objects. When acquiring ground point cloud data, it can capture details such as the ground floor shops of super high-rise buildings and the ground facilities around the buildings. However, this technology is limited by the scanning range and angle, making it difficult to cover the entire super high-rise building. Especially when facing dense super high-rise building clusters, the occlusion between buildings will lead to a large amount of missing point cloud data, making it difficult to fully present the overall picture of the super high-rise building cluster. Based on this, this solution proposes a method for constructing a real-scene 3D model of super high-rise building clusters by fusing multi-source data to solve the above problems. Summary of the Invention

[0004] The purpose of this invention is to provide a method for constructing a real-scene 3D model of a super high-rise building complex based on multi-source data fusion. This method can achieve complete 3D reconstruction of the super high-rise building complex by using high-precision point clouds and multi-view texture images generated from multi-source image data, through data registration, fusion optimization and semantic segmentation.

[0005] The specific technical solution adopted by this invention is as follows:

[0006] A method for constructing a real-scene 3D model of a super high-rise building complex through multi-source data fusion includes:

[0007] Multi-source image data of the observation area were collected, and an initial 3D model was constructed and generated based on the multi-source image data through aerial triangulation and triangulation.

[0008] Based on the initial 3D model, the blurred areas of the super high-rise building are identified, and the location of the blurred areas determines whether a data supplementation task needs to be performed. The areas that need to be supplemented are marked as the target areas for supplementation.

[0009] UAV oblique photography was used to collect supplementary images of the target area, and the supplementary images were then fused with the original images to generate a 3D model of the process.

[0010] Based on the process 3D model, the structural outline of the building is extracted and texture mapping is performed to form a single building model;

[0011] The individual building models and process 3D models are fused together in terms of spatial location and texture to generate regional 3D reality models. The regional 3D reality models are then checked and corrected for quality. Finally, the 3D reality models of each region that pass the quality check are stitched together to form a complete 3D reality model of the super high-rise building complex.

[0012] In a preferred embodiment, the step of acquiring multi-source image data of the observation area and constructing and generating an initial 3D model based on the multi-source image data through aerial triangulation and triangulation includes:

[0013] The multi-view oblique photography equipment was used to collect multi-source image data of the observation area and simultaneously record the position and attitude information of each type of image.

[0014] Aerial triangulation is performed on various types of image data. The aerial triangulation process includes extracting image feature points and matching related points, as well as combining position and pose information to complete regional adjustment calculations and determine image spatial parameters and the coordinates of densification points.

[0015] By densely matching image data from multiple perspectives, a high-density spatial point set is generated. Then, noise filtering and distribution optimization are performed on the spatial point set to generate optimized 3D point cloud data.

[0016] The optimized spatial point set is used to construct triangular facets, and terrain features are integrated to form a constrained triangular mesh. The triangular mesh is then structurally optimized and missing parts are repaired to form an initial 3D model.

[0017] In a preferred embodiment, the step of identifying ambiguous areas of the super high-rise building based on the initial 3D model, and determining whether supplementary data collection is needed based on the location of the ambiguous areas, includes:

[0018] Based on the geometric features and texture clarity of the initial 3D model surface, identify texture-deficient areas and structurally distorted areas, and mark texture-deficient areas or structurally distorted areas as blurred areas;

[0019] The ambiguous area is divided into the building top area, the vertical area of ​​the facade, and the foundation connection area according to spatial location.

[0020] When the proportion of the blurred area in the top area of ​​a building or the vertical area of ​​the exterior facade exceeds the preset blurred proportion threshold, it is designated as a supplementary sampling area, and a supplementary sampling instruction is automatically generated.

[0021] When the vertical area of ​​the facade continuously exceeds the preset continuous distortion length threshold in the vertical direction, the need for supplementary sampling is directly triggered, and a supplementary sampling instruction is generated.

[0022] In a preferred embodiment, the step of identifying texture-deficient regions and structurally distorted regions based on the geometric features and texture clarity of the initial 3D model surface, and marking the texture-deficient regions or structurally distorted regions as blurred regions, includes:

[0023] Calculate the rate of curvature change and normal vector consistency of the point cloud on the surface of the initial 3D model, and record them as the first geometric feature index and the second geometric feature index, respectively.

[0024] The first geometric feature index and the second geometric feature index are compared with the preset curvature change rate threshold and normal vector consistency threshold, respectively.

[0025] If the first geometric feature index exceeds the curvature change rate threshold, or the second geometric feature index is lower than the normal vector consistency threshold, it indicates that there is geometric distortion in the corresponding region, and this corresponding region is marked as a fuzzy region.

[0026] If the first geometric feature index is lower than the curvature change rate threshold and the second geometric feature index is higher than the normal vector consistency threshold, it indicates that the geometric shape of the corresponding region is stable, and the texture sharpness is evaluated.

[0027] When performing texture sharpness assessment, the sharpness score of each region is calculated based on the gray-level gradient distribution and feature point matching degree of the image data.

[0028] If the sharpness score is lower than the preset texture sharpness threshold, the corresponding area will be marked as a texture missing area and classified as a blurry area; otherwise, the corresponding area will be determined as a non-blurry area and retained in the initial 3D model.

[0029] In a preferred embodiment, the step of fusing the supplementary image data with the original image data to generate a three-dimensional model of the process includes:

[0030] Spatial encryption calculation is performed on the supplementary image data to extract the encrypted point set of the supplementary area, and the coordinates of the encrypted point set of the supplementary area are unified with the original encrypted point set through coordinate transformation;

[0031] The overlapping area between the supplementary image data and the original image data is obtained, and the overlapping area is aligned at the pixel level to eliminate geometric deviations and illumination differences.

[0032] Based on the alignment of the overlapping areas, the supplementary sampling point set is merged with the original encrypted point set to generate an enhanced triangular patch structure covering the supplementary sampling area;

[0033] The fused triangular facet structure is processed by removing duplicate facets, repairing the seams and cracks between the supplementary acquisition area and the original acquisition area, and smoothing the transition to eliminate geometric abrupt changes at the seams, thus forming a fused process 3D model.

[0034] In a preferred embodiment, the step of extracting the structural outline of the building based on the process 3D model and performing texture mapping to form a single-unit model of the building includes:

[0035] The outline boundary of the main building is identified by plane projection and elevation gradient detection, and continuous circumscribed polygons are extracted by combining edge connectivity and closure criteria.

[0036] The extracted circumscribed polygons are mapped onto the surface of the process 3D model to match the high-resolution texture data of the corresponding region.

[0037] Based on the mapping relationship, the texture data is bound to the surface of the 3D model one by one, and low-quality texture fragments with occlusion or view distortion are removed.

[0038] The preserved texture fragments are processed for color equalization and lighting normalization to unify the differences in lighting and color. Then, the texture seams are gradually blended to form a single building model.

[0039] In a preferred embodiment, the step of fusing the spatial location and texture of the individual building model and the process 3D model to generate a regional 3D reality model includes:

[0040] Based on the vertex coordinate system of the process 3D model, the vertex coordinates of the individual building model are normalized and transformed, and coordinate system deviation is eliminated through spatial registration;

[0041] Feature control points are extracted in the overlapping region, the spatial transformation matrix between the individualized model and the process model is calculated, and the affine transformation of the vertex coordinates is performed.

[0042] The texture and geometry consistency of the affine transformation-derived individual model and the process 3D model are matched. During the fusion process, the outline and texture of the individual model are preserved, while the overall structural continuity of the process model is inherited.

[0043] Multi-scale pixel sampling is performed on the texture seam area, and bilinear interpolation and edge-preserving filtering are combined to suppress color transition distortion, ultimately forming a 3D real-world model of the area.

[0044] In a preferred embodiment, the step of performing quality checks and corrections on the regional 3D reality model includes:

[0045] A multi-level geometric integrity detection algorithm is used to identify non-manifold edges, self-intersecting patches, and topological fracture regions on the model surface, and repair them based on local surface fitting.

[0046] Visual saliency assessment of texture mapping quality is performed to locate blurred, stretched, or misaligned texture blocks, and texture coordinates are optimized by combining image reprojection.

[0047] Semantic constraint rules are introduced to verify the rationality of the building structure and correct deformation errors caused by occlusion. Then, Gaussian curvature analysis is used to compare with ground control points to optimize the verticality of the building facade and the consistency of the roof plane, thus completing the quality check and correction of the regional three-dimensional real scene model.

[0048] This invention also provides a system for constructing a real-scene 3D model of a super high-rise building complex using multi-source data fusion, comprising:

[0049] The initialization module is used to collect multi-source image data of the observation area, and to construct and generate an initial 3D model based on the multi-source image data through aerial triangulation and triangulation.

[0050] The fuzzy recognition module is used to identify fuzzy areas of super high-rise buildings based on the initial 3D model, determine whether supplementary data collection is needed based on the location of the fuzzy areas, and mark the areas that need supplementary data collection as supplementary target areas.

[0051] The supplementary image acquisition module is used to perform UAV oblique photography supplementary image acquisition of the target area, obtain supplementary image data, and fuse the supplementary image data with the original image data to generate a 3D model of the process.

[0052] The texture mapping module is used to extract the structural outline of a building based on the process 3D model and perform texture mapping to form a single building model.

[0053] The stitching and fusion module is used to merge the individual building models with the process 3D models in terms of spatial position and texture to generate regional 3D reality models. It also performs quality checks and corrections on the regional 3D reality models and stitches and merges the 3D reality models of each region that pass the quality check to form a complete 3D reality model of the super high-rise building complex.

[0054] And, an electronic device, the electronic device comprising:

[0055] At least one processor;

[0056] and a memory communicatively connected to the at least one processor;

[0057] The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the above-described method for constructing a real-world 3D model of a super high-rise building complex based on multi-source data fusion.

[0058] The technical effects achieved by this invention are as follows:

[0059] This invention utilizes multi-source data fusion, employing multi-view oblique photography equipment to acquire multi-source image data. After a series of processes including aerial triangulation and triangulation construction, an initial 3D model is generated. During the identification of blurred areas and data supplementation, regions with missing textures and structural distortions in the initial 3D model are located. Supplementation requirements are determined based on the characteristics of different regions and preset thresholds. Data supplementation is achieved through UAV oblique photography and fused with the original data, further refining the model's details and completeness. In the texture mapping and model fusion stage, building structural outlines are extracted, high-resolution texture data is matched, and texture binding and processing are performed to form individual building models. These models are then spatially and texturally fused with the process 3D model to generate regional 3D reality models. Finally, by stitching and fusing the 3D reality models of each region, a complete 3D reality model of a super high-rise building complex is formed. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0061] Figure 2 This is a schematic diagram of the system modules of the present invention;

[0062] Figure 3 This is a schematic diagram of the electronic device structure of the present invention. Detailed Implementation

[0063] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0064] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0065] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in a preferred embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that mutually excludes other embodiments.

[0066] With the rapid development of urbanization, more and more high-rise and super high-rise buildings are emerging. Their complex structures, great heights, and dense distribution make it relatively important to accurately construct realistic 3D models of these super high-rise building clusters for urban management, public security, and various related research.

[0067] Please see Figure 1 As shown, this invention provides a method for constructing a real-world 3D model of a super high-rise building complex based on multi-source data fusion, including:

[0068] S1. Collect multi-source image data of the observation area, and construct and generate an initial three-dimensional model based on the multi-source image data through aerial triangulation and triangulation.

[0069] As described in step S1, after the observation area of ​​the super high-rise building complex is determined, multi-angle, high-resolution image data is acquired using an UAV oblique photography system to ensure complete coverage of the building facade and roof texture information. Then, based on the acquired image data, aerial triangulation encryption technology is used for feature point matching and sparse reconstruction, calculating the exterior orientation elements of each image, and constructing a high-density point cloud based on this. An initial 3D model is generated through triangulation processing. The steps of acquiring multi-source image data of the observation area and constructing and generating the initial 3D model based on the multi-source image data using aerial triangulation encryption and triangulation include:

[0070] The multi-view oblique photography equipment was used to collect multi-source image data of the observation area and simultaneously record the position and attitude information of each type of image.

[0071] Aerial triangulation is performed on various types of image data. The aerial triangulation process includes extracting image feature points and matching related points, as well as combining position and pose information to complete regional adjustment calculations and determine image spatial parameters and the coordinates of densification points.

[0072] By densely matching image data from multiple perspectives, a high-density spatial point set is generated. Then, noise filtering and distribution optimization are performed on the spatial point set to generate optimized 3D point cloud data.

[0073] The optimized spatial point set is used to construct triangular facets, and terrain features are integrated to form a constrained triangular mesh. The triangular mesh is then structurally optimized and missing parts are repaired to form an initial 3D model.

[0074] Specifically, when constructing the initial 3D model of the super high-rise building complex, the first step is to use a multi-view oblique photography device mounted on a drone to collect data from all directions of the target observation area. This process includes not only acquiring high-resolution images of the building tops and facades, but also simultaneously recording the precise position and attitude information of each image, providing basic data support for subsequent data processing. After image acquisition, the aerial triangulation stage is entered. By extracting feature points from the images and using matching algorithms to find correlation points between different images, combined with the position and attitude parameters recorded during the drone's flight, further regional adjustment calculations are performed to solve for each... The exterior orientation elements of the images and the 3D coordinates of the encrypted points are then used. Based on the multi-view image data, a dense matching operation is performed to generate a high-density point cloud containing a large number of spatial points. To ensure the quality of the point cloud data, noise filtering and distribution optimization are performed on the point cloud to remove outliers caused by occlusion, reflection, or other interference factors. The spatial distribution of the point cloud is also adjusted to improve its uniformity and accuracy. The optimized point cloud data is used to construct triangulation patches. During this process, the triangulation mesh is constrained and optimized according to the terrain features, and missing areas are repaired. This results in an initial 3D model with a complete structure and high geometric accuracy.

[0075] S2. Based on the initial 3D model, identify the fuzzy areas of the super high-rise building, determine whether to perform supplementary data collection tasks according to the location of the fuzzy areas, and mark the areas that need to be supplemented as supplementary data collection target areas.

[0076] In step S2, after the initial 3D model is determined, visual saliency analysis is performed on the facade, top edge, and connecting areas of the supertall building based on its geometric shape and texture clarity. This identifies blurred areas caused by occlusion, missing data, or matching errors. Furthermore, coordinate positioning and marking are performed on the spatial distribution and range of these blurred areas for corresponding supplementary data collection, thereby improving the initial 3D model. The steps of identifying blurred areas of the supertall building based on the initial 3D model and determining whether supplementary data collection is necessary based on the location of these blurred areas include:

[0077] Based on the geometric features and texture clarity of the initial 3D model surface, identify texture-deficient areas and structurally distorted areas, and mark texture-deficient areas or structurally distorted areas as blurred areas;

[0078] The ambiguous area is divided into the building top area, the vertical area of ​​the facade, and the foundation connection area according to spatial location.

[0079] When the proportion of the blurred area in the top area of ​​a building or the vertical area of ​​the exterior facade exceeds the preset blurred proportion threshold, it is designated as a supplementary sampling area, and a supplementary sampling instruction is automatically generated.

[0080] When the vertical area of ​​the facade continuously exceeds the preset continuous distortion length threshold in the vertical direction, the need for supplementary sampling is directly triggered, and a supplementary sampling instruction is generated.

[0081] Specifically, when identifying blurred areas in a cluster of supertall buildings, the system first collects the geometric features and texture clarity information of the initial 3D model surface. By analyzing the distribution density and normal vector changes of the point cloud data, combined with the grayscale gradient and feature point matching degree of the image data, it determines whether there are geometric distortions or texture loss issues in each area. If an area is found to have unstable geometry or insufficient texture clarity, it is marked as a blurred area and further classified according to its spatial location. For example, if the blurred area ratio of the building's top exceeds a preset blurred ratio threshold, it will be automatically designated as a supplementary sampling area, and a corresponding supplementary sampling instruction will be generated. In addition, for the vertical areas of the facade, if the continuous distortion length in the vertical direction reaches a preset continuous distortion length threshold, ... This will also directly trigger the need for supplementary data collection to ensure the integrity of the initial 3D model. The blur percentage threshold and the continuous distortion length threshold are dynamically set based on the structural characteristics of the super high-rise building and the sensor acquisition accuracy. Typically, the blur percentage threshold is set to 15% to 30% of the total area of ​​the region, while the continuous distortion length threshold is determined based on the segment height of the building facade, generally 8 to 12 meters. When a blurry area is detected that meets any of the above triggering conditions, a multi-angle supplementary data collection command will be automatically generated to achieve accurate coverage of the blurry area. The supplementary data collection command includes UAV flight path planning, oblique photography angle setting, and image overlap rate requirements, thereby ensuring multi-view, high-resolution data collection of the blurry area during the supplementary data collection process, and improving the geometric restoration accuracy and texture realism of the initial 3D model of the super high-rise building complex.

[0082] It should be noted that the steps of identifying texture-deficient areas and structurally distorted areas based on the geometric features and texture clarity of the initial 3D model surface, and marking texture-deficient or structurally distorted areas as blurred areas, include:

[0083] Calculate the rate of curvature change and normal vector consistency of the point cloud on the surface of the initial 3D model, and record them as the first geometric feature index and the second geometric feature index, respectively.

[0084] The first geometric feature index and the second geometric feature index are compared with the preset curvature change rate threshold and normal vector consistency threshold, respectively.

[0085] If the first geometric feature index exceeds the curvature change rate threshold, or the second geometric feature index is lower than the normal vector consistency threshold, it indicates that there is geometric distortion in the corresponding region, and this corresponding region is marked as a fuzzy region.

[0086] If the first geometric feature index is lower than the curvature change rate threshold and the second geometric feature index is higher than the normal vector consistency threshold, it indicates that the geometric shape of the corresponding region is stable, and the texture sharpness is evaluated.

[0087] When performing texture sharpness assessment, the sharpness score of each region is calculated based on the gray-level gradient distribution and feature point matching degree of the image data.

[0088] If the sharpness score is lower than the preset texture sharpness threshold, the corresponding area will be marked as a texture missing area and classified as a blurry area; otherwise, the corresponding area will be determined as a non-blurry area and retained in the initial 3D model.

[0089] In the above process, when identifying fuzzy regions in the initial 3D model, the curvature change rate and normal vector consistency of the point cloud on the surface of the initial 3D model are first calculated and used as indicators to judge the stability of the geometric shape. Specifically, when calculating the curvature change rate, a covariance matrix analysis method based on the neighborhood point cloud can be used. This involves analyzing the neighborhood distribution of each point in the point cloud, calculating its covariance matrix, and extracting eigenvalues. The ratio of the minimum eigenvalue to the sum of eigenvalues ​​is used to quantify the degree of curvature change and is calculated as the first geometric feature indicator. Simultaneously, normal vector consistency is assessed by statistically analyzing the standard deviation of the angle between the normal vectors of neighboring points to evaluate the consistency level of the normal vectors in the local area and recorded as the second geometric feature indicator. Then, the first geometric feature indicator is compared with a preset curvature change rate threshold. If the threshold is exceeded, it is considered that the region has obvious geometric anomalies and may have underlying problems. In cases of structural distortion, the second geometric feature index is compared with the normal vector consistency threshold. If the index is lower than the threshold, it indicates that the local normal vector dispersion is high, suggesting that the geometric structure of the region is unstable and there is significant normal vector perturbation. In this case, the corresponding region is marked as a blurred region for subsequent accurate re-sampling. When both the first and second geometric feature indices meet the corresponding threshold conditions, a texture sharpness assessment is further performed to ensure the integrity of the surface details of the region. Specifically, the mean and variance of the image grayscale gradient magnitude are calculated, and the density of SIFT feature point matching is combined to quantify the texture sharpness score. Based on this, it is determined whether the sharpness of the corresponding region meets the requirements. If the requirements are not met, the corresponding region is marked as a texture missing region and included in the blurred region category (i.e., the target region for re-sampling) for subsequent image data re-sampling.

[0090] S3. Perform oblique photography of the target area by UAV to obtain supplementary image data, and fuse the supplementary image data with the original image data to generate a three-dimensional model of the process.

[0091] In step S3, after the target area for supplementary image acquisition is determined, a drone equipped with a multi-angle oblique photography device will be used to perform supplementary image acquisition operations on the target area, thereby obtaining supplementary image data under the target area. Then, the supplementary image data and the original image data are registered and fused to obtain the process 3D model. The step of fusing the supplementary image data with the original image data to generate the process 3D model includes:

[0092] Spatial encryption calculation is performed on the supplementary image data to extract the encrypted point set of the supplementary area, and the coordinates of the encrypted point set of the supplementary area are unified with the original encrypted point set through coordinate transformation;

[0093] The overlapping area between the supplementary image data and the original image data is obtained, and the overlapping area is aligned at the pixel level to eliminate geometric deviations and illumination differences.

[0094] Based on the alignment of the overlapping areas, the supplementary sampling point set is merged with the original encrypted point set to generate an enhanced triangular patch structure covering the supplementary sampling area;

[0095] The fused triangular facet structure is subjected to processes including removing duplicate facets, repairing the seams and cracks between the supplementary acquisition area and the original acquisition area, and smoothing the transition to eliminate geometric abrupt changes at the seams, thus forming a fused process 3D model.

[0096] Specifically, when generating the process 3D model, the first step is to spatially encrypt the supplementary image data, extract the encrypted point set of the supplementary area, and unify the encrypted point set of the supplementary area with the original encrypted point set through coordinate transformation. Then, the overlapping area between the supplementary image data and the original image data is obtained, and pixel-level alignment is performed on the overlapping area to eliminate geometric deviations and lighting differences caused by factors such as shooting angle and lighting conditions. After completing the alignment of the overlapping area, the supplementary point set and the original encrypted point set are further fused to generate an enhanced triangular patch structure covering the supplementary area. In this process, the registration of the supplementary point set and the original point set ensures that the two can be seamlessly connected in geometry and texture. Subsequently, the fused triangular patch structure is post-processed, including removing duplicate patches, repairing the seams and gaps between the supplementary area and the original acquisition area, and performing smooth transition processing to eliminate possible geometric abrupt changes at the seams. Finally, a fused 3D model with higher geometric accuracy and more realistic texture is formed, namely the process 3D model.

[0097] S4. Based on the process 3D model, extract the building structure outline and perform texture mapping to form a single building model;

[0098] In step S4, after the process 3D model is output, the structural outline of the building is further extracted based on the process 3D model using edge detection and semantic segmentation algorithms. This accurately divides the components such as the roof, walls, and doors and windows, and maps the high-resolution image texture to the corresponding geometric patches through projection transformation to achieve texture mapping. This restores the detailed features of the building surface, forming a single building model. The steps of extracting the building's structural outline and performing texture mapping based on the process 3D model to form a single building model include:

[0099] The outline boundary of the main building is identified by plane projection and elevation gradient detection, and continuous circumscribed polygons are extracted by combining edge connectivity and closure criteria.

[0100] The extracted circumscribed polygons are mapped onto the surface of the process 3D model to match the high-resolution texture data of the corresponding region.

[0101] Based on the mapping relationship, the texture data is bound to the surface of the 3D model one by one, and low-quality texture fragments with occlusion or view distortion are removed.

[0102] The preserved texture fragments are processed for color equalization and lighting normalization to unify the differences in lighting and color. Then, the texture seams are gradually blended to form a single building model.

[0103] Specifically, when constructing a building's individual unit model, the first step is to perform planar projection and elevation gradient analysis on the process 3D model. This, combined with the building's geometric features, identifies the main outline boundaries. During this process, edge connectivity and closure criteria are used to extract continuous circumscribed polygons, ensuring the integrity and accuracy of the resulting outline. The extracted circumscribed polygons are then mapped onto the surface of the process 3D model. Semantic segmentation algorithms are then used to further refine the regional divisions of different components such as the roof, walls, and doors and windows. For example, semantic labels are applied to the roof and wall areas, and the locations of door and window openings are distinguished to ensure clear and identifiable boundaries for each component. Finally, based on the mapping relationship, texture data from the high-resolution image is processed step by step... Each texture is bound to the corresponding process 3D model surface, while low-quality texture fragments caused by occlusion or viewpoint distortion are removed. It is important to note that, in order to improve the overall effect of the texture mapping, color equalization and lighting normalization processing must be performed on the retained texture fragments to eliminate visual inconsistencies caused by differences in shooting conditions. Finally, progressive blending techniques (such as multi-scale weighted blending and Laplacian edge smoothing algorithms) are used at the texture seams to effectively weaken the splicing marks and improve the visual continuity and realism of the textures, smoothly transitioning the boundaries between adjacent textures, so that the building unit model presents a more natural and realistic appearance, thus completing the construction work from the process 3D model to the final building unit model.

[0104] S5. The individual building models and process 3D models are fused in terms of spatial location and texture to generate regional 3D reality models. The regional 3D reality models are then checked and corrected for quality. The 3D reality models of each region that pass the quality check are then spliced ​​and fused together to form a complete 3D reality model of the super high-rise building complex.

[0105] In step S5, after the individual building model is constructed, it is spatially registered and texture-fused with the corresponding process 3D model to ensure precise alignment of the geometric structure and high-resolution texture. This achieves seamless spatial integration of the individual building model and the process 3D model, generating a highly realistic regional 3D reality model. Then, the generated regional 3D reality model undergoes quality checks and corrections to examine its geometric accuracy, texture clarity, and seam consistency, eliminating or repairing deformed, blurred, or misaligned local areas. Finally, the 3D reality models that pass the quality check are stitched together based on their spatial coordinates and topological relationships. A global optimization algorithm is used to adjust the geometric alignment and texture transition between adjacent areas, eliminating boundary misalignments and lighting differences, ultimately forming a seamless, complete 3D reality model of the super high-rise building complex under a unified coordinate system. The step of spatially registering and texture-fused between the individual building model and the process 3D model to generate the regional 3D reality model includes:

[0106] Based on the vertex coordinate system of the process 3D model, the vertex coordinates of the individual building model are normalized and transformed, and coordinate system deviation is eliminated through spatial registration;

[0107] Feature control points are extracted in the overlapping region, the spatial transformation matrix between the individualized model and the process model is calculated, and the affine transformation of the vertex coordinates is performed.

[0108] The texture and geometry consistency of the affine transformation-derived individual model and the process 3D model are matched. During the fusion process, the outline and texture of the individual model are preserved, while the overall structural continuity of the process model is inherited.

[0109] Multi-scale pixel sampling is performed on the texture seam area, and bilinear interpolation and edge-preserving filtering are combined to suppress color transition distortion and finally form a 3D real scene model of the area.

[0110] Specifically, when constructing a regional 3D reality model, the vertex coordinates of the individual building models are first normalized based on the vertex coordinate system of the process 3D model, ensuring they are on the same coordinate reference. Spatial registration eliminates coordinate system deviations, ensuring the accuracy of subsequent fusion. Then, feature control points are extracted in the overlapping area. By calculating the spatial transformation matrix between the individual building models and the process model (specific calculation methods may include rigid registration algorithms based on least squares), an affine transformation of the vertex coordinates is performed to precisely align the individual building models and the process 3D model in space. After spatial alignment, the affine-transformed individual building models and the process 3D model are then compared. The model undergoes texture and geometry consistency matching. During the fusion process, the outline and texture of the individual models are preserved, which can better restore the real situation of the building surface. At the same time, the overall structural continuity of the 3D model is inherited, so that the fused model has rich details and maintains the overall structural integrity. For texture seam areas, a multi-scale pixel sampling method is used, combined with bilinear interpolation and edge-preserving filtering technology to suppress color transition distortion. Multi-scale pixel sampling can obtain more comprehensive texture information, bilinear interpolation can make color transition more natural, and edge-preserving filtering can preserve the edge information of the texture while smoothing the color, ultimately forming a regional 3D real scene model.

[0111] Secondly, the steps for quality inspection and correction of the regional 3D reality model include:

[0112] A multi-level geometric integrity detection algorithm is used to identify non-manifold edges, self-intersecting patches, and topological fracture regions on the model surface, and repair them based on local surface fitting.

[0113] Visual saliency assessment of texture mapping quality is performed to locate blurred, stretched, or misaligned texture blocks, and texture coordinates are optimized by combining image reprojection.

[0114] Semantic constraint rules are introduced to verify the rationality of the building structure and correct deformation errors caused by occlusion. Then, Gaussian curvature analysis and comparison with ground control points are used to optimize the verticality of the building facade and the consistency of the roof plane, thus completing the quality check and correction of the regional three-dimensional real scene model.

[0115] As described above, after the regional 3D reality model is output, it undergoes corresponding quality checks and corrections to ensure the geometric accuracy and texture realism of the final output model. Specifically, a multi-level geometric integrity detection algorithm is first used to identify non-manifold edges, self-intersecting patches, and topological break regions on the model surface. For example, abnormal geometric structures are located by detecting abrupt changes in the normal vectors of ring neighborhood patches, and the break regions are adaptively repaired based on a local surface fitting algorithm to ensure the continuity of the model surface. At the same time, visual saliency analysis is performed on the texture mapping quality to identify blurred, stretched, or misaligned texture regions. Texture coordinates are optimized by reprojecting the original image to improve visual realism. Non-manifold edges may cause unclear topological structures in certain areas of the model, affecting the overall coherence of the model. Self-intersecting patches can cause the geometric representation of the model to be chaotic and not in line with the actual building structure. Topological break regions destroy the integrity of the model and need to be repaired for continuity through local surface fitting to avoid geometric distortion. Then, visual saliency evaluation of the texture mapping quality is performed, utilizing the perceptual characteristics of the human visual system for salient features in an image to locate... Blurred, stretched, or misaligned texture blocks are identified. Blurred textures cause loss of detail on the model surface, stretched textures lead to image distortion, and misaligned textures disrupt the visual consistency of the model surface. By combining image reprojection technology, texture coordinates are recalculated based on the geometric structure of the regional 3D reality model and the image capture parameters to optimize texture mapping effects, ensuring that textures accurately fit the model surface. Then, semantic constraint rules are introduced to verify the rationality of the building structure. Based on professional knowledge in the architectural field and common building structure rules, it is determined whether the building structure in the model conforms to logic. For example, walls should be continuous, and the position and size of doors and windows should be reasonable. For deformation errors caused by occlusion, the model is corrected by analyzing the occlusion situation and prior knowledge of the building structure. Then, Gaussian curvature analysis is used to calculate the Gaussian curvature of each point on the model surface, assess the verticality of the building facade, compare it with ground control points, and check the consistency of the roof plane. This optimizes the geometric shape of the building facade and roof, ensuring that the geometric accuracy of the regional 3D reality model reaches the required level, thus completing the quality check and correction of the regional 3D reality model.

[0116] Please see Figure 2 A system for constructing a real-scene 3D model of a super high-rise building complex using multi-source data fusion, comprising the above-mentioned method for constructing a real-scene 3D model of a super high-rise building complex using multi-source data fusion, including:

[0117] The initialization module is used to collect multi-source image data of the observation area, and to construct and generate an initial 3D model based on the multi-source image data through aerial triangulation and triangulation.

[0118] The fuzzy recognition module is used to identify fuzzy areas of super high-rise buildings based on the initial 3D model, determine whether supplementary data collection is needed based on the location of the fuzzy areas, and mark the areas that need supplementary data collection as supplementary target areas.

[0119] The supplementary image acquisition module is used to perform UAV oblique photography supplementary image acquisition of the target area, obtain supplementary image data, and fuse the supplementary image data with the original image data to generate a 3D model of the process.

[0120] The texture mapping module is used to extract the structural outline of a building based on the process 3D model and perform texture mapping to form a single building model.

[0121] The stitching and fusion module is used to merge the individual building models with the process 3D models in terms of spatial position and texture to generate regional 3D reality models. It also performs quality checks and corrections on the regional 3D reality models and stitches and merges the 3D reality models of each region that pass the quality check to form a complete 3D reality model of the super high-rise building complex.

[0122] The execution process of the above-mentioned real-scene 3D model construction system corresponds exactly to the steps of the aforementioned method, so it will not be repeated here.

[0123] Please see Figure 3 An electronic device, comprising:

[0124] At least one processor;

[0125] and memory that is communicatively connected to at least one processor;

[0126] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that at least one processor can execute the above-mentioned method for constructing a real-world 3D model of a super high-rise building complex by multi-source data fusion.

[0127] The processor of the aforementioned electronic device can be at least one central processing unit (CPU) or graphics processing unit (GPU) to execute computer program instructions stored in the memory, thereby achieving efficient fusion of multi-source data and accurate reconstruction of the 3D model. The memory provides space for storing data and programs, and can take various forms, such as random access memory (RAM), read-only memory (ROM), hard disk, solid-state drive (SSD), etc., to meet different data storage needs. In this embodiment, taking the collaborative architecture of external and internal memory as an example, the external memory can be a large-capacity SSD for long-term storage of multi-source image data, initial 3D models, and supplementary image data, while the internal memory can be high-speed RAM. The system temporarily stores the image data being processed and the program instructions in operation. By storing the computer program in memory, it can quickly read and execute the corresponding instructions when the processor needs to execute them, ensuring the efficiency and stability of the entire process of constructing a realistic 3D model of a super high-rise building complex. In actual operation, the processor will call the initialization module, fuzzy recognition module, supplementary sampling module, texture mapping module, and stitching and fusion module in sequence according to the logical order of the computer program. In addition, the electronic device may also include an arithmetic unit, input devices, and output devices. The arithmetic unit is responsible for performing various arithmetic and logical operations, the input devices are used to receive external data or user instructions, and the output devices present the processing results in a visual form.

[0128] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained in this invention are implemented according to conventional methods in the art unless otherwise specified or limited.

Claims

1. A method for constructing a real-scene 3D model of a super high-rise building complex through multi-source data fusion, characterized in that: include: Multi-source image data of the observation area were collected, and an initial 3D model was constructed and generated based on the multi-source image data through aerial triangulation and triangulation. Based on the initial 3D model, the blurred areas of the super high-rise building are identified, and the location of the blurred areas determines whether a data supplementation task needs to be performed. The areas that need to be supplemented are marked as the target areas for supplementation. UAV oblique photography was used to collect supplementary images of the target area, and the supplementary images were then fused with the original images to generate a 3D model of the process. Based on the process 3D model, the structural outline of the building is extracted and texture mapping is performed to form a single building model; The individual building models and process 3D models are fused together in terms of spatial location and texture to generate regional 3D reality models. The regional 3D reality models are then checked and corrected for quality. Finally, the 3D reality models of each region that pass the quality check are stitched together to form a complete 3D reality model of the super high-rise building complex. The step of identifying ambiguous areas of super high-rise buildings based on the initial 3D model and determining whether supplementary data collection is needed based on the location of the ambiguous areas includes: Based on the geometric features and texture clarity of the initial 3D model surface, identify texture-deficient areas and structurally distorted areas, and mark texture-deficient areas or structurally distorted areas as blurred areas; The ambiguous area is divided into the building top area, the vertical area of ​​the facade, and the foundation connection area according to spatial location. When the proportion of the blurred area in the top area of ​​a building or the vertical area of ​​the exterior facade exceeds the preset blurred proportion threshold, it is designated as a supplementary sampling area, and a supplementary sampling instruction is automatically generated. When the vertical area of ​​the facade continuously exceeds the preset continuous distortion length threshold in the vertical direction, the need for supplementary sampling is directly triggered, and a supplementary sampling instruction is generated.

2. The method for constructing a real-scene 3D model of a super high-rise building complex based on multi-source data fusion according to claim 1, characterized in that: The steps of acquiring multi-source image data of the observation area and constructing and generating an initial 3D model based on the multi-source image data through aerial triangulation and triangulation include: The multi-view oblique photography equipment was used to collect multi-source image data of the observation area and simultaneously record the position and attitude information of each type of image. Aerial triangulation is performed on various types of image data. The aerial triangulation process includes extracting image feature points and matching related points, as well as combining position and pose information to complete regional adjustment calculations and determine image spatial parameters and the coordinates of densification points. By densely matching image data from multiple perspectives, a high-density spatial point set is generated. Then, noise filtering and distribution optimization are performed on the spatial point set to generate optimized 3D point cloud data. The optimized spatial point set is used to construct triangular facets, and terrain features are integrated to form a constrained triangular mesh. The triangular mesh is then structurally optimized and missing parts are repaired to form an initial 3D model.

3. The method for constructing a real-scene 3D model of a super high-rise building complex based on multi-source data fusion according to claim 1, characterized in that: The step of identifying texture-deficient regions and structurally distorted regions based on the geometric features and texture clarity of the initial 3D model surface, and marking texture-deficient regions or structurally distorted regions as blurred regions, includes: Calculate the rate of curvature change and normal vector consistency of the point cloud on the surface of the initial 3D model, and record them as the first geometric feature index and the second geometric feature index, respectively. The first geometric feature index and the second geometric feature index are compared with the preset curvature change rate threshold and normal vector consistency threshold, respectively. If the first geometric feature index exceeds the curvature change rate threshold, or the second geometric feature index is lower than the normal vector consistency threshold, it indicates that there is geometric distortion in the corresponding region, and this corresponding region is marked as a fuzzy region. If the first geometric feature index is lower than the curvature change rate threshold and the second geometric feature index is higher than the normal vector consistency threshold, it indicates that the geometric shape of the corresponding region is stable, and the texture sharpness is evaluated. When performing texture sharpness assessment, the sharpness score of each region is calculated based on the gray-level gradient distribution and feature point matching degree of the image data. If the sharpness score is lower than the preset texture sharpness threshold, the corresponding area will be marked as a texture missing area and classified as a blurry area; otherwise, the corresponding area will be determined as a non-blurry area and retained in the initial 3D model.

4. The method for constructing a real-scene 3D model of a super high-rise building complex based on multi-source data fusion according to claim 1, characterized in that: The step of fusing the supplementary image data with the original image data to generate a 3D model of the process includes: Spatial encryption calculation is performed on the supplementary image data to extract the encrypted point set of the supplementary area, and the coordinates of the encrypted point set of the supplementary area are unified with the original encrypted point set through coordinate transformation; The overlapping area between the supplementary image data and the original image data is obtained, and the overlapping area is aligned at the pixel level to eliminate geometric deviations and illumination differences. Based on the alignment of the overlapping areas, the supplementary sampling point set is merged with the original encrypted point set to generate an enhanced triangular patch structure covering the supplementary sampling area; The fused triangular facet structure is processed by removing duplicate facets, repairing the seams and cracks between the supplementary acquisition area and the original acquisition area, and smoothing the transition to eliminate geometric abrupt changes at the seams, thus forming a fused process 3D model.

5. The method for constructing a real-scene 3D model of a super high-rise building complex based on multi-source data fusion according to claim 1, characterized in that: The steps of extracting the structural outline of a building based on a process 3D model and performing texture mapping to form a single-unit model of the building include: The outline boundary of the main building is identified by plane projection and elevation gradient detection, and continuous circumscribed polygons are extracted by combining edge connectivity and closure criteria. The extracted circumscribed polygons are mapped onto the surface of the process 3D model to match the high-resolution texture data of the corresponding region. Based on the mapping relationship, the texture data is bound to the surface of the 3D model one by one, and low-quality texture fragments with occlusion or view distortion are removed. The preserved texture fragments are processed for color equalization and lighting normalization to unify the differences in lighting and color. Then, the texture seams are gradually blended to form a single building model.

6. The method for constructing a real-scene 3D model of a super high-rise building complex based on multi-source data fusion according to claim 4, characterized in that: The step of fusing the individual building model with the process 3D model in terms of spatial location and texture to generate a regional 3D reality model includes: Based on the vertex coordinate system of the process 3D model, the vertex coordinates of the individual building model are normalized and transformed, and coordinate system deviation is eliminated through spatial registration; Feature control points are extracted in the overlapping region, the spatial transformation matrix between the individualized model and the process model is calculated, and the affine transformation of the vertex coordinates is performed. The texture and geometry consistency of the affine transformation-derived individual model and the process 3D model are matched. During the fusion process, the outline and texture of the individual model are preserved, while the overall structural continuity of the process model is inherited. Multi-scale pixel sampling is performed on the texture seam area, and bilinear interpolation and edge-preserving filtering are combined to suppress color transition distortion, ultimately forming a 3D real-world model of the area.

7. The method for constructing a real-scene 3D model of a super high-rise building complex based on multi-source data fusion according to claim 1, characterized in that: The steps for quality inspection and correction of the regional 3D reality model include: A multi-level geometric integrity detection algorithm is used to identify non-manifold edges, self-intersecting patches, and topological fracture regions on the model surface, and repair them based on local surface fitting. Visual saliency assessment of texture mapping quality is performed to locate blurred, stretched, or misaligned texture blocks, and texture coordinates are optimized by combining image reprojection. Semantic constraint rules are introduced to verify the rationality of the building structure and correct deformation errors caused by occlusion. Then, Gaussian curvature analysis is used to compare with ground control points to optimize the verticality of the building facade and the consistency of the roof plane, thus completing the quality check and correction of the regional three-dimensional real scene model.

8. A system for constructing a real-scene 3D model of a super high-rise building complex through multi-source data fusion, characterized in that: The method for constructing a real-world 3D model of a super high-rise building complex using multi-source data fusion as described in any one of claims 1 to 7 includes: The initialization module is used to collect multi-source image data of the observation area, and to construct and generate an initial 3D model based on the multi-source image data through aerial triangulation and triangulation. The fuzzy recognition module is used to identify fuzzy areas of super high-rise buildings based on the initial 3D model, determine whether supplementary data collection is needed based on the location of the fuzzy areas, and mark the areas that need supplementary data collection as supplementary target areas. The supplementary image acquisition module is used to perform UAV oblique photography supplementary image acquisition of the target area, obtain supplementary image data, and fuse the supplementary image data with the original image data to generate a 3D model of the process. The texture mapping module is used to extract the structural outline of a building based on the process 3D model and perform texture mapping to form a single building model. The stitching and fusion module is used to merge the individual building models with the process 3D models in terms of spatial position and texture to generate regional 3D reality models. It also performs quality checks and corrections on the regional 3D reality models and stitches and merges the 3D reality models of each region that pass the quality check to form a complete 3D reality model of the super high-rise building complex.

9. An electronic device, characterized in that: The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the data fusion method for constructing a real-world 3D model of a super high-rise building complex according to any one of claims 1 to 7.

Images