Urban scene three-dimensional modeling method and system

Abstract

The invention relates to the technical field of urban three-dimensional modeling, in particular to an urban scene three-dimensional modeling method and system, and the method comprises the steps: recognizing a data missing region in an initial three-dimensional model; according to the space coordinate set of the data missing area and unmanned aerial vehicle flight constraint conditions, generating an initial unmanned aerial vehicle blind compensation path through a multi-objective optimization algorithm; collecting blind compensation data of the data missing region in real time, and calculating the global information entropy of the currently collected blind compensation data; when it is judged that the global information entropy reaches a preset information entropy threshold value, injecting the currently collected blind compensation data into the initial three-dimensional model for incremental updating; when it is judged that the global information entropy does not reach a preset information entropy threshold value, the mutability characteristic of the global information entropy is analyzed, and a subsequent unmanned aerial vehicle blind compensation path and a parameter collection instruction are dynamically adjusted according to the mutability characteristic. The problem that the modeling result is uncertain due to the fact that quality defects exist in collected data in urban three-dimensional modeling is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of urban three-dimensional modeling, and in particular to a method and system for three-dimensional modeling of urban scenes. Background Art

[0002] 3D modeling of urban scenes is crucial for many fields, including urban planning. Accurate 3D models of urban scenes provide intuitive and comprehensive urban spatial information, supporting scientific and rational decision-making. Therefore, efficient and accurate construction of 3D models of urban scenes has become a hot topic of research.

[0003] However, a key issue commonly exists in existing 3D modeling technologies for urban scenes: the pursuit of effective coverage of blind spots while ignoring the importance of data quality. In the actual modeling process, drone blind spot filling paths are often planned to fill in the data-missing areas in the initial 3D model, enabling a comprehensive scan of the blind spots. However, this approach often results in the collected data, while covering the blind spots, suffering from quality defects. Particularly in complex scenarios such as building corners, the collected data may exhibit artifacts due to the presence of reflective materials. These artifacts can seriously affect the accuracy and reliability of the data, leading to uncertainty in the modeling results. While existing technologies can generate drone blind spot filling paths, they lack dynamic data quality assessment and optimization mechanisms when faced with such complex scenarios. This makes it impossible to effectively resolve the contradiction between "scanning well but not scanning well," resulting in the final constructed 3D model failing to meet the high-precision requirements of practical applications.

[0004] Therefore, there is an urgent need for a three-dimensional modeling method for urban scenes to overcome the data quality defects caused by simply pursuing blind spot spatial coverage, effectively eliminate modeling uncertainties, and construct high-quality three-dimensional models of urban scenes. Summary of the Invention

[0005] (1) Technical problems to be solved

[0006] The purpose of the present invention is to provide a method and system for three-dimensional modeling of urban scenes to solve the problem of data quality defects caused by simply pursuing blind spot space coverage and artifacts in reflective scenes of building corners in urban scene three-dimensional modeling, which in turn leads to uncertainty in modeling results.

[0007] (2) Technical solution

[0008] To achieve the above objectives, the present invention provides a method for three-dimensional modeling of urban scenes, the method comprising:

[0009] S1. Obtain an initial 3D model of the target city scene and identify data-missing areas in the initial 3D model; generate an initial UAV blind spot filling path using a multi-objective optimization algorithm based on the spatial coordinate set of the data-missing areas and the UAV flight constraints.

[0010] S2. Control the UAV to fly along the initial UAV blind spot filling path, collect blind spot filling data of the data missing area in real time through the onboard sensors, and calculate the global information entropy of the currently collected blind spot filling data in real time.

[0011] S3. When it is determined that the global information entropy reaches a preset information entropy threshold, the currently collected blind spot filling data is injected into the initial three-dimensional model for incremental update; when it is determined that the global information entropy does not reach the preset information entropy threshold, the mutation characteristics of the global information entropy are analyzed, and the subsequent drone blind spot filling path and acquisition parameter instructions are dynamically adjusted according to the mutation characteristics.

[0012] Furthermore, the method for generating an initial UAV blind spot filling path by a multi-objective optimization algorithm based on the spatial coordinate set of the data missing area and the UAV flight constraints includes:

[0013] Obtain an initial set of candidate paths for the UAV; calculate the spatial coverage of the candidate paths for the data missing area based on the spatial coordinate set of the data missing area; and calculate a quantitative value of the degree of violation of the flight constraints of the UAV by the candidate paths based on the flight constraints of the UAV.

[0014] By optimizing the objective function, the spatial coverage rate and the quantitative value of the violation degree are simultaneously optimized to generate a set of optimized candidate paths that simultaneously optimize the two objectives; the optimization objective function includes maximizing the spatial coverage rate of the data missing area and minimizing the quantitative value of the violation degree of the drone flight constraint condition.

[0015] A Pareto optimal solution is selected from the set of optimized candidate paths as the initial UAV blind spot filling path.

[0016] Furthermore, the method of collecting blind-filling data of the data-missing area in real time by the onboard sensor and calculating the global information entropy of the currently collected blind-filling data in real time includes:

[0017] The currently collected blind-fill data is divided into multiple local data units; the data features of each local data unit are obtained, and the local information entropy is calculated based on the data features; each local data unit is detected to determine whether there are artifacts caused by reflective materials, and the corresponding interference coefficient is calculated based on the significance of the artifacts.

[0018] The local information entropy of the corresponding local data unit is corrected by using the interference coefficient to obtain a corrected local information entropy; and the corrected local information entropies of the multiple local data units are weightedly fused to generate a global information entropy of the currently collected blind-filled data.

[0019] Furthermore, the method of detecting whether there is an artifact caused by a reflective material in each of the local data units and calculating a corresponding interference coefficient according to the significance of the artifact includes:

[0020] An optical feature vector of the local data unit is extracted, where the optical feature vector includes a pixel intensity distribution feature, a color space outlier feature, and an edge sharpness feature.

[0021] The optical feature vector is input into a pre-trained reflection artifact recognition model, and the boundary coordinates of the artifact area in the local data unit and the corresponding artifact confidence are output.

[0022] According to the area ratio of the artifact region in the local data unit and the artifact confidence, a significance evaluation value of the artifact is calculated; and the significance evaluation value is mapped to a preset interference coefficient interval to generate the interference coefficient.

[0023] Furthermore, the method of inputting the optical feature vector into a pre-trained reflective artifact recognition model and outputting the boundary coordinates of the artifact area in the local data unit and the corresponding artifact confidence includes:

[0024] The encoder of the pre-trained reflective artifact recognition model is used to perform deep feature encoding on the optical feature vector to generate a feature map containing spatial context information.

[0025] According to the feature map, pixel-level semantic segmentation is performed through the first branch of the pre-trained reflective artifact recognition model to predict the probability of each pixel belonging to the artifact category and generate an artifact area segmentation mask, and the boundary coordinates of the artifact area are extracted according to the artifact area segmentation mask; the feature map is subjected to spatial region aggregation analysis by the second branch of the pre-trained reflective artifact recognition model, and the overall region confidence corresponding to the artifact area segmentation mask is predicted as the artifact confidence.

[0026] Furthermore, the method for calculating the artifact significance evaluation value based on the area ratio of the artifact region in the local data unit and the artifact confidence level includes:

[0027] The area ratio and the artifact confidence are normalized respectively to obtain corresponding normalized area ratio values ​​and normalized confidence values; the normalized confidence values ​​are converted through a preset nonlinear mapping function to generate a confidence enhancement factor; and the artifact significance evaluation value is calculated based on the normalized area value and the confidence enhancement factor.

[0028] Furthermore, the method of performing weighted fusion on the corrected local information entropies of the multiple local data units to generate the global information entropy of the currently collected blind-filling data includes:

[0029] A geometric feature vector is extracted from each local data unit, wherein the geometric feature vector includes a surface curvature feature, a depth mutation feature, and a normal vector change feature.

[0030] The geometric feature significance of the local data unit is calculated according to the geometric feature vector; the fusion weight coefficient of the local data unit is determined according to the geometric feature significance, wherein the local data unit with higher geometric feature significance is assigned a larger fusion weight coefficient; and the global information entropy is obtained by weighted calculation of the corrected local information entropy of the multiple local data units according to the fusion weight coefficient.

[0031] Furthermore, the method of analyzing the mutation characteristics of the global information entropy and dynamically adjusting the subsequent UAV blind spot filling path and the UAV acquisition parameters according to the mutation characteristics includes:

[0032] A scalar field distribution of the global information entropy in three-dimensional space is constructed; a three-dimensional gradient vector field is obtained based on the gradient vector calculation of each position point in the three-dimensional space according to the scalar field distribution; the modulus of each vector in the three-dimensional gradient vector field is extracted as the local mutation intensity value, and the direction of each vector is extracted as the local mutation direction.

[0033] According to the spatial distribution of the local mutation intensity values ​​and the local mutation directions, a mutation intensity distribution map and a main direction distribution map are generated to characterize the mutation characteristics of the global information entropy; and the subsequent UAV blind spot filling path and acquisition parameter instructions are dynamically adjusted according to the mutation intensity distribution map and the main direction distribution map.

[0034] Furthermore, the method for dynamically adjusting subsequent UAV blind spot filling paths and acquisition parameter instructions according to the mutation intensity distribution map and the main direction distribution map includes:

[0035] According to the mutation intensity distribution map, spatial areas whose mutation intensity values ​​exceed a preset mutation intensity threshold are extracted as high-priority waypoint areas; for each high-priority waypoint area, the corresponding local main direction vector is extracted according to the main direction distribution map; the optimal pitch angle adjustment amount of the drone sensor is calculated according to the local main direction vector; and an optimized drone blind spot path and acquisition parameter instructions are generated based on the high-priority waypoint area and its corresponding optimal pitch angle adjustment amount.

[0036] On the other hand, based on the same inventive concept, the present invention also provides a three-dimensional modeling system for urban scenes, the system comprising: a blind spot path generation module, a blind spot data acquisition and information entropy calculation module, and a data quality determination and model update module, wherein the modules are sequentially connected to each other;

[0037] The blind spot filling path generation module is used to obtain an initial 3D model of the target urban scene and identify data-missing areas in the initial 3D model; based on the spatial coordinate set of the data-missing areas and the UAV flight constraints, an initial UAV blind spot filling path is generated through a multi-objective optimization algorithm.

[0038] The blind-filling data collection and information entropy calculation module is used to control the UAV to fly along the initial UAV blind-filling path, collect blind-filling data of the data-missing area in real time through the onboard sensors, and calculate the global information entropy of the currently collected blind-filling data in real time.

[0039] The data quality judgment and model update module is used to determine that when the global information entropy reaches a preset information entropy threshold, the currently collected blind-filling data is injected into the initial three-dimensional model for incremental update; when it is determined that the global information entropy does not reach the preset information entropy threshold, the mutation characteristics of the global information entropy are analyzed, and the subsequent drone blind-filling path and acquisition parameter instructions are dynamically adjusted according to the mutation characteristics.

[0040] (3) Beneficial effects

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. By calculating the global entropy of blind-filled data in real time, the quality of collected data can be accurately assessed. When the global entropy reaches the preset entropy threshold, high-quality blind-filled data is promptly injected into the initial 3D model for incremental updates. This fundamentally ensures high-quality modeling data, lays a solid foundation for building accurate 3D models, and effectively avoids modeling deviations caused by data quality issues.

[0043] 2. Beyond ensuring basic data quality, we further dynamically analyze data quality trends based on the mutational characteristics of global information entropy, and adjust subsequent drone blind spot paths and acquisition parameters accordingly. This enables drones to flexibly respond to data acquisition needs in complex scenarios, while ensuring data quality while improving data collection efficiency, avoiding ineffective acquisition and resource waste, and significantly enhancing the intelligence of the 3D modeling process.

[0044] 3. For complex scenes with severe reflections, such as building corners, where data collection is challenging, the system analyzes the optical characteristics of local data units and uses a pre-trained reflection artifact recognition model to detect artifacts and calculate interference coefficients. This effectively corrects local information entropy values ​​and generates accurate global information entropy. This significantly enhances adaptability in complex scenes, making the constructed 3D models more realistic. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flowchart of a method for three-dimensional modeling of urban scenes according to Example 1 of the present invention.

[0046] Figure 2 This is a schematic diagram of the module composition of a city scene 3D modeling system according to Example 2 of the present invention. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0048] Before giving examples, it is necessary to explain the application scenarios of the present invention. The present invention is a three-dimensional modeling method and system for urban scenes. It is applied in the three-dimensional modeling technology of urban scenes, pursuing effective coverage of blind spots, but ignoring the importance of data quality. Especially in complex scenes such as building corners, the collected data may have artifacts due to reflective materials. These artifacts will seriously affect the accuracy and reliability of the data, and thus lead to uncertainty in the modeling results.

[0049] Example 1: Figure 1 As shown, this embodiment provides a method for three-dimensional modeling of urban scenes, the method comprising:

[0050] S1. Obtain the initial three-dimensional model of the target city scene and identify the data missing areas in the initial three-dimensional model; generate the initial UAV blind spot filling path through a multi-objective optimization algorithm based on the spatial coordinate set of the data missing area and the UAV flight constraints; the initial three-dimensional model of the target city scene is derived from conventional surveying and mapping methods such as satellite remote sensing data, aerial photogrammetry or ground laser scanning. For example, for a three-dimensional modeling project of a certain downtown commercial district, the initial three-dimensional model is obtained through oblique photogrammetry technology, but due to factors such as building occlusion and imaging angle limitations, there will be data missing areas such as the back of the building, narrow alleys, and the bottom of the building in the initial three-dimensional model. Through geometric analysis and point cloud density detection algorithms, these three-dimensional spatial areas where these data are sparse or completely missing are automatically identified and marked as blind spots that require additional data collection.

[0051] S2. Control the drone to fly along the initial drone blind spot filling path, collect blind spot filling data for the missing data area in real time using onboard sensors, and calculate the global information entropy of the currently collected blind spot filling data in real time. These sensors typically include high-resolution cameras, lidar, and inertial measurement units. Global information entropy, as a quantitative indicator of data quality, can objectively reflect the information richness and reliability of the collected data.

[0052] S3. If the global information entropy is determined to have reached a preset information entropy threshold, the currently acquired blind spot filling data is injected into the initial 3D model for incremental updating. If the global information entropy is determined to have not reached the preset information entropy threshold, the mutation characteristics of the global information entropy are analyzed and, based on these mutation characteristics, the subsequent drone blind spot filling paths and acquisition parameter instructions are dynamically adjusted. The global information entropy is continuously monitored for changes. When it reaches the preset information entropy threshold, it indicates that the currently acquired blind spot filling data has met the required quality standards for modeling. This batch of high-quality blind spot filling data is then injected into the initial 3D model for incremental updating, achieving real-time model optimization. This quality-gated data fusion mechanism ensures that only data that meets quality requirements is included in the modeling process, fundamentally guaranteeing the accuracy and reliability of the final model. If the global information entropy does not reach the preset information entropy threshold, the original path is not simply continued, but rather the mutation characteristics of the global information entropy are further analyzed. Path adjustments include replanning the flight trajectory and adjusting the flight altitude and angle to avoid or re-approach areas of abnormal data quality. Acquisition parameter adjustments involve optimizing sensor settings.

[0053] The method for generating an initial UAV blind spot filling path by a multi-objective optimization algorithm based on the spatial coordinate set of the data missing area and the UAV flight constraints includes:

[0054] The method obtains an initial set of candidate paths for the drone; calculates the spatial coverage of the data-missing regions by the candidate paths based on the spatial coordinates of the data-missing regions; and calculates a quantitative value for the degree of violation of the flight constraints by the candidate paths based on the drone's flight constraints. Based on the spatial distribution of the data-missing regions, multiple candidate paths are generated using a grid-covering, spiral-covering, or heuristic path planning algorithm. Each data-missing region can be represented as a geometric volume in three-dimensional space, and the drone's sensor field of view is modeled as a dynamic three-dimensional cone or polyhedron. When the drone flies along a candidate path, the spatial scanning range of the sensor's field of view at different locations intersects with the data-missing regions. Spatial coverage is quantified by calculating the ratio of these intersection volumes to the total volume of the data-missing regions, with values ​​closer to 1 indicating better path coverage. The drone's flight constraints include multiple dimensions: an altitude constraint requires the drone to fly at a minimum altitude and not below the regulatory limit; a speed constraint limits flight speed to a safe range; a turning constraint limits the curvature of the path based on the drone's minimum turning radius; a no-fly zone constraint ensures the path does not pass through sensitive areas such as airports and military zones; and a battery constraint ensures the entire mission can be completed on a single charge. For each flight constraint, the violation degree of the candidate path is calculated and normalized, and then a comprehensive violation degree quantitative value is obtained through weighted summation.

[0055] By optimizing the spatial coverage and the violation degree quantified value simultaneously through an optimization objective function, a set of optimized candidate paths that simultaneously optimize the two objectives is generated; the optimization objective function includes maximizing the spatial coverage of the data missing area and minimizing the violation degree quantified value of the UAV flight constraint condition;

[0056] A Pareto optimal solution is selected from the set of optimized candidate paths as the initial UAV blind spot filling path.

[0057] The optimization objective function involves maximizing spatial coverage and minimizing a quantitative value for constraint violations. This is a typical Pareto multi-objective optimization problem, as these two objectives often conflict: improving spatial coverage may require more complex flight paths, increasing the risk of constraint violations; while strictly adhering to flight constraints may result in inadequate coverage of some hard-to-reach areas. A modified non-dominated sorting genetic algorithm or particle swarm optimization algorithm is used to solve the multi-objective problem, generating a set of candidate paths that strike different balances between the two objectives. From this set of optimized candidate paths, the Pareto optimal solution is selected as the initial UAV blind spot filling path. A Pareto optimal solution is one that cannot further improve any objective without compromising the others. In practice, priority weights can be set based on the specific requirements of the mission: for projects requiring extremely high accuracy, candidate paths with higher spatial coverage may be prioritized; for areas with strict safety requirements, candidate paths with lower constraint violations may be preferred. This flexible selection mechanism ensures that the generated initial UAV blind spot filling paths meet both modeling requirements and actual operational safety and regulatory requirements.

[0058] The method of collecting blind-filling data of the data-missing area in real time by the onboard sensor and calculating the global information entropy of the currently collected blind-filling data in real time includes:

[0059] The currently collected blind spot filling data is divided into multiple local data units. Data features of each local data unit are obtained and local information entropy is calculated based on these data features. Each local data unit is tested for artifacts caused by reflective materials and the corresponding interference coefficient is calculated based on the artifact's significance. As the drone flies along a preset path and collects blind spot filling data, the real-time blind spot filling data is divided into multiple local data units. The division strategy is based on two dimensions: spatially, the 3D point cloud or image data is segmented into appropriately sized spatial blocks based on the sensor's field of view and data density. Temporally, the continuous data stream is segmented into temporally continuous data segments based on the data acquisition timestamp. Each local data unit contains hundreds to thousands of data points, ensuring the effectiveness of statistical analysis while facilitating real-time processing. For each local data unit, data features are extracted and local information entropy is calculated. Data features include the point cloud's spatial distribution, intensity distribution, normal vector distribution, and color information distribution. The local information entropy is calculated based on the probability distribution of these data features and reflects the information richness of the data in that local area. High-quality data is typically characterized by high entropy values ​​in its feature distribution, while noisy or redundant data exhibits lower entropy values. In urban scenes, reflective materials at building corners can produce false highlights, leading to artifacts in the collected data. These artifacts can severely interfere with the calculation of information entropy, causing local data units containing artifacts to produce falsely high entropy values ​​due to abnormal brightness variations. To address this issue, artifact detection is performed on each local data unit to identify data anomalies caused by reflective materials and calculate a corresponding interference coefficient based on the artifact's significance.

[0060] The interference coefficient is used to correct the local information entropy of the corresponding local data unit to obtain a corrected local information entropy; the corrected local information entropies of the multiple local data units are weighted and fused to generate the global information entropy of the currently collected blind-filled data. The correction formula typically uses a weighted or modulated approach to reduce the negative impact of artifacts on the information entropy calculation, thereby obtaining a corrected local information entropy that more accurately reflects the quality of the actual data. The weighted fusion process considers factors such as the spatial position, data density, and geometric importance of each local data unit to ensure that the global information entropy can objectively reflect the quality level of the overall data.

[0061] The method of detecting whether there is an artifact caused by a reflective material in each local data unit and calculating a corresponding interference coefficient according to the significance of the artifact includes:

[0062] Extract the optical feature vector of the local data unit, and the optical feature vector includes pixel intensity distribution feature, color space outlier feature and edge sharpness feature; extract the optical feature vector of the local data unit, which is the basic data input for artifact detection. The pixel intensity distribution feature is obtained by statistically analyzing the distribution of pixel brightness values, including statistical parameters such as mean, variance, skewness, kurtosis, and the shape characteristics of the histogram. Reflection artifacts usually appear as abnormal highlights in local areas, showing obvious skewed distribution or multi-peak distribution characteristics in the intensity distribution. The color space outlier feature is identified by analyzing abnormal points in RGB, HSV or Lab color space. The reflective area often shows saturation or overexposure in certain color channels. The edge sharpness feature is obtained by calculating the image gradient and Laplace operator response. The reflective boundary usually has an abnormally sharp intensity change.

[0063] The optical feature vector is input into a pre-trained reflective artifact recognition model, which outputs the boundary coordinates of the artifact area within the local data unit and the corresponding artifact confidence. The pre-trained reflective artifact recognition model is built based on a deep learning architecture and is trained using a large amount of urban building data containing various reflective scenes. The output of the pre-trained reflective artifact recognition model includes two key pieces of information: the boundary coordinates of the artifact area and the corresponding artifact confidence. The boundary coordinates are given in the form of pixel coordinates or three-dimensional spatial coordinates, accurately calibrating the spatial extent of the artifact area; the artifact confidence is expressed as a probability value, reflecting the pre-trained reflective artifact recognition model's degree of confidence in the presence of artifacts in the area.

[0064] Based on the area ratio of the artifact region in the local data unit and the artifact confidence level, an artifact significance evaluation value is calculated. This significance evaluation value is then mapped to a preset interference coefficient range to generate the interference coefficient. The generated interference coefficient is used in the subsequent local information entropy correction process, providing a reliable basis for quantifying artifact interference for the accurate calculation of global information entropy.

[0065] The method of inputting the optical feature vector into a pre-trained reflective artifact recognition model and outputting the boundary coordinates of the artifact area in the local data unit and the corresponding artifact confidence includes:

[0066] The encoder of the pre-trained reflective artifact recognition model is used to perform deep feature encoding on the optical feature vector to generate a feature map containing spatial context information; the pre-trained reflective artifact recognition model adopts a dual-branch design to simultaneously achieve pixel-level segmentation and region-level confidence prediction, providing accurate and reliable results for artifact detection. The encoder is responsible for performing deep feature encoding on the input optical feature vector. The encoder usually adopts a convolutional neural network architecture, such as ResNet or EfficientNet, which maps the original optical features to a high-dimensional feature space through multi-layer convolution, pooling and nonlinear activation operations. During the encoding process, the shallow network focuses on local detail features, such as edges, textures, etc.; the deep network focuses on global semantic features, such as shape, structure, etc. The feature map finally generated not only contains rich semantic information, but also retains the necessary spatial context information, providing sufficient feature expression for subsequent segmentation and classification tasks.

[0067] According to the feature map, pixel-level semantic segmentation is performed through the first branch of the pre-trained reflective artifact recognition model to predict the probability of each pixel belonging to the artifact category and generate an artifact region segmentation mask, and the boundary coordinates of the artifact region are extracted according to the artifact region segmentation mask; the feature map is subjected to spatial region aggregation analysis by the second branch of the pre-trained reflective artifact recognition model, and the overall region confidence corresponding to the artifact region segmentation mask is predicted as the artifact confidence. The first branch performs pixel-level semantic segmentation tasks, and the goal is to predict the probability of each pixel belonging to the artifact category. This branch usually adopts a segmentation network architecture such as U-Net or DeepLab, restores the spatial resolution of the feature map through upsampling and jump connections, and outputs a probability map of the same size as the input. The numerical value of each pixel in the probability map represents the probability of an artifact at that location. The closer the value is to 1, the greater the possibility of an artifact. By setting an appropriate threshold (such as 0.5), the probability Figure 2 The artifact region segmentation mask is generated by quantization, and pixels with a median value of 1 in the mask constitute the artifact region. Based on the segmentation mask, the boundary coordinates of the artifact region are accurately extracted through connected domain analysis and contour extraction algorithms. The second branch performs spatial region aggregation analysis on the feature map to predict the confidence of the entire region. The second branch spatially aggregates the feature map through global average pooling or an attention mechanism, compressing the two-dimensional feature map into a one-dimensional feature vector, and then outputs the region-level classification confidence through a fully connected layer. Unlike pixel-level segmentation, region-level analysis focuses on the overall characteristics of the entire local data unit, and can comprehensively consider global information such as spatial distribution, texture consistency, and geometric structure, thereby providing a more stable and reliable confidence assessment. The design of the two branches achieves complementary functions: the pixel-level branch provides precise spatial positioning to ensure the accuracy of the artifact region boundary; the region-level branch provides reliable confidence assessment to avoid false detection due to local noise.

[0068] The method for calculating the artifact significance evaluation value according to the area ratio of the artifact region in the local data unit and the artifact confidence comprises:

[0069] The area ratio and artifact confidence are normalized to obtain corresponding normalized area ratio values ​​and normalized confidence values. The normalized confidence values ​​are transformed using a preset nonlinear mapping function to generate a confidence enhancement factor. The artifact significance evaluation value is calculated based on the normalized area value and the confidence enhancement factor. Area ratio normalization is based on its value range [0, 1] and is generally a form of normalization, but may require adjustments based on the actual data distribution, such as using a square root transformation to suppress the impact of large values ​​or a logarithmic transformation to stretch the differences of small values. Artifact confidence normalization requires considering the actual distribution characteristics of the model output. A valid value range is determined through statistical analysis, and then linear or nonlinear mapping is performed to ensure that the normalized value fully utilizes the [0, 1] interval. After obtaining the normalized area ratio and normalized confidence value, the normalized confidence value is transformed using a preset nonlinear mapping function to generate a confidence enhancement factor. The design of the nonlinear mapping function is based on the following considerations: high-confidence artifact detection results should be given greater weight, so the mapping function should nonlinearly enhance high confidence values. At the same time, to avoid completely ignoring low-confidence results, the mapping function should maintain a certain degree of smoothness. Commonly used mapping functions include power functions, exponential functions, or piecewise functions, with parameters determined through calibration of experimental data. The final calculation of the significance evaluation value is based on a comprehensive analysis of the normalized area value and the confidence enhancement factor. The calculation formula typically uses a weighted product, which not only reflects the synergistic effect of area and confidence but also avoids excessive dominance of a single factor. Specifically, when both the area ratio and confidence are high, the significance evaluation value should be close to the maximum value; when both are low, the evaluation value should be close to the minimum value; and when one is high while the other is low, the evaluation value should be at an intermediate level. The specific value depends on the balance between accuracy and robustness required by the actual application. The two key dimensions of artifact detection are fully considered: spatial impact range and detection reliability. Normalization eliminates the influence of different dimensions, nonlinear transformation enhances the discrimination of high-confidence results, and comprehensive calculations achieve objective quantification of artifact significance. The resulting significance evaluation provides reliable input for subsequent interference coefficient mapping.

[0070] The method of weightedly fusing the corrected local information entropies of the multiple local data units to generate the global information entropy of the currently collected blind-filling data includes:

[0071] The geometric feature vector of each local data unit is extracted, and the geometric feature vector includes surface curvature feature, depth mutation feature and normal vector change feature; the geometric feature vector of each local data unit is extracted to reflect the complexity and modeling importance of the local three-dimensional structure. The surface curvature feature is obtained by calculating the principal curvature, mean curvature and Gaussian curvature of the point cloud. The areas with drastic curvature changes usually correspond to important structural features such as the corners and decorative details of the building. The data quality of these areas has a key impact on the final modeling effect. The depth mutation feature is calculated by analyzing the depth difference between adjacent points. The area with obvious depth mutation often corresponds to the front and back occlusion relationship of the building, the structural hierarchy and other spatial information, and is the focus of attention in three-dimensional reconstruction. The normal vector change feature is obtained by statistically analyzing the distribution and change degree of the normal vector in the local area. The area with drastic normal vector changes usually corresponds to complex geometric structures, such as building joints, decorative components, etc.

[0072] The geometric feature significance of each local data unit is calculated based on the geometric feature vector. Fusion weight coefficients for each local data unit are determined based on the geometric feature significance, with higher geometric feature significance being assigned larger fusion weight coefficients. The modified local information entropies of the multiple local data units are weighted according to the fusion weight coefficients to obtain global information entropy. Geometric feature significance calculation is achieved through multi-feature fusion, with common methods including principal component analysis, linear discriminant analysis, or entropy-based multi-attribute decision-making methods. During the calculation, the relative importance and complementarity of each geometric feature are comprehensively considered to avoid redundant effects between features. The geometric feature significance value reflects the importance of the local region in the overall three-dimensional structure; higher significance regions have a greater impact on the final modeling quality. Fusion weight coefficients for each local data unit are determined based on the geometric feature significance. Weight allocation follows the principle of geometric significance: higher geometric feature significance local data units are assigned larger fusion weight coefficients. This allocation strategy ensures that the data quality of important structural regions has a greater voice in the global assessment while reducing the impact of data fluctuations in flat or simple areas on the overall assessment. The specific calculation of weight coefficients typically uses soft max normalization or proportional distribution methods to ensure that the sum of all weight coefficients is 1, meeting the requirements of the probability distribution. The modified local information entropy of all local data units is weighted according to the determined fusion weight coefficients to obtain the global information entropy. The weighted calculation formula is the cumulative sum of the products of each local information entropy and the corresponding weight coefficient. In other words, the global information entropy is equal to the weighted average of all modified local information entropies.

[0073] The method of analyzing the mutation characteristics of the global information entropy and dynamically adjusting subsequent UAV blind spot filling paths and UAV acquisition parameters according to the mutation characteristics includes:

[0074] A scalar field distribution of the global information entropy in three-dimensional space is constructed. A three-dimensional gradient vector field is calculated based on the gradient vectors of the scalar field distribution at each location in the three-dimensional space. The modulus of each vector in the three-dimensional gradient vector field is extracted as the local mutation intensity value, and the direction of each vector is extracted as the local mutation direction. The scalar field is constructed using spatial interpolation methods to map the information entropy values ​​of discrete local data units into a continuous three-dimensional space. Common interpolation methods include trilinear interpolation, radial basis function interpolation, or kriging interpolation. The selected method must balance computational efficiency and interpolation accuracy. The constructed scalar field is stored as a three-dimensional grid, with each grid node corresponding to an information entropy value, forming a continuous representation of the information entropy distribution across the entire acquisition space. Gradient calculation uses the finite difference method, approximating the derivative of the information entropy between adjacent grid points. The three-dimensional gradient vector consists of three components, x, y, and z, representing the rate of change of the information entropy in the corresponding direction. The direction of the gradient vector points to the direction of the fastest entropy growth, and the modulus reflects the severity of the entropy change. The construction of the gradient vector field provides the mathematical foundation for subsequent mutation feature analysis. The formula for calculating the modulus of each vector is the square root of the sum of the squares of the gradient vector components. A larger modulus indicates a more dramatic change in information entropy at that location, which in turn indicates greater spatial variability in data quality. The direction of each gradient vector is extracted as the local mutation direction. This directional information is expressed using direction cosines or spherical coordinates, indicating the primary spatial direction of information entropy change.

[0075] Based on the spatial distribution of the local mutation intensity values ​​and local mutation directions, a mutation intensity distribution map and a principal direction distribution map are generated to characterize the mutational characteristics of global information entropy. Subsequent drone blind spot filling paths and acquisition parameter instructions are dynamically adjusted based on these maps. Based on the spatial distribution of the local mutation intensity values ​​and directions, two key analytical graphs are generated: a mutation intensity distribution map and a principal direction distribution map. The mutation intensity distribution map is displayed as a heat map or contour map, visually demonstrating the spatial distribution pattern of data quality mutations. Areas with high mutation intensity typically correspond to data collection difficulties or areas of abnormal quality. The principal direction distribution map is displayed as a vector field or streamline map, revealing the dominant direction of data quality changes and spatial flow trends, providing directional guidance for path planning. By analyzing the mutation intensity distribution, spatial areas requiring special attention can be identified; by analyzing the principal direction distribution, the optimal approach direction and acquisition angle can be determined.

[0076] The method for dynamically adjusting subsequent UAV blind spot filling paths and acquisition parameter instructions according to the mutation intensity distribution map and the main direction distribution map includes:

[0077] Based on the mutation intensity distribution map, spatial regions with mutation intensity values ​​exceeding a preset mutation intensity threshold are identified as high-priority waypoint regions. For each high-priority waypoint region, the corresponding local principal direction vector is extracted based on the principal direction distribution map. The optimal pitch angle adjustment for the drone sensor is calculated based on the local principal direction vector. An optimized drone blind spot path and acquisition parameter instructions are generated based on the high-priority waypoint regions and their corresponding optimal pitch angle adjustment values. All spatial locations with mutation intensity values ​​exceeding a preset mutation intensity threshold are identified, and connected component analysis is used to cluster adjacent high-intensity points into continuous spatial regions. These continuous spatial regions are marked as high-priority waypoint regions, indicating that data quality has significantly changed at these locations and that targeted drone re-acquisition or parameter adjustment is required. The identification of high-priority regions ensures that limited flight resources are prioritized for the spatial locations most in need of improvement. For each identified high-priority waypoint region, the corresponding local principal direction vector is extracted based on the principal direction distribution map. The principal direction vectors are extracted by statistically analyzing the gradient vectors within the high-priority waypoint region. Common methods include vector averaging, principal component analysis, or cluster analysis. The local principal direction vector indicates the dominant direction of change in information entropy within the region, which is also the direction with the greatest potential for improving data quality. By adjusting the acquisition angle along this direction, the drone can obtain higher-quality blind spot data. The calculation of the pitch angle adjustment requires comprehensive consideration of the principal direction vector, the drone's current attitude, the sensor installation orientation, and the geometric characteristics of the target area, so that it is optimally aligned with the principal direction of the target area to improve data acquisition accuracy. The calculation process typically involves three-dimensional geometric transformations and optimization algorithms to align the sensor's principal optical axis with the local principal direction vector as closely as possible while satisfying the drone's flight capability constraints. The optimal pitch angle adjustment amount is given as an angle increment, guiding the drone to make corresponding attitude adjustments when it reaches the waypoint.

[0078] Example 2: Based on the same inventive concept, Figure 2 As shown, this embodiment also provides a three-dimensional modeling system for urban scenes, the system comprising: a blind spot path generation module, a blind spot data collection and information entropy calculation module, and a data quality determination and model update module, wherein the modules are sequentially connected to each other;

[0079] The blind spot filling path generation module is used to obtain an initial 3D model of the target urban scene and identify data-missing areas in the initial 3D model; based on the spatial coordinate set of the data-missing areas and the UAV flight constraints, an initial UAV blind spot filling path is generated through a multi-objective optimization algorithm.

[0080] The blind-filling data collection and information entropy calculation module is used to control the UAV to fly along the initial UAV blind-filling path, collect blind-filling data of the data-missing area in real time through the onboard sensors, and calculate the global information entropy of the currently collected blind-filling data in real time.

[0081] The data quality judgment and model update module is used to determine that when the global information entropy reaches a preset information entropy threshold, the currently collected blind-filling data is injected into the initial three-dimensional model for incremental update; when it is determined that the global information entropy does not reach the preset information entropy threshold, the mutation characteristics of the global information entropy are analyzed, and the subsequent drone blind-filling path and acquisition parameter instructions are dynamically adjusted according to the mutation characteristics.

[0082] It should be noted that, regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0083] Finally, it should be noted that although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A three-dimensional modeling method for urban scenes, characterized in that: The method comprises: Obtain an initial 3D model of the target city scene and identify data-missing areas in the initial 3D model; generate an initial UAV blind spot filling path using a multi-objective optimization algorithm based on the spatial coordinate set of the data-missing areas and the UAV flight constraints; Controlling the UAV to fly along the initial UAV blind spot filling path, collecting blind spot filling data of the data missing area in real time through the onboard sensor, and calculating the global information entropy of the currently collected blind spot filling data in real time; When it is determined that the global information entropy reaches a preset information entropy threshold, the currently collected blind spot filling data is injected into the initial three-dimensional model for incremental update; when it is determined that the global information entropy does not reach the preset information entropy threshold, the mutation characteristics of the global information entropy are analyzed, and the subsequent drone blind spot filling path and acquisition parameter instructions are dynamically adjusted according to the mutation characteristics.

2. A method for three-dimensional modeling of urban scenes according to claim 1, characterized in that: The method for generating an initial UAV blind spot filling path by a multi-objective optimization algorithm based on the spatial coordinate set of the data missing area and the UAV flight constraints includes: Obtaining an initial set of candidate paths for the UAV; calculating the spatial coverage of the candidate paths over the data-missing area based on the spatial coordinate set of the data-missing area; and calculating a quantitative value of the degree of violation of the flight constraints by the candidate paths based on the flight constraints of the UAV; By optimizing the spatial coverage and the violation degree quantified value simultaneously through an optimization objective function, a set of optimized candidate paths that simultaneously optimize the two objectives is generated; the optimization objective function includes maximizing the spatial coverage of the data missing area and minimizing the violation degree quantified value of the UAV flight constraint condition; A Pareto optimal solution is selected from the set of optimized candidate paths as the initial UAV blind spot filling path.

3. The method for three-dimensional modeling of urban scenes according to claim 1, characterized in that: The method of collecting blind-filling data of the data-missing area in real time by the onboard sensor and calculating the global information entropy of the currently collected blind-filling data in real time includes: Divide the currently collected blind spot data into multiple local data units; obtain data features of each local data unit and calculate local information entropy based on the data features; detect whether there is an artifact caused by reflective material in each local data unit, and calculate the corresponding interference coefficient based on the significance of the artifact; The local information entropy of the corresponding local data unit is corrected by using the interference coefficient to obtain a corrected local information entropy; and the corrected local information entropies of the multiple local data units are weightedly fused to generate a global information entropy of the currently collected blind-filled data.

4. A method for three-dimensional modeling of urban scenes according to claim 3, characterized in that: The method of detecting whether there is an artifact caused by a reflective material in each local data unit and calculating a corresponding interference coefficient according to the significance of the artifact includes: Extracting an optical feature vector of the local data unit, wherein the optical feature vector includes a pixel intensity distribution feature, a color space outlier feature, and an edge sharpness feature; Inputting the optical feature vector into a pre-trained reflective artifact recognition model, and outputting the boundary coordinates of the artifact area in the local data unit and the corresponding artifact confidence; According to the area ratio of the artifact region in the local data unit and the artifact confidence, a significance evaluation value of the artifact is calculated; and the significance evaluation value is mapped to a preset interference coefficient interval to generate the interference coefficient.

5. A method for three-dimensional modeling of urban scenes according to claim 4, characterized in that: The method of inputting the optical feature vector into a pre-trained reflective artifact recognition model and outputting the boundary coordinates of the artifact area in the local data unit and the corresponding artifact confidence includes: Performing deep feature encoding on the optical feature vector using an encoder of the pre-trained reflective artifact recognition model to generate a feature map containing spatial context information; According to the feature map, pixel-level semantic segmentation is performed through the first branch of the pre-trained reflective artifact recognition model to predict the probability of each pixel belonging to the artifact category and generate an artifact area segmentation mask, and the boundary coordinates of the artifact area are extracted according to the artifact area segmentation mask; the feature map is subjected to spatial region aggregation analysis by the second branch of the pre-trained reflective artifact recognition model, and the overall region confidence corresponding to the artifact area segmentation mask is predicted as the artifact confidence.

6. A method for three-dimensional modeling of urban scenes according to claim 4, characterized in that: The method for calculating the artifact significance evaluation value according to the area ratio of the artifact region in the local data unit and the artifact confidence comprises: The area ratio and the artifact confidence are normalized respectively to obtain corresponding normalized area ratio values ​​and normalized confidence values; the normalized confidence values ​​are converted through a preset nonlinear mapping function to generate a confidence enhancement factor; and the artifact significance evaluation value is calculated based on the normalized area value and the confidence enhancement factor.

7. A method for three-dimensional modeling of urban scenes according to claim 3, characterized in that: The method of weightedly fusing the corrected local information entropies of the multiple local data units to generate the global information entropy of the currently collected blind-filling data includes: Extracting a geometric feature vector of each local data unit, wherein the geometric feature vector includes a surface curvature feature, a depth mutation feature, and a normal vector change feature; The geometric feature significance of the local data unit is calculated according to the geometric feature vector; the fusion weight coefficient of the local data unit is determined according to the geometric feature significance, wherein the local data unit with higher geometric feature significance is assigned a larger fusion weight coefficient; and the global information entropy is obtained by weighted calculation of the corrected local information entropy of the multiple local data units according to the fusion weight coefficient.

8. The method for three-dimensional modeling of urban scenes according to claim 1, characterized in that: The method of analyzing the mutation characteristics of the global information entropy and dynamically adjusting subsequent UAV blind spot filling paths and UAV acquisition parameters according to the mutation characteristics includes: Constructing a scalar field distribution of the global information entropy in three-dimensional space; calculating a three-dimensional gradient vector field based on the gradient vector of each position point in the three-dimensional space; extracting the modulus of each vector in the three-dimensional gradient vector field as a local mutation intensity value, and extracting the direction of each vector as a local mutation direction; According to the spatial distribution of the local mutation intensity values ​​and the local mutation directions, a mutation intensity distribution map and a main direction distribution map are generated to characterize the mutation characteristics of the global information entropy; and the subsequent UAV blind spot filling path and acquisition parameter instructions are dynamically adjusted according to the mutation intensity distribution map and the main direction distribution map.

9. A method for three-dimensional modeling of urban scenes according to claim 7, characterized in that: The method for dynamically adjusting subsequent UAV blind spot filling paths and acquisition parameter instructions according to the mutation intensity distribution map and the main direction distribution map includes: According to the mutation intensity distribution map, spatial areas whose mutation intensity values ​​exceed a preset mutation intensity threshold are extracted as high-priority waypoint areas; for each high-priority waypoint area, the corresponding local main direction vector is extracted according to the main direction distribution map; the optimal pitch angle adjustment amount of the drone sensor is calculated according to the local main direction vector; and an optimized drone blind spot path and acquisition parameter instructions are generated based on the high-priority waypoint area and its corresponding optimal pitch angle adjustment amount.

10. A three-dimensional modeling system for urban scenes, characterized in that: The system includes: a blind spot filling path generation module, a blind spot filling data collection and information entropy calculation module, and a data quality determination and model update module, and each module is sequentially connected to each other; The blind spot filling path generation module is used to obtain an initial 3D model of the target city scene and identify data-missing areas in the initial 3D model; based on the spatial coordinate set of the data-missing areas and the UAV flight constraints, an initial UAV blind spot filling path is generated using a multi-objective optimization algorithm; The blind spot data collection and information entropy calculation module is used to control the UAV to fly along the initial UAV blind spot path, collect blind spot data of the data missing area in real time through the onboard sensors, and calculate the global information entropy of the currently collected blind spot data in real time; The data quality judgment and model update module is used to determine that when the global information entropy reaches a preset information entropy threshold, the currently collected blind-filling data is injected into the initial three-dimensional model for incremental update; when it is determined that the global information entropy does not reach the preset information entropy threshold, the mutation characteristics of the global information entropy are analyzed, and the subsequent drone blind-filling path and acquisition parameter instructions are dynamically adjusted according to the mutation characteristics.

Images