Multi-unmanned aerial vehicle rapid three-dimensional reconstruction and decision-making auxiliary method for disaster site and related equipment

By collaboratively collecting image data from multiple drones, constructing uncertainty indicators, selecting the optimal viewpoint for supplementary data collection, filtering key frames, and fusing depth and semantic information, a three-dimensional Gaussian sputtering model is constructed. This solves the problems of data blind spot identification and automated decision-making in three-dimensional reconstruction of disaster sites, enabling rapid and accurate three-dimensional reconstruction and disaster report generation.

CN121999181AActive Publication Date: 2026-05-08CHINA ENERGY ENG GRP GUANGDONG ELECTRIC POWER DESIGN INST CO LTD +1

Patent Information

Application Number
CN202610443507.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-04-07
Publication Date
2026-05-08
Estimated Expiration
2046-04-07

AI Technical Summary

Technical Problem

Existing technologies cannot perceive data blind spots in real time during 3D reconstruction of disaster sites, resulting in empty models. This necessitates manual reorganization of flight operations, slowing down rescue progress. Furthermore, the technologies suffer from high data redundancy, low reconstruction efficiency, slow 3D reconstruction speed, and a disconnect between results and decision-making, lacking automated decision-making capabilities.

Method used

By employing multi-UAV collaborative image data acquisition, uncertainty indicators are constructed to identify high-uncertainty areas. The optimal viewpoint is selected for supplementary data acquisition to form a closed-loop acquisition system. Keyframes are screened, and depth prior and semantic information are fused to construct a three-dimensional Gaussian sputtering model. Structured elements are extracted to generate rescue routes and disaster decision reports.

Benefits of technology

It achieves one-time full coverage mapping of disaster sites, shortens the data collection cycle, improves reconstruction efficiency, meets the requirement of mapping within 1 hour, has a positioning error of less than 1 meter, and a measurement accuracy of centimeter level, and automatically generates disaster decision reports.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121999181A_ABST
    Figure CN121999181A_ABST
Patent Text Reader

Abstract

The invention provides a disaster site-oriented multi-unmanned aerial vehicle rapid three-dimensional reconstruction and decision-making auxiliary method and related equipment, and belongs to the technical field of emergency surveying and mapping and three-dimensional reconstruction. The method comprises the steps that multiple unmanned aerial vehicles are used for cooperatively collecting image data of a disaster site, and uncertainty indexes used for representing the data missing degree are constructed; taking the uncertainty index as input, selecting an optimal viewpoint to generate a supplementary collection task and executing the supplementary collection task based on the contribution degree of candidate viewpoints to reduction of uncertainty and the comprehensive cost of executing the viewpoints, and obtaining an image data set with coverage integrity; screening a key frame set for three-dimensional reconstruction from the image data set; performing motion recovery structure and space partitioning based on the key frame set, and fusing depth priori and semantic information to obtain an initialized point cloud; and based on a three-dimensional Gaussian sputtering model, extracting structured elements, constructing a risk cost map, and obtaining a rescue route and a disaster situation decision report. According to the invention, emergency command and rescue actions can be supported.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of emergency mapping and 3D reconstruction technology, and in particular to a method and related equipment for rapid 3D reconstruction and decision support using multiple unmanned aerial vehicles (UAVs) at disaster sites. Background Technology

[0002] Many related technologies employ fixed-route blind flights, making it impossible to detect data blind spots during the data collection process. If obstructions occur, resulting in holes in the model, manual intervention and re-organization of the flight are required, severely slowing down the rescue progress.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The main objective of this application is to propose a method and related equipment for rapid 3D reconstruction and decision support using multiple unmanned aerial vehicles (UAVs) at disaster sites. This method enables 3D modeling of disaster sites and automatic generation of decision-making schemes to support emergency command and rescue operations.

[0005] To achieve the above objectives, one aspect of this application proposes a method for rapid 3D reconstruction and decision support using multiple unmanned aerial vehicles (UAVs) at disaster sites, the method comprising the following steps: Multiple drones are used to collaboratively collect image data of disaster sites. The target area is discretized into grid cells. Based on the observation coverage, imaging quality and semantic importance of each grid cell, an uncertainty index is constructed to characterize the degree of data missing. Using the uncertainty index as input, high uncertainty regions where the uncertainty index exceeds a preset threshold are identified. For the high uncertainty regions, based on the contribution of candidate viewpoints to reducing uncertainty and the comprehensive cost of executing the viewpoint, the optimal viewpoint is selected to generate a supplementary acquisition task and execute it, forming an evaluation-driven acquisition closed loop to obtain an image dataset with complete coverage. From the image dataset, a set of keyframes for 3D reconstruction is selected by considering image quality and coverage contribution. Based on the set of keyframes, motion reconstruction structure and spatial segmentation are performed, and depth prior and semantic information are fused to obtain the initial point cloud; A semantic 3D Gaussian sputtering model is constructed based on the initialized point cloud; Based on the aforementioned three-dimensional Gaussian sputtering model, structured elements are extracted, a risk cost map is constructed, and rescue routes and disaster decision reports are obtained.

[0006] In some embodiments, the method of using multiple UAVs to collaboratively collect image data of a disaster site discretizes the target area into grid cells, and constructs an uncertainty index to characterize the degree of data missing based on the observation coverage, imaging quality, and semantic importance of each grid cell, including: Multiple UAVs are used to collaboratively collect image data of the disaster site. The target area is discretized into several grid cells. For each grid cell, a normalized uncertainty index is constructed based on the semantic importance of the grid cell, the number of effective observations, the sufficiency of the viewpoint baseline, and the overall imaging quality. The uncertainty index is used to quantify the degree of data loss and the priority of data re-collection for the grid cell. The calculation of the uncertainty index includes: multiplying the weight of the semantic importance by the result obtained by weighted summation of the observation count attenuation term, the viewpoint baseline deficiency term, and the imaging quality deficiency term; The weights of semantic importance are used to characterize the priority of roads, bridges, embankments, densely populated areas, and disaster boundaries that meet preset conditions; The observation count decay term is calculated exponentially based on the effective observation count; the observation count decay term is used to characterize the rate at which uncertainty decreases as the number of observations increases; the effective observation count is the number of observation sets available for the grid cell under quality and geometric constraints; The insufficient angle baseline term is obtained by subtracting the angle baseline sufficiency index; the angle baseline sufficiency index is used to measure whether the included angle between different observation lines and the baseline distribution meet the geometric conditions for triangulation and surface reconstruction. The image quality deficiencies are obtained by subtracting the comprehensive image quality index; the comprehensive image quality index is used to characterize the impact of sharpness, exposure, smoke, rain and fog obstruction and motion blur on image quality. The observation count attenuation term is configured with a first weighting coefficient; the first weighting coefficient is used to adjust the proportion of the observation count attenuation term in the uncertainty index according to the type of on-site task. The insufficient viewpoint baseline is configured with a second weighting coefficient; the second weighting coefficient is used to adjust the proportion of the insufficient viewpoint baseline in the uncertainty index according to the type of on-site task. The imaging quality deficiency item is configured with a third weighting coefficient; the third weighting coefficient is used to adjust the proportion of the imaging quality deficiency item in the uncertainty index according to the type of on-site task.

[0007] In some embodiments, the uncertainty index is used as input to identify high-uncertainty regions where the uncertainty index exceeds a preset threshold. For these high-uncertainty regions, based on the contribution of candidate viewpoints to reducing uncertainty and the overall cost of executing the viewpoint, the optimal viewpoint is selected to generate and execute a supplementary acquisition task, forming an evaluation-driven acquisition closed loop to obtain an image dataset with complete coverage, including: Using the uncertainty index as input, high uncertainty regions where the uncertainty index exceeds a preset threshold are identified; the high uncertainty regions are areas with missing or insufficient data coverage. For the high uncertainty region, based on the contribution of candidate viewpoints to reducing uncertainty and the comprehensive cost of executing the candidate viewpoints, the optimal viewpoint is selected to generate a supplementary sampling task and execute it; the contribution of the candidate viewpoints to reducing uncertainty is measured by the information gain function. The information gain function is calculated by summing the products of the uncertainty index and the visibility function of all grid cells observable from the candidate viewpoint; the information gain function is used to quantify the total amount of uncertainty eliminated by resampling from the candidate viewpoint; The comprehensive cost is a weighted sum of time cost, energy cost, and risk cost; the time cost is used to characterize the time required for the transfer time, climb time, descent time, and flight path length for performing the replenishment mission; the energy cost is used to characterize the battery consumption required for performing the replenishment mission; and the risk cost is used to characterize the safety risks faced in performing the replenishment mission, such as no-fly zones, areas with dense obstacles, areas with strong winds, low-altitude cables, and high-risk areas of smoke and dust. The conditions for selecting the optimal viewpoint include: in the set of feasible viewpoints that meet the preset constraints of UAV range, communication, no-fly zone and mission, the candidate viewpoint with the largest difference between the information gain calculated by the information gain function and the comprehensive cost is selected as the optimal viewpoint. After the supplementary acquisition task is performed, the newly acquired image data is fused with the original image data to obtain an image dataset with complete coverage.

[0008] In some embodiments, the step of selecting a set of keyframes for 3D reconstruction from the image dataset by comprehensively considering image quality and coverage contribution includes: From the image dataset, each frame of the image is comprehensively scored, and a set of keyframes for 3D reconstruction is selected based on the comprehensive score. The comprehensive score is obtained by weighted summation of image quality score, coverage contribution score, baseline contribution score, important area weighted score, and pose confidence score; The image quality score is used to comprehensively measure the image's sharpness, exposure, degree of obstruction by smoke, rain, and fog, and degree of motion blur. The coverage contribution score is used to measure the amount of new coverage that the image frame adds to the high uncertainty area or the uncovered area; The baseline contribution score is used to characterize the viewpoint difference and baseline length gain between the image frame and the selected set of keyframes; The weighted score of important regions is derived from semantic recognition or feature detection results; the weighted score of important regions is used to increase the sampling density of roads, bridges, waterlogged boundaries, landslides, collapsed buildings, and areas where people or vehicles gather to a preset range. The pose confidence score is used to reduce the interference of images with poor pose quality to a preset range; the poor pose quality includes cases of dynamic differential positioning loss of lock or visual positioning drift. The image quality score, coverage contribution score, baseline contribution score, important area weighted score, and pose reliability score are each configured with adjustable weight coefficients, which are used to adjust the proportion of each factor in the comprehensive score according to the type of on-site task and computing power configuration. Based on the comprehensive score, images with a comprehensive score higher than a preset threshold are retained as the keyframe set.

[0009] In some embodiments, the step of performing motion reconstruction and spatial segmentation based on the keyframe set, fusing depth prior and semantic information to obtain an initial point cloud includes: Motion recovery structure processing is performed based on the keyframe set to obtain sparse point cloud and camera pose. Based on the sparse point cloud computing scene bounding box, an octree is used to recursively divide the three-dimensional space to obtain multiple sub-blocks. For each sub-block, a camera frustum for each keyframe is constructed based on the camera pose. Keyframes in which the camera frustum and the sub-block have spatial intersections are selected from the keyframe set and used as the set of visible keyframes for the sub-block. Select a subset of keyframes with a baseline greater than a preset value from the set of keyframes. For each selected frame, use a depth prediction model to generate a pixel-level depth map. Based on the camera intrinsic parameters, back-project the pixel coordinates to the camera coordinate system to obtain three-dimensional points in the camera coordinate system. Based on the rotation matrix and translation vector contained in the camera pose, the 3D points in the camera coordinate system are transformed to the world coordinate system to obtain the 3D points in the world coordinate system. The point cloud obtained by back-projection of multiple key frames in the key frame subset is fused, denoised and downsampled to obtain the initial point cloud.

[0010] In some embodiments, constructing a semantically represented 3D Gaussian sputtering model based on the initialized point cloud includes: The initial point cloud is converted into an initial Gaussian set of three-dimensional Gaussian sputtering, with each Gaussian point having its three-dimensional position as the mean and being assigned initial scale and color parameters. A subset of Gaussian elements within the spatial range of each sub-block is cropped from the initial Gaussian set and used as the starting point for training the sub-block. A semantic map is generated for the keyframes in the keyframe set using a visual language model. The semantic map contains pixel-level semantic category information. For each sub-block, based on the set of visible keyframes of the sub-block, 3D Gaussian sputtering training is performed starting from the Gaussian subset. During training, a joint loss function is constructed for optimization. The joint loss function includes photometric loss, semantic loss, and shape regularization term. The photometric loss is used to constrain the consistency between the rendered color image and the original image. The semantic loss is used to constrain the consistency between the rendered semantic probability distribution and the semantic map generated by the visual language model. The shape regularization term is used to constrain the anisotropy of the Gaussian points. By optimizing the joint loss function, each sub-block obtains an optimized Gaussian subset. After fusing the Gaussian subsets of each sub-block and performing boundary consistency processing, a semantic 3D Gaussian sputtering model containing geometric and semantic information is obtained.

[0011] In some embodiments, the step of extracting structured elements based on the three-dimensional Gaussian sputtering model, constructing a risk-cost map, and obtaining a rescue route and disaster decision report includes: Based on the aforementioned three-dimensional Gaussian sputtering model, structured elements are extracted in space through a three-dimensional semantic field, and these structured elements are transformed into vectorized or rasterized representations. The structured elements include road connectivity areas, water bodies or waterlogged areas, fire points and smoke-affected areas, collapsed building outlines, landslide boundaries, and locations of people or vehicles. Based on the structured elements and terrain geometry, a risk cost map is constructed. The risk cost map divides the scene into spatial locations of impassable areas, restricted access areas, and safe access areas, and defines a comprehensive cost for each spatial location. The comprehensive cost consists of a distance or time cost item, a risk cost item, and a access barrier cost item. The risk cost item is obtained by integrating semantic elements such as landslide proximity, water depth, smoke coverage intensity, and high-risk areas for secondary disasters. The access barrier cost item is used to express impassable areas such as road closures, collapsed debris, or roads that are too narrow. Based on the risk cost map, the system searches for the path with the minimum total cost from the starting point to the target point, outputs the main route and alternative routes, and provides the reasons for detours and risk explanations for each route, generating a disaster decision report. The disaster decision report includes the affected area range, distribution of key risk points, statistics on the distribution of personnel and vehicles, road accessibility conclusions, risk level distribution heat map, recommended action routes, estimated travel time, and suggestions for key material delivery points.

[0012] To achieve the above objectives, another aspect of this application proposes a multi-UAV rapid 3D reconstruction and decision support system for disaster sites, used to implement the method described above. The system includes: The first module is used to collect image data of disaster sites using multiple UAVs in collaboration, discretize the target area into grid cells, and construct an uncertainty index to characterize the degree of data missing based on the observation coverage, imaging quality and semantic importance of each grid cell. The second module is used to take the uncertainty index as input, identify high uncertainty areas where the uncertainty index exceeds a preset threshold, and select the optimal viewpoint to generate a supplementary acquisition task and execute it based on the contribution of candidate viewpoints to reducing uncertainty and the comprehensive cost of executing the viewpoint for the high uncertainty area, thus forming an evaluation-driven acquisition closed loop and obtaining an image dataset with complete coverage. The third module is used to select a set of keyframes for 3D reconstruction from the image dataset by comprehensively considering image quality and coverage contribution. The fourth module is used to perform motion reconstruction and spatial segmentation based on the keyframe set, and to fuse depth prior and semantic information to obtain an initial point cloud; The fifth module is used to construct a semantic 3D Gaussian sputtering model based on the initialized point cloud; The sixth module is used to extract structured elements based on the three-dimensional Gaussian sputtering model, construct a risk cost map, and obtain rescue routes and disaster decision reports.

[0013] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0014] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0015] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method.

[0016] The embodiments of this application include at least the following beneficial effects: This application provides a method, system, electronic device, storage medium, and program product for rapid 3D reconstruction and decision support using multiple UAVs at disaster sites. This application includes: using multiple UAVs to collaboratively collect image data from disaster sites, constructing an uncertainty index to characterize the degree of data loss; using the uncertainty index as input, selecting the optimal viewpoint based on the contribution of candidate viewpoints to reducing uncertainty and the comprehensive cost of executing the viewpoint, generating and executing a supplementary acquisition task, and obtaining an image dataset with complete coverage; selecting a set of keyframes for 3D reconstruction from the image dataset; performing motion recovery structure and spatial segmentation based on the keyframe set, fusing depth priors and semantic information to obtain an initial point cloud; extracting structured elements based on a 3D Gaussian sputtering model, constructing a risk cost map, and obtaining rescue routes and disaster decision reports. This application can support emergency command and rescue operations. Attached Figure Description

[0017] Figure 1 This is a flowchart of a method for rapid 3D reconstruction and decision support using multiple unmanned aerial vehicles (UAVs) at disaster sites, provided in an embodiment of this application. Figure 2 This is a general technical flow diagram provided in the embodiments of this application; Figure 3 This is a flowchart of the 3DGS training process provided in the embodiments of this application; Figure 4 This is a flowchart of the intelligent agent disaster decision-making assistance provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0020] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0021] 1) SfM (Structure from Motion): Structure from Motion is a technique for recovering the structure of a three-dimensional scene and the camera pose from multiple two-dimensional images.

[0022] 2) 3DGS (3D Gaussian Splatting): Three-dimensional Gaussian sputtering, a new radiation field rendering technology based on point clouds, characterized by fast real-time rendering speed and high training efficiency.

[0023] 3) VLM (Vision-Language Model): A deep learning model that can understand image content and perform semantic segmentation or description, such as Qwen3-VL.

[0024] 4) Agent: An intelligent software entity with perception, decision-making and execution capabilities, used in this application to generate reports and plan routes based on 3D data.

[0025] 5) Keyframe: A representative, high-quality image frame selected from a video or a large number of image sequences that makes a significant contribution to 3D reconstruction.

[0026] 6) MVS (Multi-View Stereo): Multi-view stereo vision is a technique used in computer vision for 3D reconstruction.

[0027] 7) RTK (Real-Time Kinematic): Real-time dynamic differential positioning, a high-precision GPS / GNSS positioning technology.

[0028] 8) GNSS / IMU (Global Navigation Satellite System / Inertial Measurement Unit): Global Navigation Satellite System / Inertial Measurement Unit.

[0029] 9) SLAM (Simultaneous Localization and Mapping): Simultaneous localization and mapping.

[0030] 10) Depth Anything V3: A monocular depth estimation model.

[0031] 11) Dijkstra's algorithm: a breadth-first search algorithm for finding the shortest path.

[0032] 12) A*: A* algorithm, an improved version of Dijkstra's algorithm.

[0033] 13) Mesh: A mesh or 3D mesh model is a data structure used in computer graphics and 3D reconstruction to represent the surface of a 3D object.

[0034] The related technologies have the following drawbacks: 1. Lack of a closed-loop data acquisition mechanism, making it difficult to generate images in one go: Existing technologies mostly use fixed flight routes for blind flights, making it impossible to detect data blind spots during the acquisition process. Once occlusion occurs, resulting in holes in the model, it needs to be manually discovered and the flight reorganized later, which seriously slows down the rescue progress.

[0035] 2. High data redundancy and low reconstruction efficiency: Emergency data collection usually generates massive amounts of video or image data. Using all the data for calculation can lead to computing power collapse or excessive time consumption. Moreover, a large number of repeated viewpoints contribute very little to improving model accuracy.

[0036] 3. Slow 3D reconstruction speed: 3D reconstruction based on traditional MVS often takes several days, while advanced algorithms such as 3DGS take a long time in the sparse point cloud densification stage.

[0037] 4. Separation of results from decision-making: 3D modeling and disaster analysis (such as road network accessibility and disaster point statistics) are usually two separate processes, lacking the automated ability to directly generate decision-making solutions (such as rescue routes and disaster reports) from 3D data.

[0038] In view of this, this application provides a method and related equipment for rapid 3D reconstruction and decision support using multiple unmanned aerial vehicles (UAVs) at disaster sites. This solution achieves one-time full-coverage mapping by constructing a closed-loop mechanism of "acquisition-evaluation-supplementation"; significantly reduces data volume and improves parallel processing speed by utilizing keyframe filtering and octree block partitioning strategies; introduces a depth prediction model to accelerate 3DGS initialization; and combines VLM and an agent to achieve automated generation from 3D models to disaster reports and rescue routes. Ultimately, it achieves the engineering goals of mapping within one hour, positioning error less than 1 meter, and measurement accuracy down to the centimeter level.

[0039] Figure 1 This is an optional flowchart of a multi-UAV rapid 3D reconstruction and decision support method for disaster sites provided in this application embodiment. Figure 1 The method may include, but is not limited to, steps S101 to S106.

[0040] Step S101: Use multiple UAVs to collaboratively collect image data of the disaster site, discretize the target area into grid cells, and construct an uncertainty index to characterize the degree of data missing based on the observation coverage, imaging quality and semantic importance of each grid cell. Step S102: Using uncertainty indicators as input, identify high uncertainty areas where the uncertainty indicators exceed a preset threshold. For high uncertainty areas, select the optimal viewpoint to generate a supplementary acquisition task and execute it based on the contribution of candidate viewpoints to reducing uncertainty and the comprehensive cost of executing the viewpoint, thus forming an evaluation-driven acquisition closed loop and obtaining an image dataset with complete coverage. Step S103: From the image dataset, based on image quality and coverage contribution, select a set of keyframes for 3D reconstruction. Step S104: Motion reconstruction and spatial segmentation are performed based on the keyframe set, and depth prior and semantic information are fused to obtain the initial point cloud; Step S105: Construct a semantic 3D Gaussian sputtering model based on the initialized point cloud; Step S106: Based on the three-dimensional Gaussian sputtering model, extract structured elements, construct a risk cost map, and obtain rescue routes and disaster decision reports.

[0041] In steps S101 to S106 of the embodiments of this application, an uncertainty index is constructed to perform a quantitative assessment of the degree of data loss in each grid unit, thereby achieving automatic identification and location of data blind spots. This provides a quantitative basis for subsequent accurate data acquisition and avoids the subjectivity and oversight of manual judgment. Through the optimization of the balance between contribution and cost, the autonomous selection of the optimal viewpoint and the generation of acquisition tasks are performed, forming an evaluation-driven acquisition closed loop. This achieves one-time full-coverage mapping, eliminating the cost of subsequent manual review and secondary attendance, and shortening the data acquisition cycle. Through a comprehensive image quality and coverage contribution screening mechanism, the construction of keyframe sets is performed, reducing the amount of raw data by about 70%. This reduces the subsequent computing power burden while ensuring the reconstruction accuracy of key areas, meeting the timeliness requirement of one-hour mapping. Through spatial segmentation and depth prior fusion, the rapid construction of the initial point cloud is performed, significantly reducing the cost of 3DGS cold start, shortening the iteration number of the densification process, improving early visualization quality, and laying a geometric foundation for subsequent semantic reconstruction. By jointly optimizing semantic and geometric information, a semantic construction of a 3D Gaussian sputtering model is performed, making the model more stable at semantic boundaries and more controllable in geometric form. This provides directly extractable structured elements for downstream disaster analysis and route planning. Through structured element extraction and agent-based decision-making, risk cost map construction and rescue route planning are performed, achieving automated generation from 3D models to disaster reports and rescue routes. This connects the link between model production and command decision-making, providing actionable auxiliary decision-making products.

[0042] In some embodiments, step S101 may include, but is not limited to, step S111: Step S111: Use multiple UAVs to collaboratively collect image data of the disaster site, and discretize the target area into several grid cells. For each grid cell, construct a normalized uncertainty index based on the semantic importance, effective observation count, sufficiency of the viewpoint baseline, and overall imaging quality of the grid cell. The uncertainty index is used to quantify the degree of data loss and the priority of data acquisition for the grid cell. The calculation of the uncertainty index includes: multiplying the weight of the semantic importance by the result obtained by weighted summation of the observation count attenuation term, the viewpoint baseline deficiency term, and the imaging quality deficiency term; The weights of semantic importance are used to characterize the priority of roads, bridges, embankments, densely populated areas, and disaster boundaries that meet preset conditions; The observation count decay term is calculated exponentially based on the effective observation count; the observation count decay term is used to characterize the rate at which uncertainty decreases as the number of observations increases; the effective observation count is the number of observation sets available for the grid cell under quality and geometric constraints; The insufficient angle baseline term is obtained by subtracting the angle baseline sufficiency index; the angle baseline sufficiency index is used to measure whether the included angle between different observation lines and the baseline distribution meet the geometric conditions for triangulation and surface reconstruction. The image quality deficiencies are obtained by subtracting the comprehensive image quality index; the comprehensive image quality index is used to characterize the impact of sharpness, exposure, smoke, rain and fog obstruction and motion blur on image quality. The observation count attenuation term is configured with a first weighting coefficient; the first weighting coefficient is used to adjust the proportion of the observation count attenuation term in the uncertainty index according to the type of on-site task. The insufficient viewpoint baseline is configured with a second weighting coefficient; the second weighting coefficient is used to adjust the proportion of the insufficient viewpoint baseline in the uncertainty index according to the type of on-site task. The imaging quality deficiency item is configured with a third weighting coefficient; the third weighting coefficient is used to adjust the proportion of the imaging quality deficiency item in the uncertainty index according to the type of on-site task.

[0043] The formulas used in step S101 include: ; in, This is an indicator of uncertainty. For grid cells; This is a semantic importance weight used to emphasize key element regions; The effective number of observations for a grid cell. This represents the set of observations available for use in the mesh element under quality and geometric constraints. As an indicator of baseline sufficiency; For comprehensive imaging quality indicators; , and These are the weighting coefficients; This is the observation count attenuation coefficient.

[0044] In step S111 of the embodiments of this application, semantic importance weights are introduced. This involves weighted amplification of key element areas (such as roads, bridges, embankments, and densely populated areas) to automatically prioritize resampling in key rescue areas. This is achieved through an exponential decay term. This involves automatically reducing the weight of already sufficiently observed areas, rapidly decreasing uncertainty as the number of observations increases, avoiding resource waste caused by repeated sampling of the same area, and improving sampling efficiency. This is achieved by introducing... This involves penalizing areas with insufficient viewpoint distribution, prioritizing the acquisition of data in areas with insufficient parallax and poor triangulation conditions, ensuring the geometric stability of 3D reconstruction, and avoiding degraded geometry problems caused by excessively close viewpoints. This is achieved by introducing... This process penalizes areas with poor image quality (blurred, overexposed, or obscured by smoke), prioritizing the acquisition of areas with insufficient clarity and severe interference. This ensures that keyframes possess stable features and reliable textures, improving reconstruction accuracy. Adjustable weighting coefficients are used to achieve this. , and It dynamically adjusts the importance of different evaluation factors, enabling flexible configuration of supplementary sampling strategies based on the type of on-site task, and maintaining optimal performance in different scenarios such as complex terrain, severe weather, or extremely high time requirements.

[0045] In some embodiments, step S102 may include, but is not limited to, steps S201 to S203: Step S201: Using the uncertainty index as input, identify high uncertainty regions where the uncertainty index exceeds a preset threshold; the high uncertainty regions are regions with missing or insufficient data coverage. Step S202: For the high uncertainty region, based on the contribution of candidate viewpoints to reducing uncertainty and the comprehensive cost of executing the candidate viewpoints, the optimal viewpoint is selected to generate a supplementary sampling task and execute it; the contribution of the candidate viewpoints to reducing uncertainty is measured by the information gain function. The information gain function is calculated by summing the products of the uncertainty index and the visibility function of all grid cells observable from the candidate viewpoint; the information gain function is used to quantify the total amount of uncertainty eliminated by resampling from the candidate viewpoint; The comprehensive cost is a weighted sum of time cost, energy cost, and risk cost; the time cost is used to characterize the time required for the transfer time, climb time, descent time, and flight path length for performing the replenishment mission; the energy cost is used to characterize the battery consumption required for performing the replenishment mission; and the risk cost is used to characterize the safety risks faced in performing the replenishment mission, such as no-fly zones, areas with dense obstacles, areas with strong winds, low-altitude cables, and high-risk areas of smoke and dust. The conditions for selecting the optimal viewpoint include: in the set of feasible viewpoints that meet the preset constraints of UAV range, communication, no-fly zone and mission, the candidate viewpoint with the largest difference between the information gain calculated by the information gain function and the comprehensive cost is selected as the optimal viewpoint. Step S203: After performing the supplementary acquisition task, the newly acquired image data is fused with the original image data to obtain an image dataset with complete coverage.

[0046] The formulas used in step S102 include: ; ; ; in, This is the information gain function, used to measure the contribution of candidate viewpoints to reducing uncertainty; Candidate viewpoints; For grid cells; For a grid set; This is an indicator of uncertainty. Visibility function; The overall cost of implementing the viewpoint; For the cost of time; For the cost of energy consumption; For the cost of risk; , and These are the weighting coefficients; The optimal viewpoint; For the drone serial number; In response to drones The set of feasible viewpoints.

[0047] In steps S201 to S203 of this embodiment, the information gain function performs a quantitative evaluation of the information value of candidate viewpoints by summing the product of uncertainty weighting and visibility function, achieving a maximization-oriented approach to supplementary collection benefits—ensuring priority is given to viewpoints that can observe more high-uncertainty areas and have the best observation effects, avoiding resource waste caused by blind supplementary collection. The comprehensive cost of executing viewpoints is quantitatively evaluated by summing the weighted costs of time, energy consumption, and risk, achieving a balance between the safety and economy of supplementary collection flights—while pursuing data integrity, fully considering flight constraints and safety risks at emergency sites, avoiding secondary accidents caused by supplementary collection. The optimal viewpoint is determined by maximizing the difference between benefits and costs, performing autonomous decision-making for the optimal supplementary collection viewpoint, achieving "intelligent supplementary collection with maximum net benefit"—within limited time, battery capacity, and safety margin, selecting the viewpoint that "eliminates the most blind spots while minimizing costs," achieving the optimal solution for resource utilization.

[0048] In some embodiments, step S103 may include, but is not limited to, steps S301 to S302: Step S301: From the image dataset, each frame of the image is comprehensively scored, and a set of keyframes for 3D reconstruction is selected based on the comprehensive score. The comprehensive score is obtained by weighted summation of image quality score, coverage contribution score, baseline contribution score, important area weighted score, and pose confidence score; The image quality score is used to comprehensively measure the image's sharpness, exposure, degree of obstruction by smoke, rain, and fog, and degree of motion blur. The coverage contribution score is used to measure the amount of new coverage that the image frame adds to the high uncertainty area or the uncovered area; The baseline contribution score is used to characterize the viewpoint difference and baseline length gain between the image frame and the selected set of keyframes; The weighted score of important regions is derived from semantic recognition or feature detection results; the weighted score of important regions is used to increase the sampling density of roads, bridges, waterlogged boundaries, landslides, collapsed buildings, and areas where people or vehicles gather to a preset range. The pose confidence score is used to reduce the interference of images with poor pose quality to a preset range; the poor pose quality includes cases of dynamic differential positioning loss of lock or visual positioning drift. The image quality score, coverage contribution score, baseline contribution score, important area weighted score, and pose reliability score are each configured with adjustable weight coefficients, which are used to adjust the proportion of each factor in the comprehensive score according to the type of on-site task and computing power configuration. Step S302: Based on the comprehensive score, retain the images with a comprehensive score higher than a preset threshold as the keyframe set.

[0049] The formula used in step S103 includes: ; in, A comprehensive scoring function for selecting keyframes; Image quality score; Contribute points to coverage; Contribute points to the baseline; Weighted scores are assigned to key regions; Posture reliability score; , , , and These are the weight parameters.

[0050] In steps S301 to S302 of this embodiment, by introducing an image quality score, low-quality images such as blurred, overexposed, underexposed, and smoke-covered images are automatically filtered out. This ensures that each frame entering the reconstruction process has clearly distinguishable texture features and stable imaging quality, avoiding contamination of 3D reconstruction by low-quality data and improving the reliability and accuracy of the reconstruction results. By introducing a coverage contribution score, images that contribute significantly to high uncertainty areas or uncovered areas are prioritized, ensuring that key images that can see data blind spots and fill model gaps are not missed, guaranteeing complete coverage of the reconstructed scene and avoiding model gaps caused by missing data. By introducing a baseline contribution score, images with significant differences in perspective from the selected keyframe set are prioritized, ensuring that the keyframe set has sufficient perspective diversity and baseline length, providing good geometric conditions for subsequent triangulation, avoiding degraded geometry problems caused by too close a perspective, and enhancing the geometric stability of 3D reconstruction. By introducing weighted scores for key areas, the sampling density of these areas—such as roads, bridges, waterlogged boundaries, landslides, collapsed buildings, and areas with high concentrations of people and vehicles—is increased. This ensures that the image coverage density of critical decision-making areas is higher than that of general areas, thereby guaranteeing the accuracy and completeness of the reconstruction of these crucial elements in the 3D model. By introducing pose reliability scores, images with poor pose quality, such as those exhibiting RTK lock-off or visual positioning drift, are suppressed or removed. This avoids 3D reconstruction drift or deformation caused by pose error propagation, ensuring that the entire reconstruction process is performed under an accurate spatial reference and guaranteeing that the final positioning accuracy meets centimeter-level requirements. Adjustable weighting coefficients are used to further enhance the accuracy. , , , and The importance of different screening factors is dynamically adjusted to achieve adaptive optimization of the keyframe screening strategy for different emergency scenarios. In areas with complex terrain, the weight of the baseline contribution score can be increased; in areas with prominent key targets, the weight of the weighted score of important areas can be increased; and when computing power is limited, the quality requirements can be appropriately relaxed to maintain the best screening effect.

[0051] In some embodiments, step S104 may include, but is not limited to, steps S401 to S405: Step S401: Perform motion reconstruction processing based on the keyframe set to obtain sparse point cloud and camera pose; Step S402: Based on the sparse point cloud computing scene bounding box, the three-dimensional space is recursively divided using an octree to obtain multiple sub-blocks; Step S403: For each sub-block, construct the camera frustum of each keyframe based on the camera pose, and select keyframes from the keyframe set that have spatial intersection with the sub-block as the set of visible keyframes of the sub-block. Step S404: Select a subset of keyframes with a baseline greater than a preset value from the keyframe set. For each selected frame, generate a pixel-level depth map using a depth prediction model. Based on the camera intrinsic parameters, back-project the pixel coordinates to the camera coordinate system to obtain three-dimensional points in the camera coordinate system. Step S405: Based on the rotation matrix and translation vector contained in the camera pose, the 3D points in the camera coordinate system are transformed to the world coordinate system to obtain the 3D points in the world coordinate system. The point cloud obtained by back-projection of multiple key frames in the key frame subset is fused, denoised and downsampled to obtain the initial point cloud.

[0052] In steps S401 to S405 of the embodiments of this application, sparse reconstruction of the keyframe set is performed through motion recovery structure processing to obtain the geometric skeleton (sparse point cloud) of the scene and the precise pose of each image. This provides basic geometric and positional information for subsequent spatial segmentation, visibility analysis, and depth prior fusion, allowing the entire reconstruction process to be performed under the same spatial reference. Adaptive spatial segmentation of the large-scale scene is performed through octree recursive partitioning, decomposing the entire disaster area scene into multiple independently processable sub-blocks. This lays the foundation for subsequent parallel reconstruction, effectively solving the memory overflow problem during large-scene reconstruction and achieving resource-controlled rapid reconstruction. Visibility analysis of each sub-block and keyframe is performed through view frustum intersection testing, establishing a mapping relationship between the sub-block and its visible keyframes. This ensures that each sub-block uses only a subset of images geometrically related to its subsequent training, avoiding interference from irrelevant data and significantly improving training efficiency and cache I / O efficiency. By performing depth prediction and backprojection, a transformation from 2D images to 3D point clouds is performed, generating dense depth prior information for large baseline keyframes. This provides rich geometric data for subsequent construction of the initial point cloud, compensating for the insufficient density of sparse point clouds in SfM. Through multi-frame point cloud fusion and post-processing, the depth prior information is integrated and optimized to obtain a globally initialized point cloud covering the entire scene. This provides a geometric starting point for 3DGS training, reducing the cold start cost of 3DGS and shortening the time to reach a visually usable state. Furthermore, this embodiment utilizes a depth prediction model (DepthAnythingV3) to generate depth priors for large baseline keyframes.

[0053] In some embodiments, step S105 may include, but is not limited to, steps S501 to S505: Step S501: Convert the initial point cloud into an initial Gaussian set of three-dimensional Gaussian sputtering, with each Gaussian point having its three-dimensional position as the mean, and assigning initial scale and color parameters. Step S502: Cut out a Gaussian subset within the spatial range of each sub-block from the initial Gaussian set, as the starting point for sub-block training; Step S503: Use a visual language model to generate a semantic map for the keyframes in the keyframe set. The semantic map contains pixel-level semantic category information. Step S504: For each sub-block, based on the set of visible keyframes of the sub-block, perform 3D Gaussian sputtering training starting from the Gaussian subset. During the training process, construct a joint loss function for optimization. The joint loss function includes photometric loss, semantic loss, and shape regularization term. Photometric loss is used to constrain the consistency between the rendered color image and the original image. Semantic loss is used to constrain the consistency between the rendered semantic probability distribution and the semantic map generated by the visual language model. Shape regularization term is used to constrain the anisotropy of Gaussian points. Step S505: Through optimization of the joint loss function, each sub-block obtains an optimized Gaussian subset. After fusing the Gaussian subsets of each sub-block and performing boundary consistency processing, a semantic 3D Gaussian sputtering model containing geometric and semantic information is obtained.

[0054] In steps S501 to S505 of the embodiments of this application, a transformation from discrete geometric point cloud to Gaussian point cloud is performed to provide a structurally complete initial state for 3D Gaussian sputtering training. Each Gaussian point carries position, scale, and color information, enabling the model to have relatively complete geometric coverage in the early stages of training and reducing the number of subsequent iterations. Through a spatial pruning mechanism, sub-blocks are allocated to the global initial Gaussian set, achieving independence and lightweighting of training data for each sub-block. This ensures that each sub-block only processes points within its own spatial range, keeping memory usage controllable and laying the foundation for parallel training while avoiding data interference across sub-blocks. A semantic graph is generated through a visual language model, and pixel-level semantic annotation is performed on keyframes, injecting understandable semantic information into the 3D model. This provides a semantic foundation for subsequent feature extraction and decision support, opening up the channel from image to 3D semantic field. By optimizing training with a joint loss function, collaborative learning of geometry and semantics is performed, enabling the 3D model to obtain accurate semantic attributes while maintaining high-precision geometry. Photometric loss ensures visual realism, semantic loss ensures semantic consistency, and shape regularization ensures geometric stability. These three factors work together to make the model clearer at semantic boundaries and more controllable in geometric form. Through sub-block fusion and boundary consistency processing, the independent optimization results of each sub-block are integrated to obtain a complete, continuous, and seamless semantic 3D scene model. This leverages the efficiency advantages of block-based parallel training while ensuring the overall consistency and measurement accuracy of the final model. Furthermore, the Visual Language Model (VLM) is a multimodal deep learning model capable of simultaneously understanding images and text, enabling tasks such as semantic description of image content, object recognition, and scene understanding. In this embodiment, VLM is used to generate pixel-level semantic maps for keyframes. 3D Gaussian sputtering (3DGS) is a novel 3D scene representation and rendering technique that uses a large number of attributed 3D Gaussian points (position, scale, rotation, color, opacity) to represent the scene. High-quality new perspective synthesis and rapid training are achieved through differentiable sputtering rendering.

[0055] In some embodiments, step S106 may include, but is not limited to, steps S601 to S603: Step S601: Based on the three-dimensional Gaussian sputtering model, structured elements are extracted in space through the three-dimensional semantic field, and the structured elements are transformed into vectorized or rasterized representations. The structured elements include road connectivity areas, water bodies or waterlogged areas, fire points and smoke-affected areas, collapsed building outlines, landslide boundaries, and locations of people or vehicles. Step S602: Based on structured elements and terrain geometry features, a risk cost map is constructed. The risk cost map divides the scene into spatial locations of impassable areas, restricted access areas, and safe access areas, and defines a comprehensive cost for each spatial location. The comprehensive cost consists of a distance or time cost item, a risk cost item, and a access barrier cost item. The risk cost item is obtained by combining semantic elements of landslide proximity, water depth, smoke coverage intensity, and high-risk areas for secondary disasters. The access barrier cost item is used to express impassable areas such as road closures, collapsed debris, or narrow roads. Step S603: Based on the risk cost map, search for the path with the minimum total cost from the starting point to the target point, and output the main route and alternative routes, and give the reasons for detours and risk explanations for each route, and generate a disaster decision report; the disaster decision report includes the affected area range, distribution of key risk points, statistics on the distribution of personnel and vehicles, road accessibility conclusions, risk level distribution heat map, recommended action routes, estimated time, and suggestions for key material delivery points.

[0056] In steps S601 to S603 of the embodiments of this application, structured elements are extracted through a three-dimensional semantic field, and a transformation from implicit neural representation to explicit quantifiable elements is performed. This transforms the three-dimensional model, originally used only for visual display, into vectorized data that can be directly used for decision analysis, providing structured input for subsequent risk assessment and route planning. Through the construction of a risk cost map, quantitative fusion of multi-source risk factors is performed, transforming the complex disaster situation into a unified metric of access cost. This allows rescue route planning to comprehensively consider distance efficiency, safety risks, and access obstacles within the same framework, providing a computable mathematical basis for subsequent intelligent decision-making. Through route search and report generation, the transformation from quantified costs to executable solutions is performed, achieving automated integration from disaster data to rescue decisions. This not only outputs the optimal route but also provides alternative solutions and risk explanations, while generating a complete report covering disaster statistics, risk distribution, and material recommendations, providing decision support products for the command center.

[0057] Furthermore, the overall process of this application is as follows: Figure 2 As shown, it mainly includes: data acquisition and quality assessment, closed-loop supplementary acquisition planning, key frame screening, octree segmentation and SfM, depth prediction initialization, semantically guided segmented 3DGS reconstruction, and agent-based decision support output.

[0058] Overall, the system uses parallel data acquisition by multiple UAVs as the data entry point, automatic data replenishment is achieved through quality assessment and coverage uncertainty estimation, keyframe filtering and octree partitioning reduce computational scale and improve parallelism, deep prior initialization shortens 3DGS cold start time, and semantic-guided training enhances interpretability and structured feature extraction capabilities. Finally, an agent directly transforms the 3D results into actionable decision-making outputs such as disaster reports, risk cost maps, and rescue routes. All steps are coupled through a unified spatiotemporal reference (timestamp alignment and geographic coordinate constraints) to ensure the output of localizable, measurable, and interpretable 3D situational awareness results within a short timeframe.

[0059] 1. Self-assessment and closed-loop data collection for missing data: The main challenge in emergency scenarios lies in the incompleteness of information during the data acquisition phase: obstructions, smoke / fog, highly reflective water surfaces, communication fluctuations, and temporary no-fly zones can create systematic blind spots on pre-defined flight paths. Relying on manual review and subsequent re-flight will significantly prolong the delivery cycle. This application introduces an uncertainty modeling method oriented towards "coverage sufficiency—geometric recoverability—image validity—task importance" during or before data acquisition. This method automatically locates areas with insufficient observations and generates re-acquisition tasks, thus forming an "evaluation-driven acquisition closed loop" to achieve one-time map output.

[0060] In implementation, the target region is discretized into two-dimensional meshes or three-dimensional voxel units, denoted as... The system for each Maintain a normalized uncertainty index This is used to characterize whether the unit needs supplementary sampling and its priority. To accommodate the biased needs of emergency operations for key areas, this application adopts a combination form with semantic weights: ; in, This is a semantic importance weight used to emphasize key elements such as roads, bridges, embankments, densely populated areas, and suspected disaster boundaries; For grid The number of effective observations This represents the set of observations available for this mesh under quality and geometric constraints. The baseline adequacy index measures whether the angle / baseline distribution between different lines of sight satisfies the geometric conditions for triangulation and surface reconstruction (the larger the value, the more adequacy the geometry). It is a comprehensive imaging quality index used to characterize factors such as sharpness, exposure, smoke, rain and fog obstruction, and motion blur (the higher the value, the better the quality). These are the weighting coefficients of the three factors. The observation count decay coefficient is used to control the number of observations. The rate at which uncertainty decreases as it increases.

[0061] when Exceeding the preset threshold When this occurs, the system marks it as a missing / high uncertainty region and proceeds to the generation of supplementary mining viewpoints and task allocation. To ensure that supplementary mining decisions are quantifiable and executable, this application denotes candidate supplementary mining viewpoints as... (Including parameters such as position, altitude, camera orientation, gimbal pitch, and shooting mode), define an information gain function to measure the viewpoint's contribution to reducing global uncertainty: ; in, A visibility function, used to characterize visibility from the viewpoint. Observation grid The effectiveness of this method can be comprehensively considered by taking into account factors such as camera field of view, maximum line-of-sight distance, occlusion detection, and incident angle limitations. Furthermore, to constrain flight time, energy consumption, and safety risks at emergency sites, this application addresses the limitations of unmanned aerial vehicles (UAVs). Define the cost function: ; in, This indicates the time cost required to perform the re-collection (including relocation, climb / descent, and flight path length). Indicates energy consumption cost; Indicates the risks and costs (no-fly zones, areas with dense obstacles, areas with strong winds, low-altitude power lines, high-risk areas for smoke and dust, etc.); These are weighting coefficients. The system selects the optimal or near-optimal supplementary sampling strategy based on these coefficients, for example: ; in, In response to drones A set of feasible viewpoints (satisfying range, communication, no-fly zones, and mission constraints). Through the above mechanism, supplementary data collection shifts from "human experience-driven" to "quantitative evaluation-driven," thereby maximizing one-time coverage and reducing the risk of subsequent model gaps under time constraints.

[0062] 2. Keyframe selection strategy: Emergency site data collection by multiple drones often generates a large number of repetitive viewpoints and low-quality images. If all of these images are used for SfM or 3DGS training, it will lead to uncontrollable computational resources and time budgets, making it difficult to meet the engineering goal of "outputting images within ≤1 hour". This application proposes a keyframe selection strategy for emergency reconstruction, which significantly reduces the input size while ensuring coverage integrity and geometric recoverability. In the embodiment, the goal of keyframe selection is to reduce the amount of original image data by about 70%, that is, to retain about 30% of the images that contribute more to reconstruction as the keyframe set.

[0063] In terms of workflow, the system first performs unified preprocessing on multi-source data, including timestamp alignment, duplicate frame removal, and removal of obviously blurred / overexposed / underexposed frames, and calculates the quality and geometric contribution features for each frame. Subsequently, a comprehensive scoring function is constructed for candidate frames. Sort and select: ; in, Image quality is scored by comprehensively evaluating factors such as sharpness, exposure, degree of obstruction by smoke and rain, and degree of motion blur, to ensure that keyframes have stable features and reliable textures. Coverage contribution score, used to measure frame New coverage for areas with high uncertainty or no coverage; Baseline contribution score, used to characterize frames The viewpoint difference and baseline length gain between the selected keyframe set are used to enhance triangulation stability and reduce degenerate geometry. Weighted scores are assigned to important areas based on rapid semantic recognition / feature detection results, which are used to increase the sampling density of areas such as roads, bridges, waterlogged boundaries, landslides, collapsed buildings, and areas where people / vehicles gather. This represents the pose confidence score, used to suppress the interference of poor pose quality images on subsequent optimization. (Coefficients) This is a weighting parameter that can be adjusted based on the type of task and computing power configuration on site.

[0064] Regarding the pose source, this application prioritizes using fixed RTK solutions (or equivalent high-precision GNSS / IMU fusion solutions) as extrinsic priors to ensure the absolute positioning accuracy of the 3D results in the geographic coordinate system. Simultaneously, it allows the introduction of visual methods (such as visual odometry / visual SLAM) to continuously reinforce and fine-tune the trajectory in areas where RTK is occluded or temporarily lost lock, thus balancing global coordinate constraints and trajectory stability in complex environments. In addition to the main set of keyframes, a subset of keyframes with larger baselines is further selected from the keyframe output for subsequent depth prediction initialization, further reducing the time consumed in the densification process of the sparse point cloud obtained by 3DGS from SFM (see point 4 below).

[0065] 3. Octree spatial partitioning and SfM acceleration: Large-scale emergency scenarios are typically characterized by large spatial scale, complex structure, and heavy local occlusion. Using a single global model for 3DGS training would result in excessively high peak memory usage and wasted training iterations in redundant regions. This application, after obtaining coarse pose and sparse point cloud in the SfM stage, uses an octree to adaptively partition the 3D space, decomposing the global problem into multiple sub-problems that can be processed in parallel or pipelined manner, thereby achieving "resource-controlled rapid reconstruction".

[0066] Specifically, the sparse point cloud output by SfM is denoted as The set of cameras is denoted as The system is based on Calculate the scene bounding box and construct an octree, then recursively partition the space into sub-blocks. The stopping criteria for partitioning can be limited by engineering parameters, such as an upper limit on the number of points in a sub-block, a threshold for the spatial scale of a sub-block, or a maximum recursion depth, to balance the number of sub-blocks with the training cost per block. For each sub-block... The system establishes its set of visible keyframes through view frustum intersection tests or projection coverage determination: ; in, A set of keyframes Representing a frame Corresponding to the camera's view frustum, This represents spatial intersection relationships. Through this mapping, each sub-block is bound only to a subset of images that are geometrically related to it, effectively reducing the amount of training data per block and improving cache I / O efficiency.

[0067] The block-based strategy offers two main benefits: first, training can run in parallel on a sub-block basis, adapting to multi-GPU or multi-process environments; second, even under single-machine, single-card conditions, a pipelined approach can be used to rebuild and fuse block by block, avoiding memory overflow and long-tail latency caused by global training. To ensure the consistency of the block-based results, this application retains necessary shared frames and shared geometric constraints in the overlapping areas of adjacent sub-blocks, and performs coordinate unification and boundary consistency processing (such as similarity transformation alignment and global fine-tuning) during the fusion stage, thereby reducing the risk of inter-block seams and drift.

[0068] 4. Initial point cloud construction based on depth prediction: While 3DGS training is highly efficient, its early initialization and densification stages are sensitive to the initial geometric distribution. Random initialization can easily lead to insufficient structural coverage, requiring numerous iterations to obtain a usable model. Furthermore, using sparse point clouds obtained through SFM for initialization results in significant time consumption during the subsequent densification process. This application utilizes a depth prediction model (Deepth Anything V3 in this example) to generate depth priors for large baseline keyframes. It then constructs an initial point cloud through backprojection and transforms it into an initial Gaussian set, thereby significantly reducing the cold start cost of 3DGS and shortening the time to reach a visually usable state.

[0069] Specifically, from the set of keyframes Selecting a large baseline subset The selection of this method can be based on metrics such as viewpoint differences, baseline length, or coverage increments to ensure more stable depth estimation and multi-view fusion. For each frame... Depth prediction yields pixel-level depth maps. , is the horizontal coordinate (column coordinate) of the pixel in the image. These are vertical coordinates (row coordinates). Let the pixel homogeneous coordinates be... Depth value The camera intrinsic parameter matrix is Then, the 3D point projected from the pixel onto the camera coordinate system is: ; Then utilize the camera extrinsic parameters (rotation matrix) With translation vector Transform to world coordinate system: ; in, A 3D point in the camera coordinate system These are 3D points in the world coordinate system. The initial point cloud is obtained by fusing multiple frames of back-projected point clouds. The fusion process may include denoising, voxel downsampling, outlier removal, and confidence screening to improve the stability of the geometric prior. Then... Convert to 3DGS initial Gaussian set: each point is used as the Gaussian mean. The initial scale can be set based on local point density or a preset scale, and the initial color can be obtained by fusing the corresponding pixel color or multi-view colors. Through this initialization method, 3DGS can obtain relatively complete structural coverage in the early stages of training, thereby reducing the number of iterations in the densification stage and improving early visualization quality.

[0070] 5. Semantic-guided 3DGS reconstruction: refer to Figure 3 Emergency decision-making demands not only "visibility" of 3D results, but also "interpretability, extractability, and quantifiability." Reconstruction based solely on photometric consistency is prone to ambiguity or drift at semantic boundaries (e.g., the intersection of roads and floodwaters, smoke-covered boundaries, and collapsed building cross-sections), thus affecting the reliability of feature extraction and measurement. This application introduces a Visual Language Model (VLM) to generate semantic maps from keyframes and integrates semantic constraints and shape regularization during 3DGS training, making the model more stable at semantic boundaries and more controllable in geometric form, thereby improving the usability of downstream disaster analysis and route planning.

[0071] The semantic category set in this embodiment includes at least nine key elements: roads, water bodies, vegetation, buildings, people, vehicles, smoke, fire, and bare ground (or exposed ground). For each frame... VLM outputs a semantic graph. Its form can be either a category ID or a category probability distribution. During 3DGS training, from the perspective... Rendering results in a color map The semantic rendering result is obtained by accumulating the semantic distribution carried by Gaussian during rendering. The joint loss function is defined as: ; in, This is a photometric loss, used to ensure that the rendered colors are consistent with the real image: ; Represents the horizontal and vertical coordinates of the image. is the horizontal coordinate (column coordinate) of the pixel in the image. It is the vertical coordinate (row coordinate). These are the actual image pixel values. The pixel values ​​for rendering the image. For semantic loss, used to constrain semantic consistency, cross-entropy is employed (one example): ; in, For semantic category indexing, For true semantics in categories The probability of one-hot encoding. To render semantic probabilities. This is a shape regularization used to suppress excessive anisotropy in Gaussian shapes, thereby improving geometric stability and measurement consistency. Let the first... The scale parameters of the Gaussian along the three principal axes are: Then anisotropy ratio constraints can be used: ; in, and The weighting coefficients are used to balance the influence of semantic and shape constraints on photometric fitting. Through the above joint optimization, the model can maintain a more stable structural expression under the conditions of weak texture, occlusion, and semi-transparent interference commonly encountered in emergency scenarios, and provide a consistent semantic-geometric correspondence for subsequent 3D semantic element extraction.

[0072] 6. Agent-based emergency decision support: To shorten the path from "model production" to "command and decision-making," this application, after completing the 3D reconstruction, further transforms the semantic 3DGS results into a structured set of elements, which is then handed over to an intelligent agent to perform comprehensive emergency-oriented analysis and automated output. Unlike the traditional "manual analysis switching between multiple systems," this application emphasizes traceable evidence chains: each analytical conclusion can be linked to its 3D spatial location, corresponding semantic category, and key perspective images or rendering results supporting the conclusion, facilitating rapid verification and reuse by the command center.

[0073] In terms of element structuring, the system utilizes a 3D semantic field to extract elements such as road connectivity areas, water / waterlogged areas, fire and smoke-affected zones, collapsed building outlines, suspected landslide boundaries, and personnel / vehicle locations in space, generating vectorized or rasterized representations to provide input for path planning and statistical analysis. Subsequently, a risk cost map is constructed, and the raster or nodes are... Define the overall cost: ; in, This is a distance or time cost term used to guide the minimization of route length; The risk cost item can be obtained by combining semantic elements such as landslide proximity, water depth, smoke coverage intensity, and high-risk slopes for secondary disasters with topographic information; This is a cost item for obstacles to passage, used to express road closures, collapsed debris, narrow roads, or impassable areas; The weighting coefficients can be dynamically adjusted based on the rescue method (walking, vehicle, boat) and mission strategy (speed priority or safety priority). For path planning, algorithms such as A* or Dijkstra can be used to search for the path with the minimum total cost, while simultaneously outputting the main route and alternative routes, along with explanations of the reasons for detours and the associated risks.

[0074] In terms of deliverables, the Agent automatically generates disaster reports (including the affected area, key risk points, personnel and vehicle distribution, road accessibility conclusions, etc.) and outputs 3D browsable results (supporting positioning and measurement), route overlay results, and 3DGS roaming rendering videos along the routes to meet the command center's needs for rapid browsing, reporting, and dispatching. Through this closed loop, the system connects "one-time emergency site data collection—rapid 3D reconstruction—semantic element extraction—automatic decision output" into a unified chain, thereby providing actionable decision support products under strict time constraints.

[0075] Figure 4 This is a schematic diagram of the disaster response decision support process based on VLM Agent in this application embodiment. The specific processing flow includes the following four key steps: Step 1: Virtual View Rendering and Multidimensional Projection. The system first receives a semantic 3DGS model as the data base and, based on the rescue mission instructions (such as "finding the optimal path to the school"), drives the rendering engine to generate RGB images, depth maps, and semantic masks from a specific viewpoint. This step discretizes the continuous three-dimensional Gaussian field into a two-dimensional feature map that can be understood by the VLM.

[0076] Step 2: VLM Visual Reasoning and Scene Understanding. The VLM Agent receives the multi-dimensional image data mentioned above and, combined with the contextual understanding capabilities of the visual language model, automatically labels risk sources such as collapsed buildings, flooded areas, and road cracks, as well as identifies key elements such as passable areas.

[0077] Step 3: Risk Cost Map Construction and Route Planning. Based on the identified risk factors and terrain geometry, the agent divides the scene into impassable areas (high risk), restricted areas (medium risk), and safe areas, constructing a weighted risk cost map. Point and line nodes are created for the traversable areas, forming a topology map. Subsequently, a route planning algorithm (such as an improved A or RRT algorithm) is invoked to search the cost map for the optimal rescue route that balances safety and timeliness.

[0078] Step 4: Multimodal Information Fusion and Report Generation. Finally, the Agent fuses quantified statistical data, planned route vector information, and risk heatmaps using a multimodal approach to automatically generate a structured disaster decision-making report. The report includes risk level distribution, recommended action routes, estimated timeframes, and suggestions for key material delivery points, providing commanders with directly actionable decision-making support.

[0079] Key points of this application: 1. A closed-loop re-imaging method based on uncertainty assessment: By calculating uncertainty indicators of coverage, viewpoint baseline, and semantic importance in real time, the method autonomously generates and executes re-imaging tasks, achieving one-time full-coverage mapping for emergency scenarios. This reduces the manpower and time costs of subsequent manual re-imaging.

[0080] 2. A key frame selection strategy for emergency reconstruction: By combining image quality, geometric baseline contribution and semantic weight, the amount of original data is reduced by about 70%, which reduces the computational burden while ensuring the reconstruction accuracy of key areas.

[0081] 3. A fast reconstruction method combining octree segmentation and depth prior: The scene is segmented using an octree, and the Depth Anything V3 depth prediction model is used to generate accurate and scaled depth point clouds for 3DGS point cloud initialization, which significantly improves reconstruction speed and densification efficiency.

[0082] 4. A semantically guided 3DGS training and decision support system: Semantic and shape regularization constraints are introduced into the reconstruction loss function to support accurate segmentation of elements such as roads, water bodies, and people, and based on this, disaster reports and rescue routes are automatically generated by the Agent.

[0083] Advantages of this application: 1. Significantly improved timeliness: By leveraging keyframe filtering (reducing data volume by 70%) and block-based parallel processing, combined with deep prediction initialization, the time from data back to delivery is compressed to less than 1 hour, which is much faster than traditional SfM / Mesh processes.

[0084] 2. High degree of automation: It achieves a closed loop from "blind spot detection" to "automatic supplementary sampling", avoiding the costs of manual verification and secondary attendance; at the same time, it realizes the automatic generation from "model" to "decision report".

[0085] 3. Balancing accuracy and usability: While ensuring that the positioning error is less than 1 meter and the measurement accuracy reaches the centimeter level, the 3D model is endowed with rich semantic information (such as distinguishing between water accumulation and road surface), so that it can be directly used for rescue decision-making, rather than just for display purposes.

[0086] This application provides alternative solutions, including: 1. Alternatives to depth estimation models: Depth Anything V3 can be replaced with other high-performance monocular depth estimation networks, or initialized using disparity maps from binocular cameras.

[0087] 2. Alternatives to the block partitioning strategy: Octree space partitioning can be replaced by K-Means-based clustering partitioning or adaptive partitioning based on road network topology.

[0088] 3. Replacement of semantic acquisition methods: The large VLM model can be replaced by a lightweight real-time semantic segmentation network (such as BiSeNet) to further adapt to edge computing devices.

[0089] This application also provides a multi-UAV rapid 3D reconstruction and decision support system for disaster sites, used to implement the aforementioned method. The system includes: The first module is used to collect image data of disaster sites using multiple UAVs in collaboration. The target area is discretized into grid cells, and an uncertainty index is constructed to characterize the degree of data missing based on the observation coverage, imaging quality and semantic importance of each grid cell. The second module is used to take uncertainty indicators as input, identify high uncertainty areas where the uncertainty indicators exceed a preset threshold, and select the optimal viewpoint for high uncertainty areas based on the contribution of candidate viewpoints to reducing uncertainty and the comprehensive cost of executing the viewpoint, generate a supplementary acquisition task and execute it, forming an evaluation-driven acquisition closed loop, and obtain an image dataset with complete coverage. The third module is used to select a set of keyframes for 3D reconstruction from the image dataset by comprehensively considering image quality and coverage contribution. The fourth module is used to perform motion reconstruction and spatial segmentation based on the keyframe set, and to fuse depth prior and semantic information to obtain the initial point cloud; The fifth module is used to construct a semantic 3D Gaussian sputtering model based on the initialized point cloud; The sixth module is used to extract structured elements based on the three-dimensional Gaussian sputtering model, construct a risk cost map, and obtain rescue routes and disaster decision reports.

[0090] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0091] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0092] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0093] Another embodiment of the hardware structure of the electronic device, the electronic device including: The processor can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to achieve the technical solutions provided in the embodiments of this application. The memory can be implemented in the form of read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory and called by the processor to execute the methods described in the embodiments of this application. Input / output interfaces are used to implement information input and output; The communication interface is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). A bus is used to transfer information between various components of a device, such as processors, memory, input / output interfaces, and communication interfaces. The processor, memory, input / output interfaces, and communication interfaces communicate with each other within the device via a bus.

[0094] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0095] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0096] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0097] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0098] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0099] This application provides a method, system, electronic device, storage medium, and program product for rapid 3D reconstruction and decision support using multiple unmanned aerial vehicles (UAVs) at disaster sites. It aims to solve the problems of slow mapping, voids, large data redundancy, and the disconnect between 3D models and decision-making applications in traditional emergency rescue scenarios. This system can be applied to earthquake rescue, landslide monitoring, flood disaster assessment, and urban emergency command, and is particularly suitable for scenarios with high timeliness requirements (hourly levels), complex environments, and the need for high-precision 3D data to support decision-making.

[0100] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0101] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0102] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0103] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0104] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0105] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0106] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0107] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0108] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0109] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for rapid 3D reconstruction and decision support using multiple unmanned aerial vehicles (UAVs) at disaster sites, characterized in that... The method includes the following steps: Multiple drones are used to collaboratively collect image data of disaster sites. The target area is discretized into grid cells. Based on the observation coverage, imaging quality and semantic importance of each grid cell, an uncertainty index is constructed to characterize the degree of data missing. Using the uncertainty index as input, high uncertainty regions where the uncertainty index exceeds a preset threshold are identified. For the high uncertainty regions, based on the contribution of candidate viewpoints to reducing uncertainty and the comprehensive cost of executing the viewpoint, the optimal viewpoint is selected to generate a supplementary acquisition task and execute it, forming an evaluation-driven acquisition closed loop to obtain an image dataset with complete coverage. From the image dataset, a set of keyframes for 3D reconstruction is selected by considering image quality and coverage contribution. Based on the set of keyframes, motion reconstruction structure and spatial segmentation are performed, and depth prior and semantic information are fused to obtain the initial point cloud; A semantic 3D Gaussian sputtering model is constructed based on the initialized point cloud; Based on the aforementioned three-dimensional Gaussian sputtering model, structured elements are extracted, a risk cost map is constructed, and rescue routes and disaster decision reports are obtained.

2. The method according to claim 1, characterized in that, The method of using multiple UAVs to collaboratively collect image data of the disaster site discretizes the target area into grid cells, and constructs uncertainty indicators to characterize the degree of data missing based on the observation coverage, imaging quality, and semantic importance of each grid cell, including: Multiple UAVs are used to collaboratively collect image data of the disaster site. The target area is discretized into several grid cells. For each grid cell, a normalized uncertainty index is constructed based on the semantic importance of the grid cell, the number of effective observations, the sufficiency of the viewpoint baseline, and the overall imaging quality. The uncertainty index is used to quantify the degree of data loss and the priority of data re-collection for the grid cell. The calculation of the uncertainty index includes: multiplying the weight of the semantic importance by the result obtained by weighted summation of the observation count attenuation term, the viewpoint baseline deficiency term, and the imaging quality deficiency term; The weights of semantic importance are used to characterize the priority of roads, bridges, embankments, densely populated areas, and disaster boundaries that meet preset conditions; The observation count decay term is calculated exponentially based on the effective observation count; the observation count decay term is used to characterize the rate at which uncertainty decreases as the number of observations increases; the effective observation count is the number of observation sets available for the grid cell under quality and geometric constraints; The insufficient angle baseline term is obtained by subtracting the angle baseline sufficiency index; the angle baseline sufficiency index is used to measure whether the included angle between different observation lines and the baseline distribution meet the geometric conditions for triangulation and surface reconstruction. The image quality deficiencies are obtained by subtracting the comprehensive image quality index; the comprehensive image quality index is used to characterize the impact of sharpness, exposure, smoke, rain and fog obstruction and motion blur on image quality. The observation count attenuation term is configured with a first weighting coefficient; the first weighting coefficient is used to adjust the proportion of the observation count attenuation term in the uncertainty index according to the type of on-site task. The insufficient viewpoint baseline is configured with a second weighting coefficient; the second weighting coefficient is used to adjust the proportion of the insufficient viewpoint baseline in the uncertainty index according to the type of on-site task. The imaging quality deficiency item is configured with a third weighting coefficient; the third weighting coefficient is used to adjust the proportion of the imaging quality deficiency item in the uncertainty index according to the type of on-site task.

3. The method according to claim 1, characterized in that, The process involves using the uncertainty index as input to identify high-uncertainty regions where the uncertainty index exceeds a preset threshold. For these high-uncertainty regions, based on the contribution of candidate viewpoints to reducing uncertainty and the overall cost of executing the viewpoint, the optimal viewpoint is selected to generate and execute a supplementary acquisition task, forming an evaluation-driven acquisition closed loop. This results in an image dataset with complete coverage, including: Using the uncertainty index as input, high uncertainty regions where the uncertainty index exceeds a preset threshold are identified; the high uncertainty regions are areas with missing or insufficient data coverage. For the high uncertainty region, based on the contribution of candidate viewpoints to reducing uncertainty and the comprehensive cost of executing the candidate viewpoints, the optimal viewpoint is selected to generate a supplementary sampling task and execute it; the contribution of the candidate viewpoints to reducing uncertainty is measured by the information gain function. The information gain function is calculated by summing the products of the uncertainty index and the visibility function of all grid cells observable from the candidate viewpoint; the information gain function is used to quantify the total amount of uncertainty eliminated by resampling from the candidate viewpoint; The comprehensive cost is a weighted sum of time cost, energy cost, and risk cost; the time cost is used to characterize the time required for the transfer time, climb time, descent time, and flight path length for performing the replenishment mission; the energy cost is used to characterize the battery consumption required for performing the replenishment mission; and the risk cost is used to characterize the safety risks faced in performing the replenishment mission, such as no-fly zones, areas with dense obstacles, areas with strong winds, low-altitude cables, and high-risk areas of smoke and dust. The conditions for selecting the optimal viewpoint include: in the set of feasible viewpoints that meet the preset constraints of UAV range, communication, no-fly zone and mission, the candidate viewpoint with the largest difference between the information gain calculated by the information gain function and the comprehensive cost is selected as the optimal viewpoint. After the supplementary acquisition task is performed, the newly acquired image data is fused with the original image data to obtain an image dataset with complete coverage.

4. The method according to claim 1, characterized in that, The step of selecting a set of keyframes for 3D reconstruction from the image dataset, considering factors such as image quality and coverage contribution, includes: From the image dataset, each frame of the image is comprehensively scored, and a set of keyframes for 3D reconstruction is selected based on the comprehensive score. The comprehensive score is obtained by weighted summation of image quality score, coverage contribution score, baseline contribution score, important area weighted score, and pose confidence score; The image quality score is used to comprehensively measure the image's sharpness, exposure, degree of obstruction by smoke, rain, and fog, and degree of motion blur. The coverage contribution score is used to measure the amount of new coverage that the image frame adds to the high uncertainty area or the uncovered area; The baseline contribution score is used to characterize the viewpoint difference and baseline length gain between the image frame and the selected set of keyframes; The weighted score of important regions is derived from semantic recognition or feature detection results; the weighted score of important regions is used to increase the sampling density of roads, bridges, waterlogged boundaries, landslides, collapsed buildings, and areas where people or vehicles gather to a preset range. The pose confidence score is used to reduce the interference of images with poor pose quality to a preset range; the poor pose quality includes cases of dynamic differential positioning loss of lock or visual positioning drift. The image quality score, coverage contribution score, baseline contribution score, important area weighted score, and pose reliability score are each configured with adjustable weight coefficients, which are used to adjust the proportion of each factor in the comprehensive score according to the type of on-site task and computing power configuration. Based on the comprehensive score, images with a comprehensive score higher than a preset threshold are retained as the keyframe set.

5. The method according to claim 1, characterized in that, The process of performing motion reconstruction and spatial segmentation based on the keyframe set, fusing depth prior and semantic information to obtain an initial point cloud, includes: Motion recovery structure processing is performed based on the keyframe set to obtain sparse point cloud and camera pose. Based on the sparse point cloud computing scene bounding box, an octree is used to recursively divide the three-dimensional space to obtain multiple sub-blocks. For each sub-block, a camera frustum for each keyframe is constructed based on the camera pose. Keyframes in which the camera frustum and the sub-block have spatial intersections are selected from the keyframe set and used as the set of visible keyframes for the sub-block. Select a subset of keyframes with a baseline greater than a preset value from the set of keyframes. For each selected frame, use a depth prediction model to generate a pixel-level depth map. Based on the camera intrinsic parameters, back-project the pixel coordinates to the camera coordinate system to obtain three-dimensional points in the camera coordinate system. Based on the rotation matrix and translation vector contained in the camera pose, the 3D points in the camera coordinate system are transformed to the world coordinate system to obtain the 3D points in the world coordinate system. The point cloud obtained by back-projection of multiple key frames in the key frame subset is fused, denoised and downsampled to obtain the initial point cloud.

6. The method according to claim 5, characterized in that, The construction of a semantic 3D Gaussian sputtering model based on the initialized point cloud includes: The initial point cloud is converted into an initial Gaussian set of three-dimensional Gaussian sputtering, with each Gaussian point having its three-dimensional position as the mean and being assigned initial scale and color parameters. A subset of Gaussian elements within the spatial range of each sub-block is cropped from the initial Gaussian set and used as the starting point for training the sub-block. A semantic map is generated from the keyframes in the keyframe set using a visual language model. The semantic map contains pixel-level semantic category information. For each sub-block, based on the set of visible keyframes of the sub-block, 3D Gaussian sputtering training is performed starting from the Gaussian subset. During training, a joint loss function is constructed for optimization. The joint loss function includes photometric loss, semantic loss, and shape regularization term. The photometric loss is used to constrain the consistency between the rendered color image and the original image. The semantic loss is used to constrain the consistency between the rendered semantic probability distribution and the semantic map generated by the visual language model. The shape regularization term is used to constrain the anisotropy of the Gaussian points. By optimizing the joint loss function, each sub-block obtains an optimized Gaussian subset. After fusing the Gaussian subsets of each sub-block and performing boundary consistency processing, a semantic 3D Gaussian sputtering model containing geometric and semantic information is obtained.

7. The method according to claim 1, characterized in that, The process of extracting structured elements based on the three-dimensional Gaussian sputtering model, constructing a risk-cost map, and obtaining rescue routes and disaster decision-making reports includes: Based on the aforementioned three-dimensional Gaussian sputtering model, structured elements are extracted in space through a three-dimensional semantic field, and these structured elements are transformed into vectorized or rasterized representations. The structured elements include road connectivity areas, water bodies or waterlogged areas, fire points and smoke-affected areas, collapsed building outlines, landslide boundaries, and locations of people or vehicles. Based on the structured elements and terrain geometry, a risk cost map is constructed. The risk cost map divides the scene into spatial locations of impassable areas, restricted access areas, and safe access areas, and defines a comprehensive cost for each spatial location. The comprehensive cost consists of a distance or time cost item, a risk cost item, and a access barrier cost item. The risk cost item is obtained by integrating semantic elements such as landslide proximity, water depth, smoke coverage intensity, and high-risk areas for secondary disasters. The access barrier cost item is used to express impassable areas such as road closures, collapsed debris, or roads that are too narrow. Based on the risk cost map, the system searches for the path with the minimum total cost from the starting point to the target point, outputs the main route and alternative routes, and provides the reasons for detours and risk explanations for each route, generating a disaster decision report. The disaster decision report includes the affected area range, distribution of key risk points, statistics on the distribution of personnel and vehicles, road accessibility conclusions, risk level distribution heat map, recommended action routes, estimated travel time, and suggestions for key material delivery points.

8. A rapid 3D reconstruction and decision support system for multi-UAVs at disaster sites, used to implement the method as described in any one of claims 1 to 7, characterized in that, The system includes: The first module is used to collect image data of disaster sites using multiple UAVs in collaboration, discretize the target area into grid cells, and construct an uncertainty index to characterize the degree of data missing based on the observation coverage, imaging quality and semantic importance of each grid cell. The second module is used to take the uncertainty index as input, identify high uncertainty areas where the uncertainty index exceeds a preset threshold, and select the optimal viewpoint to generate a supplementary acquisition task and execute it based on the contribution of candidate viewpoints to reducing uncertainty and the comprehensive cost of executing the viewpoint for the high uncertainty area, thus forming an evaluation-driven acquisition closed loop and obtaining an image dataset with complete coverage. The third module is used to select a set of keyframes for 3D reconstruction from the image dataset by comprehensively considering image quality and coverage contribution. The fourth module is used to perform motion reconstruction and spatial segmentation based on the keyframe set, and to fuse depth prior and semantic information to obtain an initial point cloud; The fifth module is used to construct a semantic 3D Gaussian sputtering model based on the initialized point cloud; The sixth module is used to extract structured elements based on the three-dimensional Gaussian sputtering model, construct a risk cost map, and obtain rescue routes and disaster decision reports.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Citation Information

Patent Citations

  • Large-scene three-dimensional reconstruction method based on three-dimensional Gaussian sputtering

    CN120472121A

  • Multi-modal data fusion for enhanced 3D perception for platforms

    US20200184718A1

Cited By

  • Video linkage emergency rescue management method based on GIS multi-modal technology

    CN122175301A

  • An unmanned aerial vehicle based method and system for autonomous reconstruction of an incident scene

    CN122347778A