Unmanned aerial vehicle cluster forest fire scene three-dimensional situation construction and updating method and system

Abstract

The invention provides an unmanned aerial vehicle cluster forest fire scene three-dimensional situation construction and updating method and system, and the method comprises the steps: receiving real-time fire scene data from an unmanned aerial vehicle cluster, and building an initial fire scene three-dimensional situation model based on a UNet network and a digital elevation model; the fire scene three-dimensional situation model comprises a fire scene three-dimensional situation map and a pyramid hierarchical structure; when a preset updating period is reached, constructing a pyramid hierarchical structure of the key updating area based on the real-time fire scene data; according to the pyramid hierarchical structure of the key updating area, the pyramid hierarchical structure in the to-be-updated fire scene three-dimensional situation model is compressed based on space-time analysis driving of information entropy, then robust registration fusion is carried out, and the fire scene three-dimensional situation model is updated.

Description

Technical Field

[0001] This invention belongs to the field of forest fire monitoring, specifically involving a method and system for constructing and updating three-dimensional situation of forest fires using unmanned aerial vehicle (UAV) swarms. Background Technology

[0002] Existing forest fire monitoring methods primarily rely on satellite remote sensing, drone patrols, and ground-based detection. However, due to complex forest terrain, vegetation obstruction, and rapid fire development, a single sensor often struggles to acquire complete and continuous three-dimensional fire information. Traditional fire monitoring systems mostly employ two-dimensional maps or simple three-dimensional modeling, lacking the ability to promptly perceive dynamic fire evolution and failing to meet the needs of precise firefighting and prevention. While some existing three-dimensional reconstruction methods can generate fire models, they often cannot be updated in real time and are susceptible to sensor drift, data redundancy, and obstruction, resulting in insufficient model accuracy and timeliness. Furthermore, dense smoke, clouds, or nighttime conditions can affect the observation performance of optical and infrared sensors, making fireline detection even more difficult. Data fusion from drone swarm collaborative collection is challenging, requiring effective data stitching and update strategies, for which existing technologies have not provided targeted solutions. In addition, the storage and real-time processing of massive amounts of sensor data pose challenges to computing resources, necessitating parallel and incremental algorithms to improve efficiency.

[0003] Therefore, there is an urgent need for a new technical solution that can efficiently construct and continuously update the overall three-dimensional situation of forest fires under complex terrain and rapidly changing fire conditions, so as to provide accurate spatial information support for command, dispatch and rescue. Summary of the Invention

[0004] To overcome the limitations of existing 3D fire reconstruction methods, which cannot be updated in real time and are susceptible to problems such as sensor drift, data redundancy, and occlusion, resulting in insufficient model accuracy and timeliness, this invention provides a method and system for constructing and updating 3D fire situation maps using UAV swarms. By constructing a pyramid-level hierarchical structure and its constraint mechanism, it achieves a balance between macroscopic coherence and local precision, enabling rapid construction of 3D fire situation maps while providing complex modeling of key areas. Through an entropy-driven compression strategy and a deep learning registration algorithm, it improves the timeliness and robustness of data processing, achieving spatiotemporal optimization and iterative incremental modeling, significantly improving the accuracy and efficiency of 3D fire reconstruction.

[0005] According to one aspect of the present invention, a method for constructing and updating a three-dimensional situation of a forest fire by a swarm of unmanned aerial vehicles (UAVs) is provided, comprising:

[0006] The system receives real-time fire data from a drone swarm and establishes an initial three-dimensional fire situation model based on the UNet network and digital elevation model. The three-dimensional fire situation model includes a three-dimensional fire situation map and a pyramid hierarchical structure.

[0007] When the preset update cycle is reached, a pyramid-shaped hierarchical structure of key update areas is constructed based on real-time fire data;

[0008] Based on the pyramid hierarchy structure of the key update regions, the pyramid hierarchy structure in the three-dimensional fire situation model to be updated is compressed using spatiotemporal analysis driven by information entropy. Then, robust registration and fusion are performed to update the three-dimensional fire situation model.

[0009] As a further technical solution, it also includes: fine-tuning the pose and time information in real-time fire data based on soft spatiotemporal constraints, and aligning and stitching the local point clouds between consecutive frames.

[0010] As a further technical solution, the steps for establishing an initial three-dimensional fire situation map based on the UNet network and digital elevation model include:

[0011] Fire scene video frames under the same time information are input into the UNet network for fire line segmentation to obtain the fire line segmentation mask. The fire line segmentation mask and point cloud are back-projected onto the same three-dimensional voxel mesh using the fine-tuned pose information.

[0012] A pyramid hierarchical structure is constructed based on the three-dimensional voxel mesh obtained by back projection.

[0013] The pyramid-shaped hierarchical structure is projected back into the digital elevation model to generate a three-dimensional instantaneous combustion mask. Multi-source observation information is then recorded and weighted by confidence level to obtain an initial three-dimensional fire situation map.

[0014] As a further technical solution, the pyramid hierarchical structure consists of low-resolution voxels and their corresponding high-resolution patches, as well as virtual edges connecting the two; among them, the low-resolution voxels and high-resolution patches are optimized for cross-layer consistency by introducing virtual edges and spatiotemporal consistency constraints.

[0015] As a further technical solution, the steps for reconstructing the pyramid hierarchy structure of the key update area include:

[0016] Project the key update area onto the low-resolution layer of the 3D fire situation map and mark all low-resolution voxels to be reconstructed.

[0017] The low-resolution voxels are adaptively divided into small blocks according to the target resolution. The multi-source point clouds in each small block are denoised, coarsely registered, locally aligned, and the point cloud is completed to reconstruct them. In this way, all low-resolution voxels are reconstructed into high-resolution patches.

[0018] Calculate and save the point cloud feature statistics of all high-resolution patches as virtual edges, and connect the low-resolution voxels and high-resolution patches with corresponding virtual edges to obtain the pyramid-level structure of the key update region.

[0019] As a further technical solution, the steps of compressing the pyramid hierarchy structure in the three-dimensional fire situation model to be updated based on information entropy spatiotemporal analysis include:

[0020] The pyramid hierarchy structure in the updated 3D fire situation model is partitioned, and the pyramid hierarchy structure of the key updated area is integrated into the corresponding partition as new observation. The information entropy of each partition and the observation time difference with the new observation are calculated. The thresholds for information entropy and observation time difference are preset. Partitions with information entropy values ​​that are consistently below the threshold or observation time differences that exceed the threshold are compressed, and the latest observation is retained.

[0021] As a further technical solution, the steps for robustly registering and fusing the compressed 3D fire situation map with the pyramid-level structure of key updated areas include:

[0022] Obtain the original real-time fire data corresponding to the pyramid-level structure of the key update area, and use a pre-trained multimodal feature extraction network to process and extract geometric descriptors, appearance features and semantic features.

[0023] Geometric descriptors are used to perform feature matching between the pyramid hierarchy structure of the key update region and the compressed 3D fire situation map to be updated, so as to obtain the initial alignment.

[0024] The initial alignment is combined with appearance features, semantic features, and temporal confidence to jointly optimize and obtain the updated three-dimensional fire situation map.

[0025] According to another aspect of this specification, a system for constructing and updating a three-dimensional situation of a forest fire using unmanned aerial vehicle (UAV) swarms is provided, including:

[0026] The fire scene 3D situation model construction module is used to receive real-time fire scene data from the UAV cluster and establish an initial fire scene 3D situation model based on the UNet network and digital elevation model; the fire scene 3D situation model includes a fire scene 3D situation map and a pyramid hierarchical structure;

[0027] The update information acquisition module is used to construct a pyramid-shaped hierarchical structure of key update areas based on real-time fire data when the preset update cycle is reached.

[0028] The fire scene 3D situation model update module is used to compress the pyramid hierarchy structure in the fire scene 3D situation model to be updated based on the pyramid hierarchy structure of the key update area and driven by spatiotemporal analysis based on information entropy. Then, it is robustly registered and fused to update the fire scene 3D situation model.

[0029] According to another aspect of this specification, an electronic device is provided, including a memory and a processor, wherein the memory stores program instructions that are executed by the processor, and the processor invokes the program instructions to execute a method for constructing and updating a three-dimensional situation of a forest fire by a drone swarm.

[0030] According to another aspect of this specification, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium storing computer instructions that cause the computer to execute a method for constructing and updating a three-dimensional situation of a forest fire caused by a drone swarm.

[0031] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention divides the overall process into three stages: "rapid situation construction - global multi-scale optimization - real-time situation update". It can achieve high-precision, real-time construction and updating of the three-dimensional situation of forest fires under complex terrain and dynamic fire conditions, realize timely perception of the three-dimensional situation of the fire under complex terrain and dynamic fire conditions, and provide reliable decision support for fire emergency command. It has the following advantages and significant effects:

[0032] (1) This invention achieves rapid construction of a three-dimensional fire situation map that balances macroscopic coherence and local precision by constructing a pyramid hierarchical structure and its constraint mechanism;

[0033] (2) Based on the soft spatiotemporal constraint fine-tuning, the present invention obtains the pose and time information in the fire field data in real time, constructs a unified time and space alignment benchmark, and realizes the accurate fusion of multi-source data obtained by UAV clusters;

[0034] (3) By constructing a pyramid-shaped hierarchical structure for key update regions, the number of new observations in streaming data is reduced, the acquisition of redundant data is reduced, and lower computational costs are achieved.

[0035] (4) By using entropy-driven compression strategy and deep learning registration algorithm, the timeliness and robustness of data processing are improved, realizing spatiotemporal optimization and iterative incremental modeling of fire three-dimensional situation map, which significantly improves the accuracy and efficiency of fire three-dimensional reconstruction. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 A flowchart illustrating the method for constructing and updating a three-dimensional situation of a forest fire using a drone swarm, as provided in an embodiment of the present invention.

[0038] Figure 2 This is a schematic diagram of the structure of the UAV swarm forest fire three-dimensional situation construction and updating system provided in an embodiment of the present invention;

[0039] Figure 3 A schematic diagram illustrating the workflow of an optional unmanned aerial vehicle (UAV) swarm forest fire three-dimensional situation construction and updating system provided in an embodiment of the present invention;

[0040] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0041] It should be noted that:

[0042] The terms “comprising” and “having”, and any variations thereof, in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, product, or apparatus that includes a series of steps or units, not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0043] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices. The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be decomposed, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form new technical solutions. Such combinations are not bound by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0045] like Figure 1 As shown, the method for constructing and updating the three-dimensional situation of a forest fire using drone swarms includes:

[0046] Step 1: Receive real-time fire data from the UAV cluster and establish an initial three-dimensional fire situation model based on the UNet network and digital elevation model; the three-dimensional fire situation model includes a three-dimensional fire situation map and a pyramid hierarchical structure.

[0047] Step 2: When the preset update cycle is reached, construct a pyramid-shaped hierarchical structure of key update areas based on real-time fire data;

[0048] Step 3: Based on the pyramid hierarchy structure of the key update area, compress the pyramid hierarchy structure in the three-dimensional fire situation model to be updated using spatiotemporal analysis driven by information entropy, and then perform robust registration and fusion to update the three-dimensional fire situation model.

[0049] Furthermore, the method also includes: fine-tuning the pose and time information in real-time fire data based on soft spatiotemporal constraints, and aligning and stitching the local point clouds between consecutive frames.

[0050] The fine-tuning steps include:

[0051] Step (a): Calculate the visual overlap index of adjacent frames based on pose information, and take the corresponding adjacent frames that exceed the preset threshold level as stitching candidate pairs;

[0052] Step (b): Construct a weighted spatiotemporal consistency error function based on soft spatiotemporal constraints as the optimization objective. Within the sliding time window, use the pose correction and time deviation of each frame in the candidate splicing pair as independent variables to solve and output the fine-tuned pose and time information.

[0053] Specifically, in step 1, each drone in the cluster carries sensors such as lidar, camera, infrared, INS / GNSS, and LiDAR; real-time fire scene data includes, but is not limited to, fire scene images, point clouds, and pose and time information of the captured images.

[0054] Preferably, before fine-tuning based on soft spatiotemporal constraints, time alignment and DEM correction are also included. By synchronizing the time and correcting the pose of sensor data from multiple UAVs, the flight paths of each UAV are matched with the pre-acquired digital elevation model (DEM), suppressing positioning deviations and attitude drift caused by terrain occlusion; clock synchronization is used to ensure the time alignment of different sensor data frames, eliminating time inconsistencies caused by clock drift, communication delays, or frame loss, and ensuring that subsequent spatial registration and stitching are based on high-quality timing.

[0055] Specifically, time alignment is manifested in clock synchronization and inter-frame alignment. Hardware or software-level clock synchronization schemes ensure that the timestamps of all sensor data are precisely aligned under the same unified clock. Specific measures include: when hard synchronization is available, using GNSS-PPS or PTP to lock the system clock of airborne sensors, with a target frame timestamp error of <50ms. For sensors that cannot be hard synchronized, cross-correlation curves are calculated based on event sequences (such as infrared temperature abrupt changes, laser echo peaks, and feature changes in images). Maximum cross-correlation is used to determine time offset. Linear / spline interpolation is performed on pose and frame data using a unified reference time axis to generate millisecond-level aligned frame pairs. Anomalies such as frame loss or out-of-order processing caused by communication delays or buffer reordering are detected and automatically compensated for. During time alignment, the original timestamp, corrected timestamp, and confidence level of each frame are recorded and written into the time alignment metadata.

[0056] DEM correction is manifested as track-DEM matching correction. Utilizing an existing Digital Elevation Model (DEM) as a terrain prior, it establishes a spatial correspondence between the UAV track / pose and the DEM, correcting the track's altitude reference and vertical drift. This provides a spatially consistent pose sequence suitable for projection, serving as the DEM-corrected pose information and providing an accurate reference for subsequent overlap determination, coarse registration, and projection onto the DEM. Specific measures include: projecting the initial UAV track output from GNSS / INS onto the plane coordinate system used by the DEM to ensure consistent vertical references. Optionally, a local ENU / UTM plane coordinate system can be used. Preferably, if different vertical references exist, a vertical reference transformation is required first. Based on the track points and the grid features of the DEM (such as ridges, valley lines, or known landmarks), a least-squares fitting method is used to obtain the overall rigid body transformation, eliminating overall reference offset. Within a local sliding window (e.g., ±5 frames), point-to-surface ICP optimization is performed on the frames and DEM point clouds or terrain contours. Through a loop of "establishing corresponding point clouds, minimizing geometric errors, and iteratively updating transformations", adjacent frames or point clouds are precisely aligned with the DEM, and the height and pitch angle of each frame are refined to obtain the updated pose sequence.

[0057] Optionally, in step (a), the field of view overlap index is the field of view overlap area or the projection overlap ratio; the transition zone strategy adopts multi-resolution fusion / weighted hybrid smooth splicing to avoid abrupt changes in brightness / height.

[0058] Specifically, in step (b), the pose information after DEM correction is used as the coarse alignment reference, a small-range pose correction is made to the stitching candidate pairs, soft spatiotemporal constraints are introduced, and a transition band strategy is used to smoothly stitch the overlapping areas of the field of view.

[0059] Preferably, to prevent dynamic changes in the fire line from affecting registration, geometry-based fine registration (such as point-to-surface or voxel-to-voxel iterative registration) or image-based multi-scale optical flow / depth consistency optimization is performed in the overlapping areas of the view to eliminate sub-pixel / centimeter-level errors.

[0060] Furthermore, in step (b), the soft spatiotemporal constraints allow for local geometric changes but restrict unreasonable jumps. Based on the temporal alignment and DEM correction, the candidate stitching pairs are jointly adjusted in space and time to obtain seamless local alignment, ensuring both spatial continuity and temporal consistency. Specifically, for the candidate stitching pairs, a weighted spatiotemporal consistency error function is constructed based on the soft spatiotemporal constraints. This function is used as the optimization objective to solve and output the final pose and temporal information, achieving the purpose of fine-tuning the pose and temporal deviations.

[0061] The mathematical expression for the weighted spatiotemporal consistency error function is:

[0062]

[0063] in, This represents the weighted spatiotemporal consistency error; It is an index, representing the sequence number of all matching point pairs in the concatenation candidate pairs or the sliding time window (a sliding window contains several concatenation candidate pairs); It is the total number of matching point pairs used for optimization; It is the first reference frame The spatial coordinates of each matching point pair are obtained from the pose sequence after DEM correction; For the frame to be corrected The spatial coordinates of each matching point pair are obtained by pose transformation and time correction; For the reference frame The reference timestamp for each matching point pair is the corrected timestamp obtained during the time alignment process; For the frame to be corrected The correction time for each matching point pair is estimated based on the current time deviation. , Spatial and temporal weights can be customized. As a regularization term, this application employs a pose smoothing term or a coupling term with a DEM fitting term to suppress over-adjustment and ensure global consistency.

[0064] The logic for fine-tuning pose and temporal deviation based on soft spatiotemporal constraints is as follows: In the overlapping regions of the view domain of the stitching candidate pairs, a set of matching point pairs is generated through feature matching (e.g., image keypoints, point cloud nearest neighbors, or voxel centroids), and the initial geometric distance and temporal difference of each pair are calculated. A confidence weight is assigned to each pair (determined by feature matching strength, view consistency, and temporal alignment confidence). A weighted spatiotemporal consistency error is constructed according to the above formula, and the confidence weight is incorporated into the spatial and temporal terms. Within a sliding time window (e.g., the most recent 5 frames), nonlinear least squares are solved using Gauss-Newton or Levenberg-Marquardt methods, with the variables being the micro-pose correction and temporal deviation for each frame.

[0065] Furthermore, in step 1, the steps of establishing an initial three-dimensional fire situation map based on the UNet network and digital elevation model include:

[0066] Step 1-1: Input the fire scene video frames under the same time information into the UNet network for fire line segmentation to obtain the fire line segmentation mask. Use the fine-tuned pose information to back-project the fire line segmentation mask and point cloud onto the same three-dimensional voxel mesh.

[0067] Steps 1-2: Construct a pyramid hierarchy structure based on the 3D voxel mesh obtained by back projection;

[0068] Steps 1-3: Project the pyramid hierarchy back into the digital elevation model to generate a three-dimensional instantaneous combustion mask, and record the multi-source observation information weighted by confidence level to obtain the initial three-dimensional fire situation map.

[0069] It also includes: spatiotemporal alignment and preprocessing: real-time fire data such as point clouds and images are denoised, distorted and interpolated according to a unified time base and projection coordinate system, and the confidence level is standardized.

[0070] Specifically, in step 1-1, the Unet network processes fire scene video frames, segments them to obtain a real-time fire mask, and outputs the confidence scores of each fire point pixel, extracting fire line correlations from them. Then, the process of back-projecting the fire line segmentation mask and point cloud onto the same 3D voxel mesh is as follows: Each fire point pixel in the fire line segmentation mask is back-projected into world coordinates using the intersection of the camera projection and the DEM depth, or by estimating the depth, based on the final pose information. The point cloud is also transformed to the world coordinate system according to the final pose information. For each projection point, its corresponding voxel is found, and the voxel attributes are updated using weighted cumulative updates.

[0071] The pyramid hierarchy in this invention is a two-level voxel structure comprising a low-resolution layer and a high-resolution candidate layer. The high-resolution candidate layer, actually composed of high-resolution patches, is activated at the fire front or in critical forest belts. The low-resolution layer, actually composed of low-resolution voxels, is used to cover large fire areas, for rapid updates, and for connectivity detection. Virtual edges record the point cloud feature statistics of the high-resolution patches. Optionally, it is constructed using an Octree / octree management system.

[0072] Optionally, the point cloud feature statistics include geometric centroid, average height, normal distribution, point density, observation source distribution, fusion confidence score, and timestamp interval. These statistics serve as virtual edges and parent voxels for inter-layer constraints. Each block outputs a high-resolution point cloud, a triangular mesh, and the block's statistical metadata and confidence score.

[0073] For pyramid-level structures, virtual edges are introduced to treat the voxel poses of low-resolution layers as constraints of high-resolution layers (i.e., treating low-resolution layers as pseudo-observations of high-resolution layers), and these edges are applied simultaneously during optimization to ensure spatial consistency across layers.

[0074] Preferably, in steps 1-2, virtual edges and spatiotemporal consistency constraints are introduced into the pyramid hierarchy to ensure spatial and temporal consistency between different resolution levels. This achieves cross-layer consistency optimization by jointly correcting low-resolution voxel meshes with multiple high-resolution patches spatially and temporally, achieving "consistency between detail and the whole." The goal is to maintain global connectivity and geometric consistency while ensuring high-precision local details, avoiding conflicts or discontinuous seams between different resolutions.

[0075] The meaning of virtual edge constraint is: the centroid / surface statistics of the high-resolution patch should be consistent with the geometric statistics (centroid, mean elevation, normal direction) of the parent voxel in space.

[0076] For a pyramid-shaped hierarchical structure where the same location changes over time, the temporal consistency constraint incorporates a time smoothing term. This allows for realistic changes in fire conditions while suppressing short-term fluctuations caused by observational noise. Time smoothing is achieved by applying exponential decay or sliding window filtering to historical statistics, ensuring consistency between new observation layers and historical layers on short timescales.

[0077] The specific method is as follows: Each high-resolution patch is treated as a node, and its parent voxel (corresponding to a low-resolution voxel) is treated as a low-level node. A conflict detection and priority strategy is adopted: When a conflict exists between a high-resolution patch and a low-level node, the processing strategy is determined by confidence level (based on sensor type, data recentity, multi-source consistency, and registration residuals): If the patch has high confidence and is spatially well connected to its neighborhood, the patch's data overwrites and replaces the low-level statistics, while triggering local retopology to repair the geometry of adjacent voxels. If the patch has low confidence or is isolated, the low-resolution data is retained, and the patch is marked as "pending confirmation," awaiting further observations in the background. Overlapping patch fusion: When multiple high-resolution patches overlap in the same region, height and normal information are merged by weighted confidence level, and a smooth transition band strategy is used to handle the boundary, prioritizing the maintenance of normal continuity to avoid sharp surface discontinuities. Iterative fine-tuning process: Cross-layer consistency adopts an iterative process. During the initialization phase, patch statistics are written into the inter-layer data. Then, the residual of each virtual edge is repeatedly evaluated and small pose or statistical adjustments are made to local nodes (patches or voxels) until the residual converges or the maximum number of iterations is reached. Finally, high-precision optimization of local details and high global connectivity are achieved.

[0078] In steps 1-3, when projecting the pyramid hierarchy back into the digital elevation model, the upper surface of the voxel mesh or the center of the voxel is vertically projected onto the DEM raster cells. The attributes of the corresponding voxels are accumulated into the statistics of the raster cells (cumulative combustion energy, number of observations, average confidence; the energy indicator of combustion is the confidence of the fire point pixels obtained from Unet segmentation). An instantaneous combustion mask is generated from the cumulative statistics. If the cumulative combustion energy exceeds a threshold (e.g., 0.6), the DEM cell is labeled as "burning".

[0079] Preferably, in steps 1-3, if observations indicate a significant drop in the canopy / surface area, and under conditions of high confidence and multi-source consistency (e.g., confidence > 0.8 and ≥ 2 observation sources), the surface elevation change ΔZ is estimated for the grids in the 3D instantaneous combustion mask that meet the high confidence and multi-source consistency criteria, and recorded as an additional layer of surface elevation change. This is exported as additional information for the initial 3D fire situation map, providing an instantaneous and cumulative representation of surface height changes and combustion damage, and displaying these elevation differences in real time on the situation map. The voxel connectivity information provides a structured representation of the spatial topology, ensuring model connectivity and guiding repair and priority decisions. The synergy of both is a key data foundation for achieving both precise and reliable dynamic 3D fire situation construction and maintenance.

[0080] In step 2, the key updated reconstruction areas include the front line of the fire and high-risk areas. The identification process includes:

[0081] Step 2-1-1: Use the UNet network to segment the real-time fire scene image to obtain the real-time fire mask; project the real-time fire mask onto the initial three-dimensional fire situation map to obtain the real-time fire area; identify the heat source and fire line from the real-time fire area and determine it as the leading edge area of ​​the fire line.

[0082] Step 2-1-2: If the combustion probability or energy accumulation of certain voxels after real-time fire mask voxelization exceeds the threshold, they are marked as high-risk areas. The front line of the fire and the high-risk areas are designated as key update areas.

[0083] Specifically, in step 2-1-2, the selection of key update areas is as follows: After obtaining the real-time fire area from the multi-UAV imagery using UNet fire segmentation in step 2-2-1, the heat source and fire line are identified from the real-time fire area and judged as the fire front area; according to step 1-1, after the real-time fire mask is voxelized, the combustion probability or energy accumulation of some voxels exceeds the threshold and is marked as a high-risk area, and the fire front and high-risk areas are used as key update areas.

[0084] Furthermore, in step 2, the steps for reconstructing the pyramid hierarchy of the key update regions include:

[0085] Step 2-2-1: Project the key update area onto the low-resolution layer in the pyramid hierarchy of the fire situation model and mark all low-resolution voxels to be reconstructed.

[0086] Step 2-2-2: Divide the low-resolution voxels into small blocks according to the target resolution. Perform denoising, coarse registration, local alignment and point cloud completion on the multi-source point cloud in each small block to complete the reconstruction. In this way, all low-resolution voxels are reconstructed into high-resolution patches.

[0087] Step 2-2-3: Calculate and save the point cloud feature statistics of all high-resolution patches as virtual edges, and connect the low-resolution voxels and high-resolution patches with corresponding virtual edges to obtain the pyramid hierarchical structure of the key update region.

[0088] In this approach, each high-resolution patch is treated as a node, and its corresponding low-resolution parent voxel is treated as a low-level node. Virtual edges are then established to connect the two.

[0089] Specifically, steps 2-2-1 and 2-2-2 involve reconstructing key areas using a low-resolution layer in the 3D fire situation map to obtain a high-fidelity point cloud / surface model at the centimeter or decimeter level, i.e., a high-resolution patch. The specific steps include:

[0090] The key update area is projected onto the low-resolution layer of the pyramid hierarchy in the initial 3D fire situation model. A buffer zone (e.g., 10–50m, adjusted according to observation height and computing power) is established with the voxel center of the low-resolution layer as a reference. Low-resolution voxels within the buffer zone are marked as candidate blocks to be reconstructed. For each candidate block, a denser set of points is extracted from all real-time fire data covering that block, including but not limited to corrected point clouds, depth points obtained from multi-view image reprojection, and 3D points projected from infrared hotspot pixels. The pose is corrected, and these points are unified to the world coordinate system using a sliding cube or O... The ctree subdivision divides the candidate block into multiple smaller blocks (the size of each block is related to the target accuracy: for example, when the target accuracy is 0.1–0.5m, the block size can be set to 5–20m). For each smaller block, outlier denoising and initial coarse GNSS / INS pose registration are performed on the multi-source point cloud. Then, multi-level ICP (coarse-to-fine) or feature-based registration is used to refine the local alignment. Normals are estimated on the aligned point cloud. If occlusions or holes exist, a deep learning-based point cloud completion module is called to fill in the missing parts to restore surface continuity, completing the smaller block reconstruction. In this way, the candidate block is reconstructed into a high-resolution patch.

[0091] As a supplement, a deep learning-based point cloud completion module takes a point cloud and its confidence / visualization angle information as input, and outputs a completed point cloud or surface sketch. The point cloud is then converted into a smooth triangular mesh using voxel subdivision or surface reconstruction algorithms such as Poisson, moving cube, or local surface fitting.

[0092] Step 3, the steps of compressing the pyramid hierarchy structure in the three-dimensional fire situation model to be updated based on information entropy-driven spatiotemporal analysis, include:

[0093] Step 3-1: Divide the pyramid hierarchy structure in the three-dimensional fire situation model to be updated into partitions. Integrate the pyramid hierarchy structure of the key updated area into the corresponding partition as new observations. Calculate the information entropy of each partition and the observation time difference with the new observations. Preset the thresholds for information entropy and observation time difference. Compress partitions whose information entropy value is lower than the threshold for a long time or whose observation time difference exceeds the threshold, and retain the latest observations.

[0094] Information entropy is determined by the information content of each region. The information in each region consists of multi-dimensional data, optionally including, but not limited to, point cloud density histograms, echo / reflection intensity distributions, histograms of image grayscale or temperature, observation time series (recent observations and observation frequencies), and statistics on multi-source consistency. In practice, this information is first mapped to a normalized probability distribution, and then the entropy metric is used to measure the amount of information. Low entropy indicates concentrated distribution and information redundancy, while high entropy indicates abundant information or drastic changes.

[0095] Specifically, step 3-1 is essentially an entropy-driven compression process. First, the constructed model is evaluated by partitioning it using spatiotemporal analysis based on information entropy: the fire area is divided into several statistically independent spatial partitions, and the information entropy of each partition is calculated. When the information entropy value of a certain area remains below a threshold for an extended period or the observation time difference exceeds a threshold, the system triggers the compression process for that area to save storage and subsequent computational costs while retaining key information supporting decision-making. The compression primarily targets non-critical update areas, i.e., areas that are not of much interest, such as non-fired areas or inactive fired areas. The logic for deciding which area to compress is: if a region frequently receives new observation data, it proves that the region is important and requires high-precision real-time updates; if a region has no observation data for a long time, it proves that the region does not require focused observation, and therefore compression can be performed to save space and improve efficiency.

[0096] Preferably, during scene partitioning, the scene division is not just a fixed grid, but an adaptive generation combining multi-source information. Two types of partitioning are preferred: one is rule-based partitioning based on geographic raster or voxels, which facilitates fast indexing and parallel processing; the other is semantic or event-based partitioning, such as partitioning according to fireline buffer zones, forest stand units, and road / building boundaries. The system will merge the partitioned grid with semantic partitioning: using finer partitions near the fireline and larger grids in flat or long-term stable areas, thus making entropy measurement both efficient and semantically meaningful.

[0097] Preferably, the compression process employs an adaptive threshold and elimination strategy: the system maintains a dynamic threshold, which is jointly determined by the fire situation status (e.g., whether the fire line is active), the rate of new observations within the time window, and storage pressure. If a region has no new observations for a long period and its entropy remains below the threshold, layered compression is performed according to the strategy: first, lossless or low-loss downsampling is performed; only when the region is confirmed as historical and storage needs to be released is the original data further archived or deleted. Before deletion, the system checks the connectivity impact of the region; if deletion would disrupt model connectivity, it first attempts to replace it with a summary or postpones deletion.

[0098] Preferably, priority and retention rules can be set during the compression process: when partitions involving critical infrastructure, important ecological areas or command and control areas are involved, retention rules are added so that they will not be automatically deleted even if the entropy is low; priority is set for voxels / mesh that can significantly affect path connectivity or visual presentation, and they are retained first.

[0099] Preferably, the compression process provides metadata and an audit process: each removal or compression generates metadata records, including but not limited to the original version index, summary information, removal reason and time, so as to trace back and roll back if necessary.

[0100] Step 3, the robust registration and fusion steps include:

[0101] Step 3-2-1: Obtain the original real-time fire data corresponding to the pyramid hierarchy structure of the key update area, and use a pre-trained multimodal feature extraction network to process and extract geometric descriptors, appearance features and semantic features.

[0102] Step 3-2-2: Use geometric descriptors to perform feature matching between the pyramid hierarchy structure of the key update region and the pyramid hierarchy structure in the compressed 3D fire situation model to be updated, and obtain initial alignment;

[0103] Step 3-2-3: Combine the initial alignment with appearance features and semantic features for joint optimization to obtain the updated pyramid hierarchy structure. Project it onto the digital elevation model to obtain the updated three-dimensional situation map, thus completing the update of the three-dimensional situation model of the fire site.

[0104] Specifically, step 3-2-1 uses deep learning to enhance multi-source registration, and feature extraction adopts a multimodal design (multimodal feature extraction network): the point cloud processing branch uses a point cloud processing network (such as PointNet++, KPConv or a lighter point-based convolutional point network) to extract geometric descriptors; the image processing branch uses a point cloud processing network (intermediate features of CNN or UNet) to extract semantic and appearance features.

[0105] Preferably, for multi-source images acquired by the same UAV, such as infrared images and visible light, the image processing branch uses parallel encoding and fuses their high-level semantics. The extracted features include both local geometric information and semantic labels (such as flames, smoke, and vegetation), thus providing discriminability even in occluded or textured areas.

[0106] Step 3-2-2 is essentially coarse matching and initialization: First, global or local feature matching is performed using the geometric descriptors learned in step 3-2-1. Robust methods such as RANSAC are then used to obtain an initial transformation matrix estimate, resulting in initial alignment. Preferably, the initial alignment prioritizes high-confidence feature pairs and filters out incorrect matches based on observation consistency within the time window.

[0107] Specifically, step 3-2-3 starts with the initial alignment and performs joint optimization of appearance, geometry, and semantics: the geometry term mainly addresses the geometric errors of point clouds or voxels, the color / appearance term ensures visual coherence of image regions, and the semantic term constrains the consistency of semantic objects such as flames and tree canopies. Preferably, different weights are assigned to new and old observations from different sources and times during the optimization process, and the weights are adaptively adjusted based on the sensor's confidence level, temporal approximation, and iterative performance of the registration residuals.

[0108] Preferably, step 3-2-3 also includes strategies to address occlusion and noise: for areas severely occluded by smoke or flames, the system uses semantic information to reduce the weight of geometric terms and increase the weight from data sources that are less susceptible to occlusion (such as LiDAR); when the LiDAR signal is sparse or interfered with by burning particles, the semantic alignment and shape prior of multi-view images are preferentially relied upon. For dynamic noise (e.g., a burning flame), a short time window and temporal consistency check are used to avoid mistaking dynamic deformation for static registration errors.

[0109] Preferably, step 3-2-3 further includes deep learning-driven confidence estimation: the deep network simultaneously outputs matching confidence and uncertainty estimation, which are directly used for subsequent weighted optimization and elimination strategies, thereby maintaining the robustness and stability of registration in complex scenarios.

[0110] Preferably, step 3-2-3 also includes online learning and model updates: in the background, the pre-trained network is updated in small batches using high-confidence pairs to improve its performance under local fire scene features (such as specific vegetation or flame morphology). All online updates are subject to version management and rollback control.

[0111] Preferably, step 3 further includes: performing connectivity analysis on the low-resolution layer of the pyramid hierarchy in the updated 3D fire situation model to repair local topology: if an update produces isolated holes or disconnected fragments, triggering local topology repair: performing nearest neighbor interpolation or smooth transition based on surrounding voxels for neighboring voxels. If topology damage is caused by missing data, it is temporarily retained as a "low-confidence placeholder" until filled by subsequent observations or corrected by background optimization.

[0112] Specifically, after compression and registration fusion, local retopology and surface repair ensure the mesh connectivity and porosity of the model: first, affected boundary loops and disconnected components are automatically identified; then, missing regions are reconstructed according to priority using spline / Laplacian subdivision or Poisson surface restoration methods; finally, the repaired segments are subjected to normal smoothing and geometric consistency checks. This completes a closed-loop process of "information-driven compression—deep robust registration—topology assurance," which both compresses redundant data to control its scale and ensures the integrity and dynamic real-time nature of key fire scene information.

[0113] Among these features, local retopology and model repair, including affected area identification, automatically locates and marks areas requiring repair by comparing mesh differences before and after registration, detecting isolated voxels, unclosed boundaries, and abrupt changes in normals. This identification also considers inter-layer consistency residuals and information entropy changes, prioritizing the marking of holes and cracks that affect connectivity or view quality.

[0114] Boundary extraction and constraint generation: Boundary loops or boundary curves are extracted for each marked region, and the geometric features of the boundaries (curvature, sampling density, normal direction) are analyzed. These boundaries serve as control constraints for reconstruction, ensuring that the newly constructed surface naturally connects with the surrounding unaffected mesh.

[0115] Optionally, the selection of topology repair methods is based on the type of damage: small holes are preferably filled using Laplacian subdivision and smooth filling; medium-sized defects are often filled using Poisson surface reconstruction or local hole filling based on moving cubes; large-scale structural defects tend to be patched using nearby high-confidence patches as templates and smoothed with local stitching algorithms.

[0116] Preferably, geometric and normal smoothing is performed: after repair, the normals of the newly generated vertices are reconstructed and smoothed, and robust filtering is used to suppress noise and preserve sharp features (such as rock ridges or tree trunk edges) to balance visual realism and geometric accuracy.

[0117] To verify the effectiveness of local topology repair, connectivity and visualization consistency verification are also provided: After repair, the system automatically verifies whether the mesh meets topological connectivity requirements, whether there are non-manifold edges or overlapping triangles, and generates a repair report (including a comparison of voxel / mesh statistics before and after repair). If the repair affects the DEM or key indicators, the changes are recorded and a strategy is used to decide whether to write back the base data; as well as versioning, rollback, and manual review: All automatic repair operations generate traceable version differences and logs; in critical areas or when repair uncertainty is high, the system marks the repair results as "awaiting manual review" and retains a rollback channel to restore to the previous stable version.

[0118] The above-mentioned technical solution of the present invention can effectively cope with the challenges of complex terrain and dynamic changes in fire, improve the continuity and accuracy of three-dimensional situational awareness of forest fire, and is suitable for UAV swarm inspection, airborne monitoring, etc., and can be used in harmony with existing fire prevention and control systems.

[0119] The technical solution described above can effectively address the challenges of complex terrain and dynamic changes in fire conditions, improving the continuity and accuracy of three-dimensional situational awareness in forest fire areas. It is suitable for drone swarm inspections, airborne monitoring, and is compatible with existing fire prevention and control systems. It should be noted that the various sensor types, data processing algorithms, and parameter configurations used in the above embodiments are merely illustrative; adjustments and optimizations can be made according to specific needs in actual applications.

[0120] The implementation of the various embodiments of the present invention is based on programmed processing through a system with processor functionality. Therefore, in practical engineering, the technical solutions and functions of the various embodiments of the present invention are encapsulated into various modules. Based on this reality, and building upon the above embodiments, the embodiments of the present invention provide a system for constructing and updating the three-dimensional situation of a forest fire using a drone swarm. This system is used to execute the method for constructing and updating the three-dimensional situation of a forest fire using a drone swarm in the above method embodiments.

[0121] See Figure 2 The system includes:

[0122] The fire scene 3D situation model construction module is used to receive real-time fire scene data from the UAV cluster and establish an initial fire scene 3D situation model based on the UNet network and digital elevation model; the fire scene 3D situation model includes a fire scene 3D situation map and a pyramid hierarchical structure;

[0123] The update information acquisition module is used to construct a pyramid-shaped hierarchical structure of key update areas based on real-time fire data when the preset update cycle is reached.

[0124] The fire scene 3D situation model update module is used to compress the pyramid hierarchy structure in the fire scene 3D situation model to be updated based on the pyramid hierarchy structure of the key update area and driven by spatiotemporal analysis based on information entropy. Then, it is robustly registered and fused to update the fire scene 3D situation model.

[0125] It should be noted that the system embodiments provided by the present invention are used not only to implement the methods in the above method embodiments, but also to implement the methods in other method embodiments provided by the present invention. The only difference is that corresponding functional modules are set. The principle is basically the same as that of the above system embodiments provided by the present invention. As long as those skilled in the art can improve the modules in the above system embodiments by referring to the specific technical solutions in other method embodiments and combining technical features to obtain corresponding technical means and technical solutions composed of these technical means, on the basis of the above system embodiments, and on the premise of ensuring the practicality of the technical solutions, they can obtain corresponding system-like embodiments for implementing the methods in other method-like embodiments.

[0126] Optionally, such as Figure 3As shown, this invention provides an optional UAV swarm forest fire 3D situation construction and update system, including a multi-source sensor system deployed on an airborne or ground platform, a data communication module, and a computing processing unit configured with a streaming incremental modeling program. The multi-source sensor system may include devices such as lidar, visible light cameras, infrared thermal imagers, and GPS, used to collect spatial point cloud and image data of the forest fire in real time. After time synchronization within the system, the data collected by each sensor is sent to the data processing unit for further processing. The system divides the processing flow into three stages: rapid initial model construction, global multi-scale optimization, and real-time model update. The execution steps of these three stages are described in detail below:

[0127] At the start of fire monitoring, the system first enters the rapid initial model building phase:

[0128] (1) The drone cluster simultaneously patrols and collects data on the fire site, generating continuous point cloud data and multi-time-series drone images (visible light images and infrared images).

[0129] (2) Time reference alignment: Time correction is performed on each sensor through a clock synchronizer so that the acquisition frames of different sensors are aligned on a unified time reference. Terrain feature matching: The flight trajectory of the UAV is mapped to the DEM reference coordinate system, and terrain feature matching is performed to suppress observation errors caused by terrain occlusion and achieve coarse correction of sensor attitude and position.

[0130] (3) The point cloud data is voxelized according to spatial coordinates to generate a low-resolution voxel mesh model, which is used to quickly depict the overall geometric outline of the fire scene. For key areas of interest in the fire scene, a higher resolution point cloud model is generated at the same time to capture richer details and form a pyramid-shaped hierarchical structure.

[0131] (4) The local point clouds between consecutive frames are aligned and stitched using soft spatiotemporal constraint technology to ensure the continuity between models at adjacent time points. After this stage, the system obtains an initial fire situation model covering the entire monitoring area.

[0132] After obtaining the initial model, the system enters the global multi-scale optimization stage:

[0133] (1) High and low resolution joint processing: The system performs joint processing of the initial low-resolution voxel mesh model and the corresponding high-resolution key area point cloud model. In the voxel mesh model, virtual edge connections are added between adjacent mesh cells to ensure the continuity of local geometry.

[0134] (2) Real-time frame alignment: For data collected at different times, the system sets time consistency constraints to align the real-time frame with the previous frame or the model at the base time, thereby reducing time drift. Model set parameter optimization: The system uses both the voxel mesh layer and the point cloud layer as optimization variables. Through graph optimization or nonlinear least squares methods, the system simultaneously considers virtual edge constraints and time consistency constraints to optimize the geometric parameters of the model.

[0135] (3) This process can be iterated repeatedly to keep the low-resolution model and the high-resolution model consistent, thereby improving the accuracy of local modeling while ensuring the macroscopic coherence of the overall fire scene model. After the global multi-scale optimization stage, the system obtains a more accurate and coherent three-dimensional situation model, which integrates information from the high-resolution area in detail, and the overall shape closely matches the DEM.

[0136] As the fire situation continues to change, the system needs to dynamically maintain and update the three-dimensional situation model to ensure the timeliness of information.

[0137] (1) Data redundancy filtering: After entering this stage, when the system receives new sensor data, it automatically removes redundant data by using an entropy-driven compression algorithm based on the information entropy changes of each region in the historical model.

[0138] (2) Real-time image model registration: The newly acquired sensor data is preprocessed and registered with the existing model. Point cloud or image features are extracted by pre-trained convolutional neural networks or graph neural networks to better cope with interference factors such as occlusion and noise when calculating the registration transformation.

[0139] (3) After registration is completed, the new data is integrated into the existing model to update the geometry of the corresponding region.

[0140] (4) After completing the data fusion, the system performs local retopology processing on the model mesh. For mesh holes or cracks caused by data deletion, the system repairs the mesh connection through interpolation or local mesh subdivision algorithms to ensure that the overall three-dimensional model has no gaps and remains connected. After the above steps, the system can update the three-dimensional situation model of the fire scene in real time, so that it always reflects the current geometric shape of the fire scene and the position of the fire line.

[0141] In summary, this invention provides a system for constructing and updating a 3D situation of a forest fire using a drone swarm under complex terrain and dynamic fire conditions. This system consists of three parts: a sensor timing correction and terrain matching subsystem, a multi-scale 3D reconstruction subsystem, and an entropy-driven compression and robust registration subsystem. The sensor timing correction and terrain matching subsystem achieves inter-frame alignment and terrain adaptation of multi-source data through high-precision clock synchronization and DEM-based trajectory correction. The multi-scale 3D reconstruction subsystem adopts a hierarchical pyramid structure, first rapidly depicting the macroscopic outline of the fire field in a low-resolution voxel grid, and then reconstructing local details at the critical fire front using high-resolution point clouds. Cross-layer consistency optimization balances global coherence and local accuracy through a joint least-squares objective function. The entropy-driven compression and robust registration subsystem automatically removes redundant historical data based on information entropy and efficiently fuses new observation data using a deep learning-enhanced registration algorithm, ultimately maintaining model coherence through local retopology repair. The system features four key characteristics: spatiotemporal integrated correction, streaming incremental reconstruction, multi-layer constraint optimization, and dynamic redundancy removal. These features significantly improve the accuracy and efficiency of three-dimensional situational awareness of forest fires, providing reliable spatiotemporal information support for fire emergency command and decision-making.

[0142] The method in this embodiment of the invention is implemented using an electronic device; therefore, it is necessary to introduce the relevant electronic device. For this purpose, embodiments of the present invention provide an electronic device, such as... Figure 4 As shown, the electronic device includes: at least one processor, a communication interface, at least one memory, and a communication bus, wherein the at least one processor, the communication interface, and the at least one memory communicate with each other via the communication bus. The at least one processor invokes logical instructions stored in the at least one memory to execute all or part of the steps of the methods provided in the foregoing method embodiments.

[0143] Furthermore, when the logical instructions in at least one of the aforementioned memories are implemented as software functional units and sold or used as independent products, they are stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, is embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (a personal computer, server, or network device) to execute all or part of the steps of the methods described in the various method embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks—various media for storing program code.

[0144] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, located in one place, or distributed across multiple network units. The purpose of this embodiment is achieved by selecting some or all of the modules according to actual needs. Those skilled in the art will understand and implement this without any inventive effort.

[0145] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0146] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0147] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0148] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0149] Based on the same technical concept as the foregoing embodiments, the present invention provides a non-transitory computer-readable storage medium that stores computer instructions that enable the computer to perform three-dimensional situational awareness construction and updating of a forest fire by a drone swarm.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing and updating a three-dimensional situation of a forest fire using unmanned aerial vehicle (UAV) swarms, characterized in that... include: Receive real-time fire data from the drone swarm and establish an initial three-dimensional fire situation model based on the UNet network and digital elevation model; The three-dimensional fire situation model includes a three-dimensional fire situation map and a pyramid hierarchical structure. When the preset update cycle is reached, a pyramid-shaped hierarchical structure of key update areas is constructed based on real-time fire data; Based on the pyramid hierarchy structure of the key update regions, the pyramid hierarchy structure in the three-dimensional fire situation model to be updated is compressed using spatiotemporal analysis driven by information entropy. Then, robust registration and fusion are performed to update the three-dimensional fire situation model.

2. The method for constructing and updating a three-dimensional situation of a forest fire by a drone swarm as described in claim 1, characterized in that, It also includes: fine-tuning the pose and time information in real-time fire data based on soft spatiotemporal constraints, and aligning and stitching local point clouds between consecutive frames.

3. The method for constructing and updating a three-dimensional situation of a forest fire by a drone swarm as described in claim 1, characterized in that, The steps for establishing an initial 3D fire situation map based on the UNet network and digital elevation model include: Fire scene video frames under the same time information are input into the UNet network for fire line segmentation to obtain the fire line segmentation mask. The fire line segmentation mask and point cloud are back-projected onto the same three-dimensional voxel mesh using the fine-tuned pose information. A pyramid hierarchical structure is constructed based on the three-dimensional voxel mesh obtained by back projection. The pyramid-shaped hierarchical structure is projected back into the digital elevation model to generate a three-dimensional instantaneous combustion mask. Multi-source observation information is then recorded and weighted by confidence level to obtain an initial three-dimensional fire situation map.

4. The method for constructing and updating a three-dimensional situation of a forest fire by a drone swarm as described in claim 1, characterized in that, The pyramid hierarchy consists of low-resolution voxels and their corresponding high-resolution patches, as well as virtual edges connecting the two. The low-resolution voxels and high-resolution patches are optimized for cross-layer consistency by introducing virtual edges and spatiotemporal consistency constraints.

5. The method for constructing and updating a three-dimensional situation of a forest fire by a drone swarm as described in claim 4, characterized in that, The steps to rebuild the pyramid hierarchy of key update regions include: Project the key update area onto the low-resolution layer of the 3D fire situation map and mark all low-resolution voxels to be reconstructed. The low-resolution voxels are adaptively divided into small blocks according to the target resolution. The multi-source point clouds in each small block are denoised, coarsely registered, locally aligned, and the point cloud is completed to reconstruct them. In this way, all low-resolution voxels are reconstructed into high-resolution patches. Calculate and save the point cloud feature statistics of all high-resolution patches as virtual edges, and connect the low-resolution voxels and high-resolution patches with corresponding virtual edges to obtain the pyramid-level structure of the key update region.

6. The method for constructing and updating a three-dimensional situation of a forest fire by a drone swarm as described in claim 1, characterized in that, The steps involved in compressing the pyramid hierarchy structure in the 3D fire situation model to be updated based on information entropy-driven spatiotemporal analysis include: The pyramid hierarchy structure in the updated 3D fire situation model is partitioned, and the pyramid hierarchy structure of the key updated area is integrated into the corresponding partition as new observation. The information entropy of each partition and the observation time difference with the new observation are calculated. The thresholds for information entropy and observation time difference are preset. Partitions with information entropy values ​​that are consistently below the threshold or observation time differences that exceed the threshold are compressed, and the latest observation is retained.

7. The method for constructing and updating three-dimensional situational awareness of forest fires using unmanned aerial vehicle (UAV) swarms as described in claim 1, characterized in that, The steps for robustly registering and fusing the compressed 3D fire situation map with the pyramid hierarchy structure of key updated areas include: Obtain the original real-time fire data corresponding to the pyramid-level structure of the key update area, and use a pre-trained multimodal feature extraction network to process and extract geometric descriptors, appearance features and semantic features. Geometric descriptors are used to perform feature matching between the pyramid hierarchy structure of the key update region and the compressed 3D fire situation map to be updated, so as to obtain the initial alignment. The initial alignment is combined with appearance features, semantic features, and temporal confidence to jointly optimize and obtain the updated three-dimensional fire situation map.

8. A system for constructing and updating three-dimensional situational awareness of forest fires using unmanned aerial vehicle (UAV) swarms, characterized in that: include: The fire scene 3D situation model construction module is used to receive real-time fire scene data from the UAV cluster and establish an initial fire scene 3D situation model based on the UNet network and digital elevation model; the fire scene 3D situation model includes a fire scene 3D situation map and a pyramid hierarchical structure; The update information acquisition module is used to construct a pyramid-shaped hierarchical structure of key update areas based on real-time fire data when the preset update cycle is reached. The fire scene 3D situation model update module is used to compress the pyramid hierarchy structure in the fire scene 3D situation model to be updated based on the pyramid hierarchy structure of the key update area and driven by spatiotemporal analysis based on information entropy. Then, it is robustly registered and fused to update the fire scene 3D situation model.

9. An electronic device, characterized in that, The method includes a memory and a processor, the memory storing program instructions that are executed by the processor, the processor invoking the program instructions to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that cause the computer to perform the method described in any one of claims 1 to 7.

Images