Disaster scene large-range modeling method and device based on unmanned aerial vehicle multi-source scanning data fusion

Abstract

The invention discloses a catastrophe scene large-range modeling method and device based on unmanned aerial vehicle multi-source scanning data fusion. The method comprises the steps that S110, acquisition and unified organization of catastrophe scene multi-source scanning data are completed through an unmanned aerial vehicle multi-source data acquisition module; s120, sequentially completing data preprocessing, geometric prior generation and multi-source fusion alignment optimization through a multi-source data fusion and geometric optimization module; and S130, completing three-dimensional model achievement generation, model lightweight and block-level incremental updating through a scene reconstruction and incremental updating release module. According to the method, a catastrophe scene large-range three-dimensional model of a unified scale can be rapidly obtained under complex conditions of weak texture, smoke shielding and the like, the modeling calculation overhead is reduced, rapid on-site iteration updating is supported, and the research, judgment and command supporting capacity of an emergency rescue scene is improved.

Description

Technical Field

[0001] This invention belongs to the field of 3D mapping and modeling technology for emergency rescue scenarios, and involves technologies such as UAV multi-source scanning data acquisition, image geometric calculation, laser point cloud processing, cross-modal fusion registration and scene 3D reconstruction. Specifically, it relates to a method and device for large-scale modeling of disaster scenarios based on UAV multi-source scanning data fusion. Background Technology

[0002] Disaster sites (such as building collapses, landslides, and earthquake damage) are often characterized by their large scale, complex structure, severe obstruction, high levels of dust, and unstable lighting conditions. Emergency rescue command relies heavily on the rapid acquisition and updating of three-dimensional spatial information to assess the overall situation. Taking a large-scale collapse scenario as an example, rescue missions typically require a very short time to grasp the overall outline of the affected area, key passages, and the distribution of debris to support personnel deployment, equipment scheduling, and risk assessment. Among existing on-site modeling methods, reconstruction schemes relying solely on UAV multi-view images are prone to problems such as unstable matching, scale drift, and model fragmentation under conditions of weak texture, smoke and dust obstruction, strong reflection, and large-scale viewpoint changes. While scanning schemes relying solely on airborne laser point clouds possess geometric stability, they are insufficient in terms of appearance representation, local detail completion, and semantic understanding, and may still form point cloud voids in densely obstructed areas. Meanwhile, disaster sites are constantly changing; secondary collapses, smoke and dust diffusion, and equipment disturbances cause the scene to be constantly updated. Using a full reconstruction approach would impose a heavy computational and transmission burden, making it difficult to meet the emergency site's requirements for "rapid base creation and continuous updates." Therefore, how to fully utilize multi-source information such as multi-view images acquired by UAVs, airborne laser point clouds, and positioning attitude data to achieve rapid construction, unified scale fusion, and incremental updates of large-scale 3D models of disaster scenes without increasing the complexity of on-site operations is a pressing technical challenge. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a method and device for large-scale modeling of disaster scenarios based on the fusion of multi-source scanning data from UAVs. This method addresses the characteristics of disaster sites, such as large area, numerous obstructions, heavy dust and smoke, unstable lighting, and continuously changing scenes. It enables unified organization, unified scale fusion modeling, and rapid iterative updates of multi-source scanning data acquired by UAVs, thereby quickly forming large-scale three-dimensional models of disaster scenarios that can be used for emergency rescue assessment and command and dispatch, while also taking into account the needs of rapid usability and continuous improvement in field applications.

[0004] The present invention solves its technical problem by adopting the following technical solution: a method for large-scale modeling of disaster scenarios based on the fusion of multi-source scanning data from unmanned aerial vehicles, comprising the following steps:

[0005] S110 uses the UAV multi-source data acquisition module to acquire and organize multi-source scanning data at the disaster site.

[0006] S120, through the multi-source data fusion and geometric optimization module, sequentially completes data preprocessing, geometric prior generation and multi-source fusion alignment optimization;

[0007] S130 completes the generation of 3D model results, model lightweighting, and block-level incremental updates through the scene reconstruction and incremental update publishing module.

[0008] A disaster scenario large-scale modeling device based on UAV multi-source scanning data fusion, used to execute the above methods, includes:

[0009] The UAV multi-source data acquisition module is used to acquire and organize multi-source scanning data at disaster sites.

[0010] The multi-source data fusion and geometric optimization module is used to sequentially complete data preprocessing, geometric prior generation, and multi-source fusion alignment optimization.

[0011] The Scene Reconstruction and Incremental Update Publish module is used to generate 3D model results, lightweight the model, and perform block-level incremental updates.

[0012] A computing device includes: at least one processor and a memory storing program instructions; when the program instructions are read and executed by the processor, the computing device performs the method.

[0013] A readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform the method.

[0014] A computer program product includes a computer program that, when executed by a processor, implements the method.

[0015] Beneficial effects:

[0016] (1) To address the problem of unstable image modeling caused by weak textures, smoke and dust obscuring, and unstable lighting at disaster sites, this invention introduces airborne laser point cloud to construct a geometric base and establishes scale consistency and geometric consistency constraints for fusion optimization, which can improve the robustness and scale stability of large-scale modeling and reduce the risk of matching failure and model breakage.

[0017] (2) In view of the problem that a single data source is difficult to simultaneously take into account geometric stability and expression integrity, this invention performs quality control and adaptive weighted fusion on multi-view images, airborne point clouds and coordinate and attitude data, which can first form the overall outline and usable model of the disaster scene, and then gradually fill in the details and improve the model expression during the continuous collection and fusion process.

[0018] (3) In view of the problem that the continuous changes in the disaster site scene lead to high cost of full reconstruction and difficulty in rapid iteration, the present invention adopts a block modeling and block-level incremental update mechanism, which triggers local reconstruction and fusion update only for the changed area, thereby reducing the computation and transmission burden and improving the efficiency of on-site modeling and updating. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall system architecture of a disaster scenario large-scale modeling method based on UAV multi-source scanning data fusion according to the present invention.

[0020] Figure 2 This is a schematic diagram of the multi-source data fusion and geometric optimization processing flow of the present invention. Detailed Implementation

[0021] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0022] This invention relates to a method for large-scale modeling of disaster scenarios based on the fusion of multi-source scanning data from unmanned aerial vehicles (UAVs). The method comprises a UAV multi-source data acquisition module 1, a multi-source data fusion and geometric optimization module 2, and a scene reconstruction and incremental update publishing module 3. It can, to a certain extent, address the characteristics of disaster sites, such as large area, numerous obstructions, heavy dust and smoke, unstable lighting, and continuously changing scenes. It enables unified-scale fusion modeling and rapid iterative updates of multi-source data, resulting in large-scale three-dimensional models of disaster scenarios that can be used for emergency rescue assessment and command and dispatch.

[0023] like Figure 1 As shown, the method includes the following steps:

[0024] The S110 uses a drone multi-source data acquisition module to acquire and organize multi-source scanning data from disaster sites. The specific implementation is as follows:

[0025] S110-1: Acquire multi-view image data of the disaster site, form an image sequence covering the target area, and record the acquisition timestamp and shooting parameter information of each frame of the image;

[0026] Drones equipped with cameras captured images of the disaster area from multiple perspectives, obtaining image sequences. And record the timestamp of each frame. In addition to the shooting parameter information. During the acquisition process, sufficient viewpoint changes and overlapping coverage are maintained in key areas to ensure that subsequent geometric calculations have the necessary viewpoint baseline and observation redundancy.

[0027] S110-2 acquires airborne laser point cloud data from the disaster site, forms a point cloud sequence representing the spatial geometry, and records the acquisition timestamp and scanning attribute information of each frame of point cloud.

[0028] The drone, equipped with an airborne lidar, scanned the disaster area and obtained point cloud sequences. And record the timestamp of each point cloud frame. In addition to scanning attribute information, point clouds are used to stably represent the spatial geometry of disaster sites, providing a reliable geometric reference for fusion modeling under conditions of weak texture, smoke and dust occlusion, and unstable lighting.

[0029] S110-3, Obtain the coordinate and attitude information corresponding to the image data and point cloud data, and bind them with the image frame and point cloud frame timestamp to establish the correlation between multiple source observations at the same time, forming a multi-source observation dataset that can be used for fusion processing.

[0030] Simultaneously acquire UAV coordinates and attitude information The data is then linked to image frames and point cloud frames using timestamps, ensuring correlation among the three types of data at the same observation time. The coordinate and attitude information provides initial alignment cues in a unified reference coordinate system and enhances the convergence stability of subsequent cross-modal registration.

[0031] S120, through the multi-source data fusion and geometric optimization module 2, data preprocessing, geometric prior generation, and multi-source fusion alignment optimization are completed sequentially. The process of the multi-source data fusion and geometric optimization module 2 is as follows: Figure 2 As shown, it includes three parts in sequence: data preprocessing, geometric prior generation, and multi-source fusion alignment optimization. The specific implementation is as follows:

[0032] S120-1 performs data preprocessing, synchronously correlates and controls the quality of multi-source observation data to form an effective observation set. This includes establishing the correspondence between image frames and point cloud frames based on timestamps, performing image correction and key frame filtering, and performing anomaly removal and filtering downsampling on point clouds to reduce the impact of smoke and dust occlusion, unstable lighting and scanning noise on modeling stability.

[0033] This process may include:

[0034] S120-1-1, establish a correspondence between image frames and point cloud frames based on timestamps, for each image frame. Select the point cloud frame with the closest time. :

[0035] ,

[0036] And associate this correspondence with the attitude data Binding creates synchronously associated observation entries. In the formula... For the first Frame image timestamp, For the first Frame point cloud timestamps. To avoid introducing false constraints through asynchronous observations, time differences can be used. Observations exceeding the threshold are downweighted.

[0037] S120-1-2 performs image correction based on camera calibration parameters and conducts quality assessments based on sharpness, exposure rationality, and the degree of smoke and dust obstruction, discarding or downweighting low-quality images. It combines coverage and viewpoint changes to select a keyframe set, ensuring that the keyframes cover both the disaster area boundaries and key accumulation structures while controlling the scale of subsequent solutions. To facilitate weighted processing, the image quality score can be denoted as... This is used for subsequent residual weighting.

[0038] S120-1-3 performs anomaly removal and noise suppression on the point cloud, and uses uniform downsampling to control the point cloud size while preserving the geometric information of regions such as disaster scene boundaries and accumulation contours to avoid oversimplification that could lead to overall contour distortion. Reliability weights can also be generated from the point cloud. This is used for subsequent block-based statistics and weighted fusion.

[0039] S120-2, Perform geometric prior generation, perform point cloud segmentation and construct segmented geometric base models to obtain initial pose and sparse structure results on the image side. This process may include: segmenting the point cloud according to spatial range and establishing a segmentation index structure; forming a segmented geometric base model at the segmentation level to represent the overall contour of the scene; simultaneously establishing multi-view geometric consistency constraints based on the keyframe set to solve for camera pose trajectory and sparse structure results; and constructing cross-modal geometric association constraints in the co-covered area of ​​the sparse structure results and the segmented geometric base model on the image side.

[0040] Specifically, the process includes:

[0041] S120-2-1, Point Cloud Segmentation and Geometric Base Construction: Dividing the point cloud into segment sets according to spatial extent. A segmented index structure is established; a segmented geometric base model is constructed at the segmented level to represent the overall outline of the scene and provide a stable geometric reference, while also statistically analyzing information such as segmented coverage, point density, and reliability. This segmented organization method enables block-level controllability in the retrieval, updating, and fusion of large-scale scenes, facilitating subsequent reconstruction and updating of only local areas.

[0042] S120-2-2, Image Pose Determination and Structure Restoration: Based on a set of keyframes, multi-view geometric consistency constraints are established to solve for camera pose trajectory and sparse structure results. The reprojection error of 3D points on the image is used as one of the consistency metrics.

[0043] ,

[0044] In the formula For sparse three-dimensional points, For the first Frame camera extrinsic parameters, This is used to represent the pose of the camera coordinate system relative to the reference coordinate system. Used to indicate the position offset of the camera coordinate system relative to the reference coordinate system. For camera internal parameters, For projection function, For observed pixels. To suppress the impact of anomalous matching caused by weak textures and occlusion, image quality scoring can be used. The error term is weighted to reduce the impact of low-quality frames on the results.

[0045] S120-2-3, Construction of Cross-Modal Geometric Association Constraints: Cross-modal geometric associations are established in the co-covered regions of the sparse structure results on the image side and the block-based geometric base model based on point clouds. The stable geometric information of the base provides constraints for subsequent scale unification and fusion alignment. Specifically, this can be achieved in the block-based model. Representative geometric elements (such as local planes and point sets near boundary contours) are selected and corresponded with the image side structure to ensure that cross-modal constraints have spatial uniformity, thereby reducing the risk of structural drift caused by accumulated errors over long routes.

[0046] S120-3 performs multi-source fusion alignment optimization, completing cross-modal fusion registration and coordinate unification under unified scale and consistency constraints, and outputting 3D structural results in a unified coordinate system. These results include unified-scale pose trajectories, fused point clouds, and indexes and statistical information associated with the segmented geometric base model. The fused point cloud refers to the set of point clouds aligned and merged in a unified coordinate system. The specific implementation is as follows:

[0047] S120-3-1, Scale Consistency and Drift Suppression: Based on cross-modal geometric correlation constraints, image side pose and structure consistency optimization is performed to maintain scale stability over a wide range of results and suppress cumulative drift caused by long flight paths. To ensure robustness, a weighted robust objective can be used to jointly constrain image consistency and geometric consistency.

[0048] ,

[0049] In the formula This represents the set of parameters to be optimized (including at least scale and pose-related parameters). For the first Frame image quality weights For the first Reliability weights for each segmented geometric base model The cross-modal geometric consistency residual is obtained by summarizing the intra-block correlation constraints. For the first The three-dimensional point at the th t Reprojection error on frame images, For robust kernel functions, This is the balance coefficient. This formula is used to suppress the disruption of scale and trajectory caused by anomalous observations under conditions of smoke and dust obscuring and weak texture.

[0050] S120-3-2, Cross-modal fusion registration: The fusion registration of the sparse structure result on the image side and the block geometric base model is completed under a unified coordinate system. The coordinate alignment relationship is represented by similarity transformation.

[0051] ,

[0052] In the formula A three-dimensional point in the visual coordinate system. To unify the corresponding three-dimensional points in the coordinate system, As a scale factor, Let be a rotation matrix. Let be the translation vector. By solving for this transformation and performing consistency optimization, a fused representation of the two types of data in a unified coordinate system is achieved.

[0053] S120-3-3 outputs 3D structural results in a unified coordinate system, including uniform-scale pose trajectory, fused point cloud, and block index and statistical information, enabling the results to have block-level organization and spatial query capability, while providing stable input for subsequent model generation, display and updating.

[0054] S130, through the scene reconstruction and incremental update publishing module, completes the generation of 3D results, model lightweighting, and block-level incremental updates. This process may include:

[0055] S130-1, 3D Scene Modeling: Based on the fused point cloud output from step S120-3, mesh reconstruction is performed to generate a contour mesh model, which is then output together with the fused point cloud as the final 3D model of the disaster scene. The meshing process aims to form a continuous visible surface, and a basic optimization form can be constructed using point-to-mesh distance and surface smoothness terms.

[0056] ,

[0057] In the formula This represents the mesh surface model to be generated. To integrate point clouds and point sets, The distance from the point to the grid surface. For surface smoothing regularization, This is a weighting factor. This process yields mesh results with continuous contours and clear structures, used to quickly represent the overall outline, spatial channels, and key packing distribution.

[0058] S130-2, Model Optimization and Lightweighting: Optimize and lightweight the 3D model output, such as by improving the structure and compressing the expression, so that the model can maintain its usability and stability under conditions of fast loading, interactive browsing and remote transmission at the command end; lightweighting may include model simplification, block trimming, hierarchical detail organization, etc., so that the command end can load outputs of different precision as needed.

[0059] S130-3, Incremental Reconstruction and Local Update: This feature performs block-level change detection and local incremental updates on newly added observation data. Reconstruction and fusion updates are triggered only for changed areas, and metadata such as coverage and confidence levels are generated simultaneously for results evaluation and release management. This supports rapid iterative updates under continuously changing conditions at disaster sites. Change area detection can establish change scores at the block level. ,when This block update is triggered when the threshold is exceeded:

[0060] ,

[0061] In the formula These represent the block coverage before and after the update. These represent the block point density before and after the update, respectively. This is a statistic of the block-merged residuals. These are the weighting coefficients. By updating only the changed blocks, the computational and transmission overhead of full reconstruction is avoided.

[0062] In summary, the present invention achieves this through... Figure 1 The three modules shown work together, and in accordance with Figure 2The illustrated process completes data preprocessing, geometric prior generation, and multi-source fusion alignment optimization, including: acquiring multi-source UAV scanning data from the disaster site and constructing a synchronous data package; the scanning data includes timestamped multi-view image data, airborne laser point cloud data, and coordinate and attitude data; performing quality assessment and anomaly removal on the multi-source data, completing keyframe screening and point cloud filtering downsampling; performing point cloud block processing, constructing a low-resolution block geometric base model in a unified coordinate system to represent the overall contour of the scene; solving for camera pose and sparse structure based on multi-view constraints, and introducing the block geometric base model for scale unification and drift suppression optimization; performing cross-modal fusion registration and coordinate unification, and performing block-level incremental reconstruction and local updates during the scene reconstruction stage, triggering reconstruction for changed areas, outputting point cloud and contour mesh models, and generating metadata such as coverage and confidence for result evaluation and publication. This invention enables the rapid generation and continuous updating of large-scale 3D models of disaster scenes under complex conditions, providing reliable support for emergency rescue assessment and command and dispatch.

[0063] This method is applicable to multi-rotor UAV platforms equipped with airborne lidar and visible light cameras, and the UAV platform has coordinate and attitude acquisition capabilities to simultaneously output multi-view image data, airborne lidar point cloud data, and coordinate and attitude data.

[0064] This invention also provides a large-scale disaster scenario modeling device based on UAV multi-source scanning data fusion, used to perform the above methods, including:

[0065] The UAV multi-source data acquisition module is used to acquire and organize multi-source scanning data at disaster sites.

[0066] The multi-source data fusion and geometric optimization module is used to sequentially complete data preprocessing, geometric prior generation, and multi-source fusion alignment optimization.

[0067] The Scene Reconstruction and Incremental Update Publish module is used to generate 3D model results, lightweight the model, and perform block-level incremental updates.

[0068] The present invention also provides a computing device, comprising: at least one processor and a memory storing program instructions; when the program instructions are read and executed by the processor, the computing device performs the method described thereon.

[0069] The present invention also provides a readable storage medium storing program instructions, which, when read and executed by a computing device, cause the computing device to perform the method described thereon.

[0070] A computer program product includes a computer program that, when executed by a processor, implements the method.

[0071] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

[0072] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered to fall within the protection scope of the present invention.

Claims

1. A method for large-scale modeling of disaster scenarios based on multi-source scanning data fusion from unmanned aerial vehicles (UAVs), characterized in that, include: S110 uses the UAV multi-source data acquisition module to acquire and organize multi-source scanning data at the disaster site. S120, through the multi-source data fusion and geometric optimization module, sequentially completes data preprocessing, geometric prior generation and multi-source fusion alignment optimization; S130 completes the generation of 3D model results, model lightweighting, and block-level incremental updates through the scene reconstruction and incremental update publishing module.

2. The method for large-scale disaster scenario modeling based on UAV multi-source scanning data fusion according to claim 1, characterized in that, S110 includes: S110-1: Acquire multi-view image data of the disaster site, form an image sequence covering the target area, and record the acquisition timestamp and shooting parameter information of each frame of the image; S110-2 acquires airborne laser point cloud data from the disaster site, forms a point cloud sequence representing the spatial geometry, and records the acquisition timestamp and scanning attribute information of each frame of point cloud. S110-3, Obtain the coordinate and attitude information corresponding to the image data and point cloud data, and bind them with the image frame and point cloud frame timestamp to establish the correlation between multiple source observations at the same time, forming a multi-source observation dataset that can be used for fusion processing.

3. The method for large-scale disaster scenario modeling based on UAV multi-source scanning data fusion according to claim 2, characterized in that, S120 includes: S120-1 performs data preprocessing, synchronously correlates and controls the quality of multi-source observation data to form an effective observation set. This includes establishing the correspondence between image frames and point cloud frames based on timestamps, performing image correction and key frame filtering, and performing anomaly removal and filtering downsampling on point clouds. S120-2, Perform geometric prior generation, construct a block geometric base model based on point cloud and obtain the initial pose and sparse structure results on the image side. This includes dividing the point cloud into blocks according to spatial range and establishing a block index structure. At the block level, a block geometric base model is formed to represent the overall contour of the scene. At the same time, multi-view geometric consistency constraints are established based on the key frame set to solve the camera pose trajectory and sparse structure results. Cross-modal geometric association constraints are constructed in the common coverage area of ​​the sparse structure results on the image side and the block geometric base model. S120-3 performs multi-source fusion alignment optimization, completes cross-modal fusion registration and coordinate unification under unified scale and consistency constraints, and outputs three-dimensional structure results in a unified coordinate system. The three-dimensional structure results include unified scale pose trajectory, fused point cloud, and index and statistical information associated with the block geometric base model.

4. The method for large-scale disaster scenario modeling based on UAV multi-source scanning data fusion according to claim 3, characterized in that, S130 includes: S130-1 performs mesh reconstruction based on the fused point cloud output by S120-3, generates a contour mesh model, and outputs it together with the fused point cloud as a 3D model result of the disaster scene. S130-2, Optimize and lightweight the 3D model output; S130-3 performs block-level change determination and local incremental updates for newly added observation data, triggers reconstruction and fusion updates only for changed areas, and simultaneously generates coverage and confidence metadata for result evaluation and release management, thereby supporting rapid iterative updates under continuously changing conditions at disaster sites.

5. A method for large-scale disaster scenario modeling based on UAV multi-source scanning data fusion according to claim 4, characterized in that, S130-3 includes: The block-level change determination process is as follows: establish change scores at the block level. ,when This block update is triggered when the threshold is exceeded: ， In the formula These represent the block coverage before and after the update. These represent the block point density before and after the update, respectively. This is a statistic of the block-merged residuals. These are the weighting coefficients.

6. The method for large-scale disaster scenario modeling based on UAV multi-source scanning data fusion as described in claim 1, characterized in that, This method is applicable to multi-rotor UAV platforms equipped with airborne lidar and visible light cameras, and the UAV platform has coordinate and attitude acquisition capabilities to simultaneously output multi-view image data, airborne lidar point cloud data, and coordinate and attitude data.

7. A disaster scenario large-scale modeling device based on UAV multi-source scanning data fusion, used to execute the method according to any one of claims 1-6, characterized in that, include: The UAV multi-source data acquisition module is used to acquire and organize multi-source scanning data at disaster sites. The multi-source data fusion and geometric optimization module is used to sequentially complete data preprocessing, geometric prior generation, and multi-source fusion alignment optimization. The Scene Reconstruction and Incremental Update Publish module is used to generate 3D model results, lightweight the model, and perform block-level incremental updates.

8. A computing device, characterized in that, include: At least one processor and a memory storing program instructions; When the program instructions are read and executed by the processor, the computing device performs the method as described in any one of claims 1-6.

9. A readable storage medium storing program instructions, characterized in that, When the program instructions are read and executed by the computing device, the computing device performs the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images