A method and system for extracting and identifying buried targets based on multi-source data fusion

Abstract

This invention provides a method and system for extracting and identifying buried targets based on multi-source data fusion, relating to the fields of disaster relief and multi-source data fusion. It enables horizontal and vertical fusion of multi-source data related to disaster situations. Specifically, it extracts disaster information and determines disaster clues through horizontal fusion, then identifies collapsed buildings based on basic geographic data to obtain preliminary judgments, and finally vertically fuses mobile phone signal data, radar signal data, etc., to confirm buried targets. This invention, by fusing multi-source data, can quickly and accurately identify buried targets awaiting rescue.

Description

Technical Field

[0001] This invention relates to the fields of disaster relief and multi-source data fusion, specifically to a method and system for extracting and identifying buried targets based on multi-source data fusion. Background Technology

[0002] From the perspective of safeguarding the lives and safety of the people, emergency management departments need to quickly locate the affected population and dispatch rescue forces according to the situation on the ground. Therefore, the rapid identification and location of targets awaiting rescue in disaster-stricken areas has become a key development requirement for the emergency response industry. Typically, due to the large area of ​​the disaster zone and the complexity of the situation, traditional target search mechanisms are insufficient to meet the timeliness requirements of post-disaster rescue, and the efficiency and accuracy of identifying various targets awaiting rescue are low. With the widespread application of technologies such as drones, remote sensing imagery, photoelectric scanning, and miniature detection radar, emergency management departments have formed a relatively complete technology and equipment system. During the disaster occurrence and post-disaster response phases, the flexible organization of various equipment and methods enables the rapid collection of on-site data from the disaster area and its aggregation in the disaster processing and analysis system. These search equipment and methods are carried out sequentially from multiple perspectives, scales, and granularities in practical applications. This leads to a large congestion of data from various sources in the disaster processing and analysis system.

[0003] Data from different sources presents the following problems in disaster analysis: 1. Disaster areas are vast, and due to limitations in equipment and technology, the coverage of collected data is limited, requiring the aggregation of multiple datasets to achieve full coverage; 2. During data aggregation, differences in the time granularity of data collection from different sources lead to inconsistencies in the timeliness of the datasets achieving full coverage; 3. For the same local area, different equipment and methods may collect data multiple times, resulting in differences in the accuracy of the acquired data and leading to some controversy in target identification results. These problems significantly impact the rapid analysis and judgment of disaster targets in disaster areas. Summary of the Invention

[0004] In response to the needs of urban disaster relief and the characteristics of search equipment, this invention aims to propose a method and system for extracting and identifying buried targets based on multi-source data fusion, which can fuse multi-source data related to disasters and quickly and accurately identify targets to be rescued.

[0005] To achieve the above objectives, the technical solution proposed by this invention is as follows:

[0006] A method for extracting and identifying buried targets based on multi-source data fusion includes the following steps:

[0007] 1) Collect multi-source data about the disaster area, including image data, third-party disaster information, basic geographic data, and mobile phone signal data;

[0008] 2) Based on the image data, disaster information is extracted and fused with the third-party disaster information to obtain disaster clues;

[0009] 3) Based on the disaster clues and the basic geographic data, identify the damage status of buildings and use the identified collapsed buildings as the preliminary judgment data;

[0010] 4) Perform spatiotemporal clustering on the mobile phone signal data, and then perform heterogeneous data vertical fusion processing with the preliminary judgment result data to obtain the identified target and the target to be investigated, and extract the buried target from the identified target;

[0011] 5) For the target to be investigated, the radar is used to scan the buried points with unknown status to obtain radar signal data and return it to step 4) to participate in the vertical fusion processing.

[0012] Furthermore, in step 2), the disaster area is first rasterized and divided into multiple horizontally and vertically arranged grids, each grid being a rectangular area; then, disaster information is extracted based on the rasterized disaster area, and the spatiotemporal attributes of the disaster information are formed.

[0013] Further, in step 2), the quality of the image data is first evaluated and labeled; then, disaster information is extracted and encapsulated in combination with its spatiotemporal attributes; then, the encapsulated disaster information is horizontally fused with third-party disaster information. During the horizontal fusion process, the disaster information from different sources is aggregated according to its spatiotemporal attributes, and then a spatiotemporal verification window is established. Multiple similar datasets entering the same window are fused, and the quality is verified according to the above-mentioned labels to obtain definite disaster clues.

[0014] Furthermore, for uncertain disaster clues, search tasks are constructed based on the specific categories of impact data, and certain disaster clues are obtained by carrying out search tasks.

[0015] Furthermore, the step of evaluating the quality of the image data includes:

[0016] First, a rasterized multi-source image dataset is obtained, and then all flipped raster information is obtained according to the dataset coverage. This raster information includes raster ID identifiers and coordinates.

[0017] Next, a local quality evaluation queue (QAAss) is established for the current dataset. The resolution and timestamp of the current dataset source are obtained, as well as the distance, field of view offset angle, height and visibility of the current dataset source to the grid center point. The QG value is then calculated.

[0018] By introducing the standard distance, resolution, offset angle, and height values ​​corresponding to the current dataset source, the QG value is standardized to obtain the PQG value.

[0019] Obtain the weight configuration corresponding to the information source of the current dataset, perform weight calculation on the PQG value, and obtain the quality value of the current dataset.

[0020] Furthermore, horizontal fusion processing validates multiple data sources based on their quality, including:

[0021] Based on the quality of the data source at the corresponding grid position of the disaster clue, the disaster clue is marked. According to the threshold setting, when the quality value exceeds the threshold, it is marked as a confirmed disaster clue; otherwise, it is marked as an uncertain disaster clue.

[0022] Under the condition of satisfying the spatiotemporal constraints, if there are multiple disaster clues from multiple data sources, these disaster clues are sorted in ascending order according to the time series. If the quality of the subsequent disaster clue is greater than the quality of the preceding disaster clue, the subsequent disaster clue is taken as the definite disaster clue at the current location; if the quality of the subsequent disaster clue is less than the quality of the preceding disaster clue, the disaster clue at the current location is taken as the definite disaster clue.

[0023] Under the condition of satisfying the spatiotemporal constraints, if there are multiple disaster clues from multiple data sources, these disaster clues are sorted in ascending order according to the time series. If the quality of the subsequent disaster clue is equal to the quality of the preceding disaster clue, the average value of the disaster information of the two is calculated to obtain the definite disaster clue at the current location.

[0024] Furthermore, during vertical fusion processing, a filtering mechanism is established based on the time series, and a time verification window is established through spatiotemporal constraints. The disaster information extracted from the heterogeneous data entering the window is aggregated, and the target information is determined by judging whether a spatiotemporal matching relationship can be formed.

[0025] Furthermore, the criteria followed in the vertical fusion processing of heterogeneous data include:

[0026] Heterogeneous data form matching relationships based on spatiotemporal constraints, and the resulting data becomes the target for investigation.

[0027] Heterogeneous data cannot form a matching relationship due to spatiotemporal constraints, and the resulting data becomes the target to be searched.

[0028] Heterogeneous data cannot form effective matching relationships, resulting in invalid targets.

[0029] Furthermore, fusion logs are recorded during horizontal and vertical fusion processing respectively; the logs are updated when new data is fused and updated; and a self-checking mechanism is established to check and maintain the update status of fusion information by checking the logs.

[0030] A system for extracting and identifying buried targets based on multi-source data fusion, comprising:

[0031] The clue extraction module includes a disaster raw database, a disaster information extraction database, and a disaster clue database. First, the collected image data and third-party disaster information are stored in the disaster raw database. Then, the disaster information of the affected points extracted based on the image data and the third-party disaster information are stored in the disaster information extraction database. Next, the disaster information is horizontally fused, and the resulting disaster clues are stored in the disaster clue database.

[0032] The preliminary assessment module includes a basic geographic database and a preliminary assessment result database. First, the collected basic geographic data is stored in the basic geographic database. Then, disaster clues are combined with the basic geographic data to identify the damage status of buildings. The identified collapsed buildings are used as preliminary assessment result data and stored in the preliminary assessment result database.

[0033] The fusion processing module includes a target database of buried locations and a raw mobile signal database. First, the collected mobile signal data is stored in the raw mobile signal database. Then, the mobile signal data is subjected to spatiotemporal clustering and vertical fusion processing with the preliminary judgment result data to obtain the identified targets and the targets to be investigated, which are then stored in the target database of buried locations.

[0034] The target identification module includes a identified target library, a target to be investigated library, and a radar scan raw database. First, it distinguishes between identified targets and targets to be investigated, stores identified targets and invalid templates in the identified target library, and stores the templates to be investigated in the target to be investigated library. Then, for targets to be investigated, it conducts reconnaissance by scanning buried points with unclear radar scan status, obtains radar signal data, and stores it in the radar scan raw database for vertical fusion processing.

[0035] This invention is used for the unified aggregation and processing of disaster area image data, mobile phone signal scanning data, radar scanning data, and third-party disaster data. It achieves the fusion processing of different types of data streams through different modules, with each module processing one type of data. Within each module, the raw database stores all incremental raw data. The process database stores all extracted valid information. The result database stores the fusion results of extracted information (of the same type but from different sources). The fusion processing of the result database within the same module is called horizontal fusion.

[0036] During horizontal fusion processing, a quality assessment model is established based on the scene-object relationship. Quality assessment calculations are generated based on factors such as the operational status of the acquisition task, visibility / occlusion, angle distance, and imaging equipment parameters during imaging at specific points. When horizontally fusing cross-over existing and incremental information, a judgment is made based on the quality comparison between the existing and incremental data. The judgment rules are as follows: when the quality of existing data is higher than that of incremental data, the information in the existing data is treated as a reliable result, and the corresponding incremental data status is set to "not adopted"; when the quality of existing data is lower than that of incremental data, the information in the incremental data is published as a reliable result in the result database, and the corresponding existing data status is changed to "not adopted"; when the quality of existing data is equal to that of incremental data, both existing and incremental data are set to an unreliable state. The weighted average of the two results is then used to form a new result, which is stored as a reliable record in the result database.

[0037] After obtaining disaster information, the results are matched with the designated disaster-stricken areas and building objects. The damage status of buildings within the area is extracted using image recognition and other methods to form a preliminary assessment, which is then published in the results database. Incremental mobile phone signal scanning data is spatially clustered to generate mobile phone signal distribution information. This information is then fused with the preliminary assessment results to identify buried targets. During this process, aggregation and discrimination are performed based on spatiotemporal coverage. If the initially assessed target and the mobile phone distribution form effective coverage, it is identified as a confirmed target; otherwise, it is identified as an invalid target. If there is a predetermined aggregation gap between the mobile phone signal and the initially assessed target in spatiotemporal space, it is identified as a target to be investigated. Confirmed and invalid target information is stored in the identified target database, while target to be investigated information is stored in the target to be investigated database. During the incremental process of the target to be investigated database, a reconnaissance mission request is established and sent to the command system to achieve target-driven mission command. The aggregated radar scanning data is spatiotemporally matched with the data in the preliminary target database, and the status of the target to be investigated is changed based on the results. In the result databases of different modules, the status of data records can be changed through manual editing, and the current record information can be recursively processed in the result databases of different modules, forming a "human-in-the-loop" intelligence processing framework. Attached Figure Description

[0038] Figure 1 This is a flowchart of the buried target extraction and identification method based on multi-source data fusion of the present invention.

[0039] Figure 2 This is a flowchart of the basic incremental data processing.

[0040] Figure 3 This is a diagram showing the area coverage.

[0041] Figure 4This is a diagram illustrating data acquisition.

[0042] Figure 5 This is a flowchart of data quality processing.

[0043] Figure 6 This is a flowchart of the horizontal fusion processing.

[0044] Figure 7 This is a flowchart of the vertical fusion processing.

[0045] Figure 8 This is a framework diagram of the buried target extraction and identification system based on multi-source data fusion of the present invention.

[0046] Figure 9 This is a flowchart of the process for merging and processing imagery with third-party disaster data.

[0047] Figure 10 This is a flowchart of data processing between mobile phones and radar.

[0048] Figure 11 This is a data self-check flowchart. Detailed Implementation

[0049] To make the above features and advantages of the present invention more apparent and understandable, specific embodiments are described below in conjunction with the accompanying drawings.

[0050] In response to the operational needs of my country's emergency rescue departments, this invention proposes a method for extracting and identifying buried targets based on multi-source data fusion. The multi-source data mainly includes the following categories:

[0051] Image data: Visible light and infrared image data of the disaster area acquired through remote sensing, aerial photography, and other methods. This type of data is used to extract disaster information and related objects using artificial intelligence and other techniques.

[0052] Mobile signal data collection: Using data collection equipment, mobile signal data is collected over a wide area of ​​the disaster zone. This data is used to extract personnel location information and identify areas where people are congregating.

[0053] Third-party disaster information: Disaster information obtained through emergency departments and other operational departments. This information is used to mark the location, level, and scope of impact of the damage points on site.

[0054] Radar signal data: Data transmitted back from the life detection radar after scanning specific buildings and spaces on site to obtain information about people;

[0055] Basic geographic data: Provides information on basic topography, building distribution, population distribution, key target buildings and facilities in the disaster area.

[0056] The ultimate goal of disaster relief is to accurately obtain information on trapped and buried individuals; this information is referred to as buried targets. In the data processing and analysis process, a panoramic dataset covering the entire region is formed by aggregating data collected over a wide area. Analysis and information extraction are performed on specific categories of data to create clues. Based on the spatiotemporal constraints provided by these clues, the state of building objects in the dataset is extracted to identify collapsed or damaged buildings, forming preliminary targets. Based on these preliminary targets, information on buried individuals is accurately extracted by combining various signal collection data, ultimately leading to the identification of buried targets within the disaster area. This basic process is as follows: Figure 1 As shown, a data processing and analysis system is formed based on the characteristics of different types of data. The final conclusion is reached through data fusion in conjunction with the needs of emergency rescue work. Based on this system architecture, this invention proposes a method for extracting and identifying buried targets based on multi-source data fusion. This method mainly includes unified processing of multi-granularity data, temporal maintenance of the entire dataset, coupling of multi-characteristic information of targets under spatiotemporal constraints, unified identity recognition of targets throughout the entire lifecycle, and target-driven collaborative acquisition tasks.

[0057] 2. Implementation Content

[0058] Generally, disaster areas cover a relatively large spatial area. Data collected on-site by various equipment and methods has limited coverage, often requiring spatial stitching to create a panoramic dataset for disaster information extraction. This dataset supports higher-level data analysis and target identification. Furthermore, the distribution of key objects within disaster areas is complex, including schools, hospitals, and vital lifeline facilities. Continuous monitoring of these objects is necessary. Therefore, from a dataset organization perspective, the disaster area space is rasterized. The disaster area space is divided into a two-dimensional grid:

[0059] TArea = { grid ij | i =1, 2, .... nj = 1,2, .... m}

[0060] grid = { ID , WNPt , SEPt , CPt , W , H}

[0061] That is, the disaster area is composed of a set of two-dimensional grid objects, each grid object being a rectangular region;

[0062] i and j represent the sequence numbers in the horizontal and vertical directions, respectively;

[0063] ID: This is the identifier of the raster object. It is globally unique and is used not only to mark the identity information of the raster, but also to encode and map the geographic relationships of multi-source data.

[0064] WNPt: Coordinates of the northwest corner vertex of this grid;

[0065] SEPt: The coordinates of the southeast corner vertex of this grid;

[0066] CPt: Coordinates of the center point of this grid;

[0067] W and H represent the width and length of the grid, respectively.

[0068] After a disaster occurs, the disaster area is divided into grids to form a grid queue. During the division process, W and H are set to constant values ​​(e.g., W=50 meters, H=50 meters).

[0069] This grid division provides support for subsequent multi-source data, including indexing, quality verification, and fusion processing.

[0070] 3.1 Data Processing

[0071] 3.1.1 Incremental processing of the dataset

[0072] During the disaster information processing, as various types of data increase, the system performs information extraction and fusion processing. Figure 2 The document illustrates the basic processing steps for a dataset. After acquiring a dataset from a source (image data or third-party disaster data), the quality of the current dataset is first evaluated and quantified based on operational information. The specific calculation methods are described in detail in the next section. After completing all quality labeling of the dataset, its purpose is determined based on its category. On this basis, disaster information is extracted from it using artificial intelligence or specialized data processing tools and encapsulated by combining its spatiotemporal attributes (i.e., the time and spatial location information corresponding to the information). After completing all processing operations on the current dataset, the extracted disaster information encapsulation results are fused with a target status database to drive target extraction or search tasks.

[0073] During the fusion process, disaster information from different sources is aggregated based on its spatiotemporal attributes. A spatiotemporal verification window is established during this process. For the results extracted from multiple similar datasets entering the same window, quality is verified to form a unique and definitive conclusion. This conclusion is then verified against historical information under current constraints to ensure the accuracy of target identification.

[0074] On the other hand, for disaster information that cannot form a definitive conclusion, a search task can be constructed based on its status and the type of information source data to assist commanders in carrying out precise search tasks and ultimately form accurate clues and conclusions through the retrieval of the task.

[0075] 3.1.2 Dataset Quality Assessment Calculation

[0076] Due to the open nature of the disaster area and the search mission, data was collected in multiple frames. During this process, different frames overlapped in time and space. Extracting disaster information from these overlapping areas required evaluating the quality of multiple data sets to arrive at accurate conclusions. Figure 3 As shown.

[0077] like Figure 4 As shown, disaster information (Tinfo) is extracted based on the collected data (DataSet). In areas with overlapping coverage, multiple data sources exist; therefore, each data source forms an extraction result set (TinfoList) in this area. Ideally, the result sets extracted from data collected by different sources should be completely consistent. However, in actual operation, the data quality varies due to factors such as the operating conditions, angle, and distance of the data sources. Conflicts exist between the disaster information result sets extracted from different sources, making it difficult to reach definitive conclusions and affecting subsequent target identification and rescue efforts. Therefore, it is necessary to establish a data quality assessment mechanism tailored to the characteristics of emergency response to ensure effective disaster information extraction.

[0078] Based on the raster division of the disaster area, the collected dataset (DataSet) typically covers a set of rasters. Its quality is composed of a set of local evaluation metrics segmented from this set of rasters (i.e., a local quality evaluation queue):

[0079] QAAss(DataSet) = { QG ij | i = p,....p+kj = q, .... q + l}

[0080] Where: i, j are the horizontal and vertical sequence numbers, p is the sequence number corresponding to the start position of the horizontal row, k is the number of grid cells in the horizontal row, and q is the sequence number corresponding to the start position of the vertical row. l The number of vertical grid cells.

[0081] QG = { gridID , DisVal , ResVal , AngVal , AltVal , VisVal ,timeStamp , DataSetID}

[0082] in:

[0083] gridID: The identifier of a specific raster covered by the current dataset;

[0084] DisVal: The distance of the data source from the center point of that raster;

[0085] ResVal: The resolution value of the information source in this dataset;

[0086] AngVal: The angle of field of view offset of the data source to the center point of the raster.

[0087] AltVal: The height value of the raster center point relative to the information source in this dataset;

[0088] VisVal: The visibility metric between the information source and the center point of the raster in this dataset;

[0089] timeStamp: The timestamp of the dataset;

[0090] DataSetID: The dataset's own identifier.

[0091] To facilitate quality assessment calculations, the values ​​in QG are processed using standard values:

[0092] PQG(QG) = { gridID , D , R , A , AL , V , timeStamp}

[0093] D = DisVal / StdD, R = ResVal / StdR, A = AngVal / StdA, AL = AltVal / StdAL, V = VisVal;

[0094] Where StdD, StdR, StdA, and StdAl are the standard distance, resolution, offset angle, and height values ​​corresponding to the current information source.

[0095] After standardization, the following calculations can be performed to quantify the quality of the current dataset:

[0096] getQAssVal(PQG(QG)) = { gridID , gridQVal , timeStamp}

[0097] gridQVal = w1* D + w2* R + w3* A + w4* Al + w5* V

[0098] Where w1, w2, w3, w4, and w5 are the weight configurations corresponding to the current information source. For example, based on experience, the weights configured for aerial image information sources are {0.15, 0.3, 0.2, 0.15, 0.2}.

[0099] Figure 5 The image shows a dataset where quality evaluation is performed on all covered rasters.

[0100] 3.2 Disaster Information Integration

[0101] In post-disaster relief efforts, a large amount of diverse, multi-source data forms the basis for disaster analysis and target identification. To reach definitive conclusions and avoid unnecessary confusion, the analytical conclusions from these data need to be compared, verified, and processed to ultimately arrive at a definitive result. This process primarily involves two types of data fusion: horizontal fusion and vertical fusion.

[0102] 1. Horizontal Fusion: Horizontal fusion refers to the fusion of disaster information extracted from multiple sources of similar data. For example, disaster information extracted from multiple remote sensing images and aerial images is fused to form the final disaster information conclusion. Horizontal fusion achieves deduplication of the multi-source data extraction results. During the horizontal fusion process, a constraint window is established based on spatiotemporal constraints, and the image data extracted within the window period is processed uniformly. During the processing, the multi-source data extracted within the same area is uniformly verified according to quality to ensure the certainty and accuracy of the final conclusion; the process is as follows: Figure 6 As shown.

[0103] In this process, the comparison and verification of result objects from multiple data sources follows these principles:

[0104] Based on the quality of the data source at the corresponding grid position of the clue result, the result is marked. According to the threshold setting, when the quality value gridQval exceeds the threshold, it is marked as a definite result; otherwise, it is marked as an uncertain result.

[0105] Under the condition of satisfying the spatiotemporal constraints (i.e., entering the same window period), if there are multiple data sources extracting result objects, i.e., disaster clues, then these objects are sorted in ascending order according to the time series. If the quality of the subsequent object is greater than the quality of the preceding object, then the subsequent object is taken as the conclusion of the current position; if the quality of the subsequent object is less than the quality of the preceding object, then the conclusion of the current position is not changed.

[0106] If multiple data sources yield result objects while satisfying spatiotemporal constraints, these objects are sorted in ascending order based on their time series. If the quality of a subsequent object equals the quality of a preceding object, the average quality values ​​of the two disaster information are calculated, and the resulting value is taken as the final result for the current location.

[0107] 2. Vertical Fusion: The final information about a target consists of multiple attributes derived from different data sources. During processing, based on spatiotemporal constraints and other conditions, information extracted from different data sources is matched with the target attributes, and then aggregated to form a complete target identification result. This process is called vertical fusion. Vertical fusion refers to the processing of disaster information extracted from heterogeneous data. For example, disaster information extracted from remote sensing image data and mobile phone signal data is fused to form the final target identification conclusion. In the vertical fusion process, filtering is established based on time series. On this basis, a verification window is established through spatiotemporal constraints. The heterogeneous data extraction conclusions entering the window are aggregated to achieve spatiotemporal matching, ultimately forming a deterministic conclusion. The process is as follows: Figure 7 As shown.

[0108] Through vertical fusion, disaster information extracted from heterogeneous data is ultimately aggregated into target objects. Due to limitations in the technical mechanism, some target characteristic information collected by reconnaissance methods may miss detections or checks. This results in vertical fusion at that location failing to reach a definitive conclusion. To address this situation, the state of the object at that location is marked as a target to be investigated. This method drives the implementation of other precise search tasks to achieve accurate perception of the overall situation in the disaster area.

[0109] Vertical fusion is used to aggregate target information. This process follows these principles:

[0110] Heterogeneous data form matching relationships based on spatiotemporal constraints, and the resulting data becomes the target for investigation.

[0111] Heterogeneous data cannot form a matching relationship due to spatiotemporal constraints, and the resulting data becomes the target to be searched.

[0112] Heterogeneous data cannot form effective matching relationships, resulting in invalid targets.

[0113] After the vertical fusion recognition results are generated, these results need to be verified with the historical results of target recognition to ensure that the target's identity is consistent over time and to avoid confusion. Please refer to the corresponding content in the next section for relevant information.

[0114] 3.3 Data Organization, Target Identification, and Status Maintenance

[0115] 3.3.1 Multi-source data fusion framework composition

[0116] To meet the requirements of rapid search and target identification in disaster areas, a fusion processing system needs to be built based on the aggregation of multi-source data, forming a production-supply relationship according to the purpose of different types of data. This system enables the organization and management of data flow based on target-driven principles. Combining the aforementioned technical approach, the overall organization of this system is as follows: Figure 8 As shown.

[0117] The buried target extraction and identification system based on multi-source data fusion provided by this invention mainly includes several basic modules such as clue extraction, preliminary judgment processing, fusion processing, and target identification. These can be displayed as four layers from bottom to top according to the processing order, as follows: Figure 8 As shown, where:

[0118] Clue Extraction Module: This module is at the bottom layer and consists of several parts, including the original disaster database, the disaster information extraction database, and the disaster clue database. The disaster clue database serves as the output of this module and the node for external services. During incremental processing, image data and third-party disaster data are exchanged and stored in the local original database. Simultaneously, after information extraction, information on affected locations is extracted and necessary characteristics are encapsulated to form a complete disaster information extraction result, which is then archived in the disaster information extraction database. Subsequently, the current incremental disaster information extraction result is horizontally fused with the historical records in the disaster clue database. The resulting data is then uniformly stored and its status is updated and maintained in the disaster clue database.

[0119] Preliminary Assessment Processing Module: This module sits at the next higher level and consists of a basic geographic database and a preliminary assessment result library. The preliminary assessment result library serves as the output and external service node for this module. During operation, it identifies the damage status of buildings based on disaster clues and basic geographic data, thus forming a set of building collapse identification results. These results are encapsulated and stored in the preliminary assessment result library for status maintenance. This provides necessary data support for identifying burial points. Simultaneously, combined with basic geographic data, a continuous status tracking mechanism is established for some important building objects to meet the requirements of disaster relief and emergency response.

[0120] Fusion Processing Module: This module sits at a higher level and mainly consists of two parts: a target database of buried locations and a raw mobile phone signal database. The target database of buried locations serves as the output of this module and the node for external services. During incremental processing, wide-area mobile phone signal distribution data collected through drones, ground vehicles, and portable devices is aggregated and stored in the local raw database. Based on this, the mobile phone signal distribution data undergoes spatiotemporal clustering and is vertically fused with the data in the preliminary judgment result database to form identified targets (buried) and targets to be investigated. After time-series verification, newly added targets are classified and stored in the corresponding databases.

[0121] Target Recognition Module: This top-level module primarily consists of a identified target database, a target database for investigation, and a raw radar scan database. On one hand, based on the processing results of the modules mentioned above, it maintains the basic status of each target in its identified target database. On the other hand, it establishes a task-driven mechanism for targets to be investigated, guiding the UAV to conduct precise reconnaissance of unidentified burial sites via radar scanning. The collected radar data is aggregated in the local raw radar scan database. After vertical fusion processing with the burial site database, the corresponding target information is maintained.

[0122] Based on the aforementioned system, this invention enables the implementation of the multi-source data fusion-based method for extracting and identifying buried targets, as provided by this invention. By combining it with several information acquisition devices, it can achieve... Figure 1-11 The entire processing procedure, including the main steps, is as follows:

[0123] Step 1: Based on the current characteristics of my country's emergency response and related industries, the regional search data mainly includes visible light / infrared image data, third-party disaster information, mobile phone signal scanning data, and radar scanning data. Identification of buried targets on-site is achieved based on this data aggregation.

[0124] Step 2: After the emergency response is initiated, connection links are established for different types of data, and the system provided by this invention is activated. This system consists of multiple modules, each processing a type of data. Each module mainly comprises three parts: raw data, process data, and a result database. Within the same module, results of the same type of data from different sources are uniformly stored in the process data, and the fusion results are stored in the result database. Between different modules, the result data of the lower-level module is vertically fused with the process data of the upper-level module.

[0125] Step 3: This system processes image data as the underlying data, extracts disaster information from it using image recognition and other methods, and stores it in the process library after version encapsulation. Simultaneously, incremental third-party disaster information is also encapsulated and stored in the process library.

[0126] Step 4: After completing Step 3, perform horizontal fusion processing on the incremental disaster information and the existing data in the results database. First, determine the overlap area between the incremental and existing data based on spatiotemporal constraints. Incremental disaster information that does not overlap is directly stored in the results database. Incremental disaster information that overlaps with existing data is horizontally fused, and the fusion result is stored in the results database.

[0127] Step 5: After completing Step 4, extract updated disaster information from the results database and, in conjunction with basic geographic data, identify the building damage status at the disaster sites. Mark collapsed or damaged buildings to form preliminary assessment targets, and publish them in the preliminary assessment layer's results database.

[0128] Step 6: During operation, acquire mobile signal scanning data through the connection link. Store the raw data in the original database of the module. Perform spatial aggregation on the incremental mobile scanning data and store the results in the process library of the module. After completing this operation, vertically fuse the spatial aggregation results with the preliminary target in the result library of the preliminary judgment layer.

[0129] Step 7: Based on the fusion processing results in Step 6, generate the identified target results and the pending target results. The identified target results are those that accurately identify cell phone signals of people buried under collapsed buildings. The pending target results are those for which cell phone signals of people buried under collapsed buildings cannot be accurately identified. The identified target results are stored in the identified target database, and the pending target results are stored in the pending target database.

[0130] Step 8: In accordance with the regulations of emergency operations and the command system, a search mission request is generated based on the target information and sent to the command system to drive the execution of the on-site precision (radar) reconnaissance mission.

[0131] Step 9: At the radar layer, incremental radar scan data is fused with the target database. Based on the conclusions, confirmed targets (collapsed buildings with buried personnel) and invalid targets (collapsed buildings without buried personnel) are generated and published in the results database. Simultaneously, the status of the corresponding data in the target database is updated.

[0132] Step 10: Throughout the entire emergency rescue process, the above data are aggregated in the system at their respective time intervals for unified processing. Different application systems extract relevant information from the result database in their respective modules until the emergency rescue work is completed and the processing system is shut down.

[0133] The above method enables the triggering and initiation of the entire data processing flow and the control of information flow between different stages through the aggregation process of external data sources. On the other hand, based on the characteristics of emergency rescue operations and the rules for implementing search tasks in open environments, a polling and monitoring mechanism is established around the output result databases of different modules. The status of data records in the node databases of different modules is monitored and maintained throughout their lifecycle at fixed intervals (e.g., according to my country's emergency management regulations, the default interval is 5-15 minutes), ensuring the timeliness and accuracy of disaster information in complex disaster areas. Please refer to subsequent chapters for related content.

[0134] 3.3.2 Object Information Modeling

[0135] As previously stated, considering the characteristics of emergency response operations and the complex environment of disaster areas, this invention proposes a method for extracting and identifying buried targets based on multi-source data fusion. It relies on a hierarchical data framework to construct a data production-supply mechanism on a data-driven basis. Horizontal fusion enables the processing of disaster information from multiple sources of similar data, while vertical fusion enables the fusion of dissimilar disaster data, ensuring the complete extraction and correct aggregation of target information. In this process, considering the inherent limitations of different data types, a discrimination mechanism is established for targets, and multi-method cross-validation is achieved through a target-driven approach to ensure the completeness and accuracy of disaster information. Furthermore, a polling detection mechanism is established around a specific database within the system to track and monitor the state changes of various objects, ensuring data timeliness and achieving full lifecycle tracking and state maintenance of targets. Through these methods, a two-layer closed data processing loop is established, improving efficiency in the application environment.

[0136] Within the aforementioned system framework, the node library and its corresponding object data model formed around each processing stage are as follows:

[0137] 1. Disaster Clues: These are mainly used to describe specific local areas where damage has occurred, and are defined as follows:

[0138] ptInfo = { evtID , ID , createTime , lastUpdateTime , location ,geoShape , geoCode , type , level , curState , detailInfo , isfocus}

[0139] in:

[0140] evtID: The identifier for the current disaster event;

[0141] ID: A globally unique identifier for the current object;

[0142] createTime: The record created when this record was created;

[0143] lastUpdateTime: The last status update record for this record;

[0144] location: The center location of the region where this record occurred;

[0145] geoShape: The projected shape of the damaged area corresponding to this record;

[0146] geoCode: The geocode of the area where the damage occurred, corresponding to this record;

[0147] type: The type of disaster that caused the damage recorded in this document;

[0148] level: The level at which this record has been corrupted;

[0149] curState: This represents the current state of this record, and mainly includes the following basic states:

[0150] curState = { none , valid , fogged , ended , canceled}

[0151] Where: none: currently in an undefined state; Valid: currently in a valid state; Fogged: currently in a state where details are pending; Ended: the current record object has completed all processing operations; Canceled: the current record object will not be processed further.

[0152] detailnfo: This is the detailed description of this record, which is used to store the JSON or text content of the detailed description.

[0153] isfocus: Whether it is a focus object.

[0154] 2. Preliminary Assessment Result: This is used to describe building facilities in a damaged state, and its definition is as follows:

[0155] objInfo = { evtID , ID , ptInfoID , objID , createTime ,lastUpdateTime , ocation , geoCode , type , level , curState , detailInfo ,isfocus}

[0156] in:

[0157] evtID: The identifier for the current disaster event;

[0158] ID: A globally unique identifier for the current object;

[0159] ptInfoID is the disaster information ID corresponding to this record;

[0160] objID: The initial judgment information corresponding to this record;

[0161] createTime: The record created when this record was created;

[0162] lastUpdateTime: The last status update record for this record;

[0163] location: The location corresponding to the building;

[0164] geoCode: The geocode corresponding to this building;

[0165] type: The type of disaster that caused the damage to the building;

[0166] level: The level of damage corresponding to the building;

[0167] curState: This represents the current state of this record, and mainly includes the following basic states:

[0168] curState = { none , confirmed , noChecked , fogged , canceled}

[0169] Where: none: currently in an undefined state; Valid: currently in a valid state; Fogged: currently in a state where details are pending.

[0170] Ended: The current record object has completed all processing operations; Cancel: The current record object will not be processed further.

[0171] detailnfo: This is the detailed description of this record, which is used to store the JSON or text content of the detailed description.

[0172] isfocus: Whether it is a focus object.

[0173] 3. Target Database: This database describes and identifies each buried point object record, and its definition is as follows:

[0174] targetInfo = { evtID , ID , objID , ptID , createTime ,lastUpdateTime , location , geoCode , type , level , manInfo , curState ,detailInfo , targetState}

[0175] in:

[0176] evtID: The identifier for the current disaster event;

[0177] ID: A globally unique identifier for the current object;

[0178] objID: The initial judgment information corresponding to this record;

[0179] ptID: Clue information corresponding to this record;

[0180] createTime: The record created when this record was created;

[0181] lastUpdateTime: The last status update record for this record;

[0182] location: The location corresponding to the burial point;

[0183] geoCode: The geocode corresponding to the burial point;

[0184] type: refers to the category of the disaster that causes damage;

[0185] level: indicates the level of damage.

[0186] manInfo: Basic information of the corresponding personnel involved in the burial.

[0187] curState: This represents the current state of this record, and mainly includes the following basic states:

[0188] curState = { none , valid , ended , fogged , canceled}

[0189] Where: none: currently in an undefined state; Valid: currently in a valid state; Fogged: currently in a state where details are pending; Ended: the current record object has completed all processing operations; Canceled: the current record object will not be processed further.

[0190] detailnfo: This is the detailed description of this record, which is used to store the JSON or text content of the detailed description.

[0191] targetState: The category corresponding to this record, defined as follows:

[0192] targetState = { none , valued , blank}

[0193] Where: None indicates the current object is in a defined state; Valued indicates the current object is a valid object; Blank indicates the current object is an invalid object.

[0194] 4. Target Database: Used to record targets for which information is incomplete and requires further investigation by other means. Its definition is as follows:

[0195] focusTargetInfo = { evtID , ID , objID , ptID , createTime ,lastUpdateTime , location , geoCode , type , level , curState , detailInfo ,isFocus}

[0196] in:

[0197] evtID: The identifier for the current disaster event;

[0198] ID: A globally unique identifier for the current object;

[0199] objID: The initial judgment information corresponding to this record;

[0200] ptID: Clue information corresponding to this record;

[0201] createTime: The record created when this record was created;

[0202] lastUpdateTime: The last status update record for this record;

[0203] location: The location corresponding to the burial point;

[0204] geoCode: The geocode corresponding to the burial point;

[0205] type: refers to the category of the disaster that causes damage;

[0206] level: indicates the level of damage.

[0207] manInfo: Basic information of the corresponding personnel involved in the burial.

[0208] curState: This represents the current state of this record, and mainly includes the following basic states:

[0209] curState = { none , waiting , fogged , issued , ended , canceled}

[0210] Where: none: currently in an undefined state; Valid: currently in a valid state; Fogged: currently in a state where details are pending; Ended: the current record object has completed all processing operations; Canceled: the current record object will not be processed further; Waiting: the current record object is in a state of pending processing.

[0211] detailnfo: This is the detailed description of this record, which is used to store the JSON or text content of the detailed description.

[0212] isFocus: Whether it is a focus of attention.

[0213] 3.3.3 Data Flow Control During Multi-Source Data Access

[0214] Based on the above basic object definitions, the overall data processing flow is as follows: Figure 9 As shown.

[0215] During operation, the increments in mobile phone signal and radar signal data are often driven by different tasks, and therefore are not synchronized with impact data, etc. The update process driven by these data is as follows: Figure 10 As shown.

[0216] 3.3.4 Target Status Detection and Update Maintenance

[0217] As the disaster situation develops, the on-site search mission continues to expand. Simultaneously, due to the limited scale of search resources, some areas are temporarily out of coverage for various reasons. Furthermore, the aforementioned system framework extracts information incrementally, compares and merges it with existing information, and performs other processing. This indirectly leads to some information being affected by the mission and losing its update status at certain stages. Target identification results formed under this background may be permanently marked as erroneous. Therefore, in addition to the aforementioned data-driven approach, the system also needs to establish a self-checking mechanism to check and maintain the update status of existing information, and guide the organization and scheduling of search missions in conjunction with emergency response procedures. This process is as follows: Figure 11 As shown.

[0218] Although the present invention has been disclosed above with reference to embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions made by those skilled in the art to the technical solutions of the present invention should be covered within the protection scope of the present invention, which is defined by the claims.

Claims

1. A method for extracting and identifying buried targets based on multi-source data fusion, characterized in that, Includes the following steps: 1) Collect multi-source data about the disaster area, including image data, third-party disaster information, basic geographic data, and mobile phone signal data; 2) Based on the image data, disaster information is extracted and fused with the third-party disaster information to obtain disaster clues; 3) Based on the disaster clues and the basic geographic data, identify the damage status of buildings and use the identified collapsed buildings as the preliminary judgment data; 4) Perform spatiotemporal clustering on the mobile phone signal data, and then perform heterogeneous data vertical fusion processing with the preliminary judgment result data to obtain the identified target and the target to be investigated, and extract the buried target from the identified target; 5) For the target to be investigated, the radar is used to scan the buried points with unknown status to obtain radar signal data and return it to step 4) to participate in the vertical fusion processing.

2. The method as described in claim 1, characterized in that, In step 2), the disaster area is first rasterized into multiple horizontal and vertical grids, each grid being a rectangular area; then, disaster information is extracted based on the rasterized disaster area, and the spatiotemporal attributes of the disaster information are formed.

3. The method as described in claim 2, characterized in that, In step 2), the quality of the image data is first evaluated and labeled; then, disaster information is extracted and encapsulated in combination with its spatiotemporal attributes; next, the encapsulated disaster information is horizontally fused with third-party disaster information. During the horizontal fusion process, the disaster information from different sources is aggregated according to its spatiotemporal attributes, and then a spatiotemporal verification window is established. Multiple similar datasets entering the same window are fused, and the quality is verified according to the above-mentioned labels to obtain definite disaster clues.

4. The method as described in claim 3, characterized in that, For uncertain disaster clues, search tasks are constructed based on the specific categories of impact data, and certain disaster clues are obtained by carrying out search tasks.

5. The method as described in claim 3, characterized in that, The steps for evaluating the quality of the image data include: First, a rasterized multi-source image dataset is obtained, and then the raster information of the entire coverage area is obtained according to the dataset coverage area. This raster information includes raster ID identifier and coordinates. Next, a local quality evaluation queue (QAAss) is established for the current dataset. The resolution and timestamp of the current dataset source are obtained, as well as the distance, field of view offset angle, height and visibility of the current dataset source to the grid center point. The QG value is then calculated. By introducing the standard distance, resolution, offset angle, and height values ​​corresponding to the current dataset source, the QG value is standardized to obtain the PQG value. Obtain the weight configuration corresponding to the information source of the current dataset, perform weight calculation on the PQG value, and obtain the quality value gridQVal of the current dataset.

6. The method as described in claim 5, characterized in that, Horizontal fusion processing validates multiple data sources based on their quality, including: Based on the quality of the data source at the corresponding grid position of the disaster clue, the disaster clue is marked. According to the threshold setting, when the quality value exceeds the threshold, it is marked as a confirmed disaster clue; otherwise, it is marked as an uncertain disaster clue. Under the condition of satisfying the spatiotemporal constraints, if there are multiple disaster clues from multiple data sources, these disaster clues are sorted in ascending order according to the time series. If the quality of the subsequent disaster clue is greater than the quality of the preceding disaster clue, the subsequent disaster clue is taken as the definite disaster clue at the current location; if the quality of the subsequent disaster clue is less than the quality of the preceding disaster clue, the disaster clue at the current location is taken as the definite disaster clue. Under the condition of satisfying the spatiotemporal constraints, if there are multiple disaster clues from multiple data sources, these disaster clues are sorted in ascending order according to the time series. If the quality of the subsequent disaster clue is equal to the quality of the preceding disaster clue, the average value of the disaster information of the two is calculated to obtain the definite disaster clue at the current location.

7. The method as described in claim 1, characterized in that, During vertical fusion processing, a filtering mechanism is established based on time series, and a time verification window is established through spatiotemporal constraints. The disaster information extracted from heterogeneous data entering the window is aggregated, and the target information is determined by judging whether a spatiotemporal matching relationship can be formed.

8. The method as described in claim 1, characterized in that, The criteria followed in the vertical fusion processing of heterogeneous data include: Heterogeneous data form matching relationships based on spatiotemporal constraints, and the resulting data becomes the target for investigation. Heterogeneous data cannot form a matching relationship due to spatiotemporal constraints, and the resulting data becomes the target to be searched. Heterogeneous data cannot form effective matching relationships, resulting in invalid targets.

9. The method as described in claim 1, characterized in that, During horizontal and vertical fusion processing, fusion logs are recorded separately; when new data is fused and updated, the logs are updated; a self-checking mechanism is established to check and maintain the update status of fusion information by checking the logs.

10. A system for extracting and identifying buried targets based on multi-source data fusion, characterized in that, include: The clue extraction module includes a disaster original database, a disaster information extraction database, and a disaster clue database. First, the collected image data and third-party disaster information are stored in the disaster original database; then, the disaster information of the affected points extracted based on the image data and the third-party disaster information are stored in the disaster information extraction database. Then, the disaster information is horizontally integrated, and the resulting disaster clues are stored in the disaster clue database. The preliminary assessment module includes a basic geographic database and a preliminary assessment result database. First, the collected basic geographic data is stored in the basic geographic database. Then, disaster clues are combined with the basic geographic data to identify the damage status of buildings. The identified collapsed buildings are used as preliminary assessment result data and stored in the preliminary assessment result database. The fusion processing module includes a target database of buried locations and a raw mobile signal database. First, the collected mobile signal data is stored in the raw mobile signal database. Then, the mobile signal data is subjected to spatiotemporal clustering and vertical fusion processing with the preliminary judgment result data to obtain the identified targets and the targets to be investigated, which are then stored in the target database of buried locations. The target identification module includes a identified target library, a target to be investigated library, and a radar scan raw database. First, it distinguishes between identified targets and targets to be investigated, stores identified targets and invalid templates in the identified target library, and stores the templates to be investigated in the target to be investigated library. Then, for targets to be investigated, it conducts reconnaissance by scanning buried points with unclear radar scan status, obtains radar signal data, and stores it in the radar scan raw database for vertical fusion processing.

Images