Space-air-ground multi-source heterogeneous data intelligent fusion method

Abstract

The invention discloses a space-air-ground multi-source heterogeneous data intelligent fusion method, which relates to the technical field of data processing, and comprises the following steps: multi-modal data cleaning and space-time alignment, dynamic knowledge graph construction and semantic association, quality-driven adaptive fusion decision, incremental optimization and explainable decision output. According to the method, wave band adaptive filtering and sliding window outlier detection are adopted, a space-time Kriging interpolation method is combined to unify grid coordinates, the alignment error of multi-source data is reduced, dynamic knowledge graph semantic disambiguation is constructed based on a space-time convolutional network, most cross-source semantic conflicts are eliminated through a high-confidence verification and manual auditing dual-channel mechanism, and the dynamic knowledge graph semantic disambiguation efficiency is improved. By establishing a coverage, definition and continuity quantitative index system and dynamically triggering fusion of a data level, a feature level and a decision level, the fusion precision can be ensured in different environments, a decision path is traced based on a streaming causal traceability engine through Kalman filtering incremental updating in combination with a causal discovery algorithm, and the response delay is reduced.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an intelligent fusion method for heterogeneous data from multiple sources in space, air, and ground. Background Technology

[0002] Multi-source heterogeneous data from space, air, and ground refers to data from different platforms such as satellite remote sensing, aerial remote sensing, and ground monitoring. It is characterized by being multi-source, heterogeneous, and massive. These data come from different sources and are of various types, including satellite imagery, aerial photography, and ground measurements. They need to be integrated through data fusion technology to improve the accuracy and reliability of monitoring. This data has wide applications in many fields such as environmental monitoring, disaster early warning, agricultural management, and urban planning. Due to the complexity and diversity of the sources of multi-source heterogeneous data from space, air, and ground, direct use may encounter difficulties. Data fusion technology plays a key role in this process.

[0003] Data fusion technology refers to the technology of integrating multi-source heterogeneous data from different sources, formats, and structures to extract effective information and form a unified expression. Its core goal is to eliminate redundancy and conflicts and generate more comprehensive, reliable, and high-value information through multi-level data processing (such as data-level, feature-level, and decision-level fusion). Data fusion technology is the key to unlocking the value of multi-source heterogeneous data from space, air, and ground. Through multi-dimensional integration and intelligent analysis, fragmented data is transformed into high-precision decision support information and is widely used in fields such as environment, agriculture, and cities, promoting the realization of a data-driven society.

[0004] Existing multi-source heterogeneous data fusion methods are mostly simple in process, resulting in data distortion due to incomplete noise filtering, fusion misalignment due to inconsistent spatiotemporal benchmarks, difficulty in adapting to data quality fluctuations due to fixed fusion strategies, and lack of traceability in the decision-making process. As a result, the fusion accuracy of multi-source heterogeneous data from space, air, and ground is significantly lost. Therefore, this invention proposes an intelligent fusion method for multi-source heterogeneous data from space, air, and ground to solve the problems existing in the prior art. Summary of the Invention

[0005] To address the aforementioned problems, the present invention aims to propose an intelligent fusion method for multi-source heterogeneous data from air, space, and ground, which solves the problems of large noise interference, inconsistent spatiotemporal benchmarks, rigid strategies, and difficulty in tracing decisions in existing multi-source heterogeneous data fusion methods.

[0006] To achieve the objectives of this invention, the invention is implemented through the following technical solution: a method for intelligent fusion of heterogeneous data from multiple sources in air, space, and ground, comprising the following steps: Step 1: Perform cross-modal noise filtering and spatiotemporal benchmark unification on the collected air-space-ground multi-source data to generate a standardized spatiotemporal data stream; Step 2: Based on standardized spatiotemporal data streams, construct a dynamic knowledge graph of multi-source data from air, space, and ground, and eliminate semantic conflicts through cross-modal alignment to generate a semantically consistent multimodal feature set; Step 3: Based on the semantic relevance and data quality evaluation results, dynamically select data-level, feature-level, or decision-level fusion strategies to generate preliminary data fusion results; Step 4: Stream the preliminary data fusion results to achieve incremental optimization, combine the causal reasoning model to generate interpretable decision paths, achieve causal tracing, and output the final fused data and tracing report.

[0007] A further improvement is that, in step one, the specific steps for cross-modal noise filtering and spatiotemporal benchmark unification are as follows: a band-adaptive filtering algorithm is used for satellite remote sensing data and aerial imagery, and noise correction is performed based on the atmospheric optical thickness attenuation coefficient and historical data regression results; a sliding window outlier detection is used for ground sensor data to remove outliers exceeding the preset standard deviation range; and satellite, aerial, and ground data from different coordinate systems are aligned to a unified spatiotemporal grid using spatiotemporal kriging interpolation.

[0008] A further improvement is that, in step two, the specific steps for constructing the dynamic knowledge graph are as follows: define the land cover category attributes of satellite remote sensing data pixel blocks, the object detection box boundaries of aerial images, and the spatiotemporal measurement values ​​of ground sensors; use a spatiotemporal graph convolutional network to associate satellite remote sensing data pixel blocks with ground sensor nodes according to spatial proximity and temporal synchronization; when semantic conflicts are detected, high-confidence data sources are used for verification first; if the conflict persists, a manual review process is triggered.

[0009] A further improvement is that, in step three, the dynamic selection rule for the fusion strategy is: to construct a multi-dimensional quality assessment model and calculate the spatial coverage S of the satellite remote sensing data. cover Aerial image clarity R res The time continuity of ground sensors C cont If S cover >0.9 and R res >200dpi, enable data-level fusion, if C cont If the value is less than 0.6 or there are sensor anomalies, switch to feature-level fusion. If serious contradictions are detected between cross-source data, trigger decision-level fusion and mark the conflict area.

[0010] A further improvement lies in: the spatial coverage S of the satellite remote sensing data cover =Number of effective pixels / Total pixels, the temporal continuity C of the ground sensor cont =1 - number of missing time periods / total number of time periods, the clarity R of the aerial image res =(pixel size × flight altitude) / camera focal length.

[0011] A further improvement lies in the following: In step four, a streaming processing engine is used to incrementally fuse the real-time incoming satellite, aerial, and ground data. The specific steps are as follows: A real-time data stream is received every 10 seconds, and the fusion result is updated using a Kalman filter algorithm. The updated results are subjected to conflict detection. If the difference from historical data exceeds a threshold, the fusion strategy in step three is re-evaluated.

[0012] A further improvement is that the implementation of the causal attribution includes: constructing a causal graph among multi-source data variables based on the causal discovery algorithm, and identifying the key influencing factors of the attribution decision result.

[0013] A further improvement is that the optimization method of the causal discovery algorithm includes: introducing domain knowledge constraints, limiting the causal relationship search space, quantifying the uncertainty of the causal edge weights, and eliminating weak causal relationships with a confidence level lower than 0.7.

[0014] The beneficial effects of this invention are as follows: This invention uses band adaptive filtering (satellite / airborne) and sliding window outlier detection (ground sensor), combined with spatiotemporal kriging interpolation to unify grid coordinates, thereby reducing the alignment error of multi-source data and solving the fusion distortion problem caused by inconsistent references in traditional methods; A dynamic knowledge graph semantic disambiguation is constructed based on a spatiotemporal convolutional network. Through a dual-channel mechanism of high-confidence verification and manual review, most cross-source semantic conflicts are eliminated, and real-time association and updating of multimodal features are achieved. Furthermore, by establishing a quantitative indicator system for coverage, clarity, and continuity, and dynamically triggering data-level, feature-level, and decision-level fusion, it can ensure fusion accuracy in different environments and is more efficient than traditional fixed strategies. Based on the streaming causal tracing engine, the decision path is traced through incremental updates using Kalman filtering and combined with causal discovery algorithms, which improves the causal interpretability of scenarios such as agricultural disaster early warning and reduces the response latency from hours to seconds. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the intelligent fusion method for multi-source heterogeneous data from air, space, and ground, as described in this invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Multi-source heterogeneous data from space, air, and ground refers to data collections of varying formats and types originating from multiple sources, including space (such as satellite remote sensing), the sky (such as aerial imagery), and the ground (such as sensor networks). Its heterogeneity is reflected in the diversity of data structures, including structured, semi-structured, and unstructured forms. This multi-source and heterogeneous nature of the data presents challenges for integration, processing, and analysis, necessitating the use of advanced technologies such as data fusion and machine learning for effective management and application. It plays a crucial role in areas such as smart cities, environmental monitoring, and disaster emergency response, providing support for scientific decision-making by integrating multi-dimensional information and improving resource utilization efficiency and responsiveness.

[0018] Multi-source heterogeneous data fusion refers to the process of integrating data from different sources with different formats and characteristics through key technologies such as cleaning, alignment, correlation, and integration, transforming it into a unified, consistent, and analyzable representation. Its core objective is to overcome the obstacles posed by data heterogeneity, deeply mine complementary and correlated information among multi-source data, eliminate redundancy and contradictions, and generate more comprehensive, accurate, and reliable high-value information or decision-making knowledge to serve applications such as situational awareness, intelligent decision-making, and predictive early warning.

[0019] Existing technologies for multi-source heterogeneous data fusion in space, air, and ground have the following core shortcomings: Severe noise interference: Traditional methods lack a collaborative filtering mechanism for atmospheric scattering noise in satellite remote sensing data, motion blur in aerial images, and random impulse interference from ground sensors. Single-mode denoising leads to the superposition of residual noise from cross-source data. Spatiotemporal reference fragmentation: The spatiotemporal alignment of satellite (WGS84 coordinate system), aerial (local coordinate system), and ground sensor (independent timestamp) data relies on simple affine transformation, which does not solve the problems of elevation differences and time drift, resulting in geometric misalignment and time asynchrony error of ≥2 pixels in the fused image; Rigid fusion strategies: Using fixed rules (such as global weighted average) or a single fusion level (only data level or feature level) cannot dynamically respond to fluctuations in data quality (such as forcing data-level fusion even when cloud coverage causes satellite data to fail), resulting in a sharp drop in fusion accuracy in abnormal scenarios. Decision-making black box: The integration process lacks a traceability mechanism. Key decisions (such as disaster early warning) rely on the output of end-to-end neural networks, making it impossible to locate the core data sources and causal chains that affect the results. As a result, 95% of abnormal decisions in actual applications cannot be effectively attributed. These shortcomings collectively result in insufficient reliability of existing technologies in complex scenarios such as multi-cloud regions and sensor failures, severely restricting critical applications such as emergency response.

[0020] See Figure 1This embodiment provides an intelligent fusion method for heterogeneous data from multiple sources in space, air, and ground, including the following steps: Step 1: Multimodal data cleaning and spatiotemporal alignment This method collects multi-source data from air, space, and ground, including satellite remote sensing data, aerial imagery, and ground sensor data. It performs cross-modal noise filtering and spatiotemporal benchmark unification on the collected multi-source data to generate a standardized spatiotemporal data stream. In this embodiment, considering the characteristics of different data sources, it adopts band adaptive filtering algorithms and sliding window outlier detection techniques to filter noise and unify spatiotemporal benchmarks. This preprocessing method for multi-modal data improves the quality and consistency of the data. In this embodiment, the specific steps for cross-modal noise filtering and spatiotemporal reference unification are as follows: A band-adaptive filtering algorithm is used to correct noise from satellite remote sensing data and aerial imagery, based on atmospheric optical thickness attenuation coefficients and regression results from historical data. The specific steps are as follows: Using MODIS AOT products (1km resolution) as prior data, atmospheric correction parameters were calculated by inputting them into the radiative transfer model (6S model).

[0021] For each band of the multispectral image (Sentinel-2), the local noise variance is calculated band by band using a sliding window (3×3 pixels), and the filter kernel weights are optimized by ridge regression. A sliding window outlier detection method is used to detect outliers in ground sensor data and remove outliers that exceed the preset standard deviation range. Using spatiotemporal kriging interpolation, satellite, aerial, and ground data from different coordinate systems are aligned to a unified spatiotemporal grid. Step 2: Dynamic Knowledge Graph Construction and Semantic Association Based on the standardized spatiotemporal data stream generated in step one, a dynamic knowledge graph of multi-source data from air, space, and ground is constructed. Semantic conflicts are eliminated through cross-modal alignment, and a semantically consistent multimodal feature set is generated. This embodiment realizes the semantic association of multi-source data by constructing a dynamic knowledge graph and using a spatiotemporal graph convolutional network for cross-modal alignment. This method, which combines spatiotemporal information and graph convolutional networks, can effectively handle semantic conflicts of multi-source heterogeneous data and improve the accuracy of data fusion. In this embodiment, the specific steps for constructing a dynamic knowledge graph are as follows: Define the land cover category attributes of satellite remote sensing data pixel blocks, the object detection box boundaries of aerial imagery, and the spatiotemporal measurement values ​​of ground sensors; Using a spatiotemporal graph convolutional network (ST-GCN), satellite remote sensing data pixel blocks are associated with ground sensor nodes based on spatial proximity and temporal synchronization; When a semantic conflict is detected, a high-confidence data source is used for verification first. If the conflict persists, a manual review process is triggered. Step 3: Quality-Driven Adaptive Fusion Decision Based on the semantic relevance of the knowledge graph and the data quality evaluation results in step two, a data-level, feature-level, or decision-level fusion strategy is dynamically selected to generate preliminary data fusion results. In this embodiment, the fusion strategy is dynamically selected based on the data quality evaluation results, which reflects the adaptability of the method. This data quality-based fusion decision can ensure that the most suitable fusion method is selected under different circumstances, thereby improving the reliability and practicality of the fusion results. In this embodiment, a multi-dimensional quality assessment model is constructed to calculate the spatial coverage (Scover) of satellite remote sensing data, the sharpness (Rres) of aerial imagery, and the temporal continuity (Ccont) of ground sensors, wherein: Spatial coverage of satellite remote sensing data: Scover = number of effective pixels / total number of pixels; The temporal continuity of the ground sensor, Ccont, is calculated as: 1 - number of missing time periods / total number of time periods. Aerial image sharpness Rres = (pixel size × flight altitude) / camera focal length; The dynamic selection rules for the fusion strategy are as follows: If Scover > 0.9 and Rres > 200 dpi, enable data-level fusion. The specific steps are as follows: stitch together satellite multispectral imagery, aerial high-resolution imagery, and ground sensor data according to spatial grid alignment; for grid cells without data coverage, use a generative adversarial network (GAN) to complete them. If Ccont < 0.6 or sensor outliers exist, switch to feature-level fusion. The specific steps are as follows: extract texture features from satellite data, SIFT features from aerial imagery, and temporal features from ground sensors; calculate feature weights through a cross-modal attention mechanism; and input the weighted fusion into an LSTM network to generate a spatiotemporal feature vector. If a serious contradiction is detected between cross-source data (e.g., satellite shows no clouds but ground light intensity <100 lux), decision-level fusion is triggered and conflict areas are marked. The specific steps are as follows: LightGBM classifiers are trained for satellite, airborne, and ground subsystems respectively; the decision results are merged using Dempster-Shafer evidence theory. Step 4: Incremental Optimization and Explainable Decision Output The preliminary data fusion results from step three are incrementally fused using a streaming engine (Apache Flink) to achieve incremental optimization, ensure data timeliness, and generate interpretable decision paths by combining a causal inference model. This enables causal tracing, tracking decision logic, enhancing interpretability, and outputting the final fused data and tracing report. This embodiment uses a streaming engine for incremental fusion and combines it with a causal inference model for causal tracing, enhancing the interpretability of decisions. This method of combining incremental optimization with causal inference helps users understand the reasons behind decisions and improves the transparency of decisions. In this embodiment, the implementation of incremental fusion includes: A real-time data stream is received every 10 seconds, and the fusion result is updated using a Kalman filter algorithm. The updated results are subjected to conflict detection. If the difference from historical data exceeds the threshold, the fusion strategy in step three is re-evaluated. In this embodiment, the implementation of causal attribution includes: Based on the causal discovery algorithm (PC algorithm), a causal graph is constructed among variables from multiple data sources to trace the key influencing factors of decision-making results. Optimization methods for causal discovery algorithms include: Domain knowledge constraints are introduced to limit the causal relationship search space (e.g., "wind speed affects temperature, but temperature does not affect wind speed"). Uncertainty quantification is performed on the weights of causal edges to eliminate weak causal relationships with confidence levels below 0.7.

[0022] This invention achieves intelligent fusion of multi-source heterogeneous data through a four-stage pipeline design (cleaning → mapping → fusion → optimization).

[0023] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent fusion of heterogeneous data from multiple sources (space, air, and ground), characterized in that, Includes the following steps: Step 1: Perform cross-modal noise filtering and spatiotemporal benchmark unification on the collected air-space-ground multi-source data to generate a standardized spatiotemporal data stream; Step 2: Based on standardized spatiotemporal data streams, construct a dynamic knowledge graph of multi-source data from air, space, and ground, and eliminate semantic conflicts through cross-modal alignment to generate a semantically consistent multimodal feature set; Step 3: Based on the semantic relevance and data quality evaluation results, dynamically select data-level, feature-level, or decision-level fusion strategies to generate preliminary data fusion results; Step 4: Stream the preliminary data fusion results to achieve incremental optimization, combine the causal reasoning model to generate interpretable decision paths, achieve causal tracing, and output the final fused data and tracing report.

2. The intelligent fusion method for multi-source heterogeneous data from air, space, and ground as described in claim 1, characterized in that, In step one, the specific steps for cross-modal noise filtering and spatiotemporal benchmark unification are as follows: a band-adaptive filtering algorithm is used for satellite remote sensing data and aerial imagery, and noise correction is performed based on the atmospheric optical thickness attenuation coefficient and historical data regression results; a sliding window outlier detection is used for ground sensor data to remove outliers that exceed the preset standard deviation range; and satellite, aerial, and ground data in different coordinate systems are aligned to a unified spatiotemporal grid using spatiotemporal kriging interpolation.

3. The intelligent fusion method for multi-source heterogeneous data from air, space, and ground as described in claim 1, characterized in that, In step two, the specific steps for constructing the dynamic knowledge graph are as follows: define the land cover category attributes of satellite remote sensing data pixel blocks, the object detection box boundaries of aerial images, and the spatiotemporal measurement values ​​of ground sensors; use a spatiotemporal graph convolutional network to associate satellite remote sensing data pixel blocks with ground sensor nodes according to spatial proximity and temporal synchronization; when semantic conflicts are detected, high-confidence data sources are used for verification first; if the conflict persists, a manual review process is triggered.

4. The intelligent fusion method for multi-source heterogeneous data from air, space, and ground as described in claim 1, characterized in that, In step three, the dynamic selection rule for the fusion strategy is as follows: construct a multi-dimensional quality evaluation model and calculate the spatial coverage S of the satellite remote sensing data. cover Aerial image clarity R res The time continuity of ground sensors C cont If S cover >0.9 and R res >200dpi, enable data-level fusion, if C cont If the value is less than 0.6 or there are sensor anomalies, switch to feature-level fusion. If serious contradictions are detected between cross-source data, trigger decision-level fusion and mark the conflict area.

5. The intelligent fusion method for multi-source heterogeneous data from air, space, and ground as described in claim 4, characterized in that: The spatial coverage S of the satellite remote sensing data cover =Number of effective pixels / Total pixels, the temporal continuity C of the ground sensor cont =1 - number of missing time periods / total number of time periods, the clarity R of the aerial image res =(pixel size × flight altitude) / camera focal length.

6. The intelligent fusion method for multi-source heterogeneous data from air, space, and ground as described in claim 1, characterized in that: In step four, a streaming processing engine is used to incrementally fuse the real-time incoming satellite, aerial, and ground data. The specific steps are as follows: A real-time data stream is received every 10 seconds, and the fusion result is updated using a Kalman filter algorithm. The updated results are subjected to conflict detection. If the difference from historical data exceeds a threshold, the fusion strategy in step three is re-evaluated.

7. The intelligent fusion method for multi-source heterogeneous data from air, space, and ground as described in claim 1, characterized in that, The implementation of the causal attribution includes: constructing a causal graph among multi-source data variables based on the causal discovery algorithm, and identifying the key influencing factors of the attribution decision results.

8. The intelligent fusion method for multi-source heterogeneous data from air, space, and ground as described in claim 7, characterized in that, The optimization method of the causal discovery algorithm includes: introducing domain knowledge constraints, limiting the causal relationship search space, quantifying the uncertainty of causal edge weights, and eliminating weak causal relationships with a confidence level lower than 0.7.

Images