A data extraction system based on multi-sensor time alignment

Abstract

The application discloses a kind of data extraction systems based on multi-sensor time alignment, including external information source layer, perception layer, central logic module layer and application effect display layer;Wherein, external information source layer is used to access multiple data sources;Perception layer is used to obtain the original data from multiple data sources from external information source layer, and the original data includes: observation target movement position information, observation target movement image frame sequence and management system fusion information;Central logic module layer is used to update original data to generate alignment data after time alignment to original data;Application effect display layer obtains alignment data from central logic module, and carries out data extraction display.According to the above technical scheme, time calibration and target characteristic matching analysis across platforms can be completed without external timing source, and high-precision consistency between the movement position information of multiple sensor records target and image frame is realized.

Description

Technical Field

[0001] This invention relates to the field of data integration, and more specifically, to a data extraction system based on multi-sensor time alignment. Background Technology

[0002] In modern information field situational awareness systems, different types of sensors use independent time bases and timing mechanisms. For example, sensor A records events using pulse counting or a local clock, optical sensor B calculates the timestamp for each frame by adding the frame rate to the initial time, and the management system generates status reports based on its own scheduling cycle. This results in significant and non-linear time discrepancies between data collected from different sources. This difference affects the effective integration of cross-platform data consistency analysis and decision support systems. Currently, solutions typically rely on external time synchronization sources or re-experimentation to obtain synchronization signals, but these methods are costly, inefficient, and unsuitable for processing large amounts of experimental data from existing historical non-uniform time systems.

[0003] Therefore, a technical solution is needed to perform time alignment and data extraction between multiple sensing ports in real-time or offline environments when there are difficulties in information synchronization between different types of sensor devices due to differences in time counting rules, without the need for an external time source. Summary of the Invention

[0004] To achieve the above objectives, this application provides a data extraction system based on multi-sensor time alignment, including an external information source layer, a perception layer, a central logic module layer, and an application effect display layer; The external information source layer is used to access multiple data sources, including electromagnetic wave sensors, optical sensors, and management systems. The perception layer is used to obtain raw data from multiple data sources from the external information source layer. The raw data includes: the motion position information of the observed target, the image frame sequence of the observed target motion, and the fusion information of the management system. The central logic module layer is used to align the original data in time and then update the original data to generate aligned data. The application effect display layer obtains alignment data from the central logic module and performs data extraction and display.

[0005] The central logic module layer includes: an anchor point identification unit, an offset modeling component, and an interpolation and resampling module; The anchor point identification unit is used to determine key common event nodes from the raw data as synchronization reference points, and constructs a sample set based on the basic information corresponding to the synchronization reference points across multiple sensors, represented as follows: ,in, For time-series samples from electromagnetic wave sensors, These are time-series samples from an optical sensor. These are time-series samples from the management system; The offset modeling component is used to time-align the raw data based on a synchronization reference point; The interpolation and resampling module is used to process the raw data that is missing data for a certain period and update it to aligned data.

[0006] Among them, key public event nodes refer to key physical moments identified from multiple joint records of electromagnetic wave sensors, optical sensors, and management systems, including: the time when the target was attacked, the time when the target was first detected, specific trajectory change points, visual feature change points, and general marker event time points in the system log.

[0007] Among them, the offset modeling component is based on the sample set relative to the same real physical moment. Based on the delay relationship, an offset function model is constructed, which supports both linear and nonlinear models; Furthermore, when the offset function model is a linear model, it is expressed as: Where j represents different data source types in the sample set, and a j b j Let be a constant for the offset function, and t be a time variable. This is the time offset after time alignment; When the offset function model is a nonlinear model, it is represented as: Where j represents different data source types in the sample set, ... Let be a constant for the offset function, and t be a time variable. This is the interpolation time.

[0008] The nonlinear model is used when the time point error of different data sources exceeds a specified threshold.

[0009] Furthermore, when the interpolation and resampling module processes the raw data that lacks data for a certain period, it infers the possible trajectory using optical image frames and motion models, and restores the continuity using cubic spline interpolation; it also resamples the input signal to a consistent frequency according to the synchronization reference point.

[0010] The central logic module layer also includes a data query interface to provide alignment data to the application effect display layer.

[0011] The application effect display layer responds to user needs, obtains the target time range, retrieves the alignment data matching the target time range from the central logic module layer to form a complete target behavior description trajectory, and displays it on the visualization interface.

[0012] When the application effect display layer displays the visual interface, it also supports the visual comparison function; the visual comparison function refers to: displaying the flight position trajectory of the target recorded by each sensor on a three-dimensional map, and judging whether the time of the observed target movement from each data source is consistent.

[0013] This invention identifies key time points related to the observed target, recorded by various data sources including multiple sensor devices and management systems. It constructs an offset model between the local time of each data source and a unified standard time point, and uses interpolation algorithms to fill in missing data. Finally, it visualizes the data. This invention can perform cross-platform time calibration and target characteristic matching analysis without an external time source, achieving high-precision consistency between the motion position information of the target recorded by multiple sensors and image frames. It is suitable for processing historical experimental data with various non-uniform timing mechanisms and can support collaborative work between multiple sensors under complex motion conditions. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of a data extraction system based on multi-sensor time alignment provided in an embodiment of the present invention. Detailed Implementation

[0015] This invention proposes a data extraction system for multi-sensor time alignment. Its core objective is to achieve high-precision time alignment among three data sources: electromagnetic wave sensors, optical sensors, and an operational management system (OMS), and to perform target characteristic matching analysis based on this alignment. In this invention, key common events observable by all available sensor channels are used as "anchor points" to establish a mapping model between the local time of each subsystem and a unified standard time node. Interpolation algorithms are then used to fill in missing segments, enabling cross-platform time alignment and target behavior pattern extraction without additional hardware support. This invention is applicable to processing historical data from heterogeneous sensors in the absence of a unified time synchronization, including time-series target motion data recorded by one sensor, video image frame sequences and their status messages recorded by another type of sensor, and target fusion information generated by the management system.

[0016] The specific implementation of the present invention will now be described in detail with reference to the accompanying drawings.

[0017] The structure of the data extraction system for multi-sensor time alignment provided by this invention is as follows: Figure 1 As shown, it consists of the P100 external information source layer, the P110 perception layer, the P120 central logic module layer, and the P130 application effect display layer. The P100 external information source layer is used to access multiple data sources, including electromagnetic wave sensors, optical sensors, and management systems. Typically, electromagnetic wave sensors provide location information data of the observed target recorded at a higher frequency, optical sensors provide a sequence of image frames with timestamps, and the management system provides the status information of the observed target as well as other fused information.

[0018] The P110 perception layer is used to obtain raw data from multiple data sources from the external information source layer. The raw data includes: the observed target motion position information, the image frame sequence of the observed target motion, and the management system fusion information.

[0019] The P120 central logic module layer is used to align the original data in time and then update the original data to generate aligned data. The central logic module layer includes: anchor point identification unit, offset modeling component, interpolation and resampling module, and data query interface; 1) The anchor point identification unit is used to determine key common event nodes as synchronization benchmarks from raw data from multiple data sources, and constructs a sample set based on the fields of the local timestamp and target identifier (target_id) corresponding to the synchronization benchmarks in multiple data sources, represented as follows: ,in, For time-series samples from electromagnetic wave sensors, These are time-series samples from an optical sensor. These are time-series samples from the management system; Among them, key public event nodes refer to key physical moments identified from multiple common records in the observation target motion position information, the image frame sequence of the observed target motion, and the fused information of the management system, including: the time when the target was attacked, the time when the target was first discovered, specific trajectory change points or visual feature change points, and general marker event time points in the system log.

[0020] 2) The offset modeling component is used to perform time alignment of the original data based on the synchronization reference point; The offset modeling component is based on the sample set relative to the same real physical moment. Based on the delay relationship, an offset function model is constructed, which supports both linear and nonlinear models; When the offset function model is a linear model, it is expressed as: Where j represents different data source types in the sample set, and a j b j Let be a constant for the offset function, and t be a time variable. This is the time offset after time alignment; When the time point error between different data sources exceeds a specified threshold, and the offset function model is a non-linear model, it is expressed as follows: Where j represents different data source types in the sample set, ... Let be a constant for the offset function, and t be a time variable. This is the interpolation time.

[0021] 3) The interpolation and resampling module is used to process the raw data that is missing data in a certain period of time, generate approximate estimation results to maintain spatiotemporal continuity, and resample all input signals to a consistent frequency according to the synchronization reference point to generate aligned data.

[0022] In the application scenario of this invention, the electromagnetic wave sensor mainly records the trajectory data of the target flight, which reflects the target's motion position information. The target's flight trajectory is usually continuous. After acquiring the first and last segments of the trajectory, an interpolation algorithm can be used to interpolate and supplement the target's motion position information that reflects the middle segment of the trajectory. For example, if the recorded data shows the position at 1 second, 1.7 seconds, 2.3 seconds, 2.8 seconds, and 3.4 seconds, a cubic spline interpolation algorithm can be used to calculate the position at 1.5 seconds, 2 seconds, 2.5 seconds, and 3 seconds. Through the above processing, the time interval of the data is kept at 0.5 seconds per beat. Based on this, it is convenient to perform time alignment of data from multiple data sources in subsequent processing.

[0023] The interpolation and resampling module is used when processing raw data lacking data for a certain period. It infers possible trajectories using optical image frames and motion models, and restores continuity using cubic spline interpolation. The input signal is resampled to a consistent frequency according to a synchronization reference point, and the resampled data is updated to aligned data for storage and management in a storage device. Cubic spline interpolation improves the quality and stability of continuous trajectory estimation under dynamic conditions, effectively reducing the impact of abrupt errors in high-speed changes and improving the accuracy of the final analysis conclusions.

[0024] 4) Data query interface, providing alignment data to the application effect display layer.

[0025] The P130 application effect display layer obtains alignment data from the central logic module and performs data extraction and display.

[0026] Users can specify display requirements by specifying the target time range and target identifier, such as specifying a total of 10 seconds before and after the target is attacked.

[0027] The application effect demonstration layer responds to user needs, quickly locates relevant records by combining the database index structure, and provides access services. Specifically, it obtains the target time range, retrieves aligned data matching the target time range from the central logic module layer to form a complete target behavior description trajectory, and simultaneously displays the consistency between the target motion trajectory and the state of the optical image frame in a graphical interface, achieving a visual interface display.

[0028] When the visualization interface is displayed, a visualization comparison function can also be realized, that is, the flight position trajectory (i.e., track) of the target recorded by each sensor is displayed on a three-dimensional map. The target trajectory recorded by each sensor can be observed from the three-dimensional map, and it can be determined whether the time of the observed target movement from each data source is consistent. The position error of the sensor detecting the target can be intuitively and conveniently compared and determined.

[0029] The data extraction system and data source of this invention constitute a complete cross-platform data analysis system, and can be used in extended fields such as evaluating the impact effect on moving targets.

[0030] This invention identifies key time points (such as the attack hit time or the first target detection time) of observed targets recorded by various data sources, including multiple sensor devices and management systems. It constructs an offset model between the local time of each data source and a unified standard time point, and uses interpolation algorithms to fill in missing data. Finally, it visualizes the data. This invention can perform cross-platform time calibration and target characteristic matching analysis without an external time source, achieving high-precision consistency between the motion position information of targets recorded by multiple sensors and image frames. It is suitable for processing historical experimental data with various non-uniform timing mechanisms and can support collaborative work between multiple sensors under complex motion conditions.

[0031] The above-disclosed embodiments are merely a few specific examples of the present invention. However, the present invention is not limited thereto, and any variations that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A data extraction system based on multi-sensor time alignment, characterized in that, It includes an external information source layer, a perception layer, a central logic module layer, and an application effect display layer; The external information source layer is used to access multiple data sources, including electromagnetic wave sensors, optical sensors, and management systems. The perception layer is used to obtain raw data from multiple data sources from the external information source layer. The raw data includes: the observed target motion position information, the image frame sequence of the observed target motion, and the management system fusion information. The central logic module layer is used to perform time alignment on the original data and then update the original data to generate aligned data. The application effect display layer obtains alignment data from the central logic module and performs data extraction and display.

2. The data extraction system based on multi-sensor time alignment according to claim 1, characterized in that, The central logic module layer includes: an anchor point identification unit, an offset modeling component, and an interpolation and resampling module; The anchor point identification unit is used to determine key common event nodes from the raw data as synchronization reference points, and construct a sample set based on the basic information corresponding to the synchronization reference points across multiple sensors, represented as follows: ,in, For time-series samples from electromagnetic wave sensors, These are time-series samples from an optical sensor. These are time-series samples from the management system; The offset modeling component is used to perform time alignment of the original data based on the synchronization reference point; The interpolation and resampling module is used to process the raw data that is missing data for a certain period of time and update it into aligned data.

3. The data extraction system based on multi-sensor time alignment according to claim 2, characterized in that, The key common event nodes refer to the critical physical moments identified from multiple shared records of electromagnetic wave sensors, optical sensors, and the management system, including: the time when the target was attacked, the time when the target was first detected, specific trajectory change points, visual feature change points, and common marker event time points in the system log.

4. The data extraction system based on multi-sensor time alignment according to claim 2, characterized in that, The offset modeling component is based on the sample set relative to the same real physical moment. Based on the delay relationship, an offset function model is constructed, which supports both linear and nonlinear models; When the offset function model is a linear model, it is expressed as: Where j represents different data source types in the sample set, and a j b j Let be a constant for the offset function, and t be a time variable. This is the time offset after time alignment; When the offset function model is a nonlinear model, it is expressed as: Where j represents different data source types in the sample set, ... Let be a constant for the offset function, and t be a time variable. This is the interpolation time.

5. The data extraction system based on multi-sensor time alignment according to claim 4, characterized in that, The nonlinear model is used when the time point error of different data sources exceeds a specified threshold.

6. The data extraction system based on multi-sensor time alignment according to claim 2, characterized in that, When the interpolation and resampling module processes raw data that lacks data for a certain period, it infers the possible trajectory using optical image frames and motion models, and restores continuity using cubic spline interpolation; it then resamples the input signal to a consistent frequency according to the synchronization reference point.

7. The data extraction system based on multi-sensor time alignment according to claim 1, characterized in that, The central logic module layer also includes a data query interface to provide alignment data to the application effect display layer.

8. The data extraction system based on multi-sensor time alignment according to claim 1, characterized in that, The application effect display layer responds to user needs, obtains the target time range, retrieves the alignment data matching the target time range from the central logic module layer to form a complete target behavior description trajectory, and displays it in a visual interface.

9. The data extraction system based on multi-sensor time alignment according to claim 8, characterized in that, When the application effect display layer displays the visual interface, it also supports a visual comparison function; the visual comparison function refers to: displaying the flight position trajectory of the target recorded by each sensor on a three-dimensional map, and judging whether the time of the observed target movement from each data source is consistent.

Images