Automobile multi-source data time alignment and fusion method and system

Abstract

The invention discloses an automobile multi-source data time alignment and fusion method and system. The method comprises the steps of obtaining first data and second data with non-uniform time references, and performing standardization processing on time; in the second data, window data are extracted from each candidate position through sliding window search, time coverage is judged, the window data are resampled to a first data time point, and a similarity distance is calculated by adopting a dynamic time warping algorithm; selecting the position with the minimum distance to establish a time mapping relation; and performing data fusion to generate fused data covering the complete or partial time range of the second data. The system comprises a data acquisition and preprocessing module, a sliding window matching module based on dynamic time warping, a time alignment relationship establishment module, a data fusion module and a data output module. According to the method, the relation between the time type and the sampling frequency of the data is not limited, universality is achieved through self-adaptive resampling, and the problems that in the prior art, the requirement for data characteristics is harsh, and the application range is narrow are solved.

Description

Technical Field

[0001] This invention relates to the field of automotive data processing technology, specifically to a method and system for time alignment and fusion of multi-source automotive data, which is applicable to solving the technical problem that different data sources cannot be directly aligned due to inconsistent time bases and large differences in sampling frequencies. Background Technology

[0002] With the development of automotive intelligence and connectivity, modern vehicles are equipped with various data recording systems, such as Event Data Recorders (EDR), T-BOX (vehicle-to-everything) communication terminals, new energy vehicle dynamic monitoring terminals, Vehicle Travelling Data Recorders (TDR), and Data Storage Systems for Automated Driving (DSSAD). These systems record the vehicle's operating status from different perspectives, providing a rich data foundation for accident investigations, driving behavior analysis, and vehicle performance evaluation. However, due to differences in design objectives, working mechanisms, and data collection strategies among different data sources, multi-source data faces the following technical challenges in practical applications: 1. Inconsistent Time Base. Different data systems may use different time recording methods. For example, EDR typically records relative time with the trigger event as the zero point (e.g., from 5 seconds before the event to the moment the event occurs), while vehicle monitoring platform data records absolute timestamps (e.g., UTC time or local time). The lack of a unified time base makes direct alignment via timestamps impossible. Even if some systems nominally record absolute timestamps, differences in clock sources (GPS clock, system clock, real-time clock chip, etc.), clock synchronization errors, and time zone settings can lead to discrepancies of several seconds or even more in timestamps. Directly using timestamps for alignment can result in serious errors.

[0003] 2. Inconsistent Sampling Frequencies. The sampling frequencies of different data sources vary significantly. According to "Automotive Event Data Recording System" (GB 39732) and "Automotive Driving Recorder" (GB / T 19056), EDR and TDR systems record vehicle operation data at relatively low sampling frequencies (e.g., 1Hz, 2Hz), while assisted / automatic vehicle data recording systems (DSSAD) typically use higher sampling frequencies. For example, "Intelligent Connected Vehicle Automated Driving Data Recording System" (GB 44497) requires DSSAD to record vehicle speed at a frequency of 10Hz, and some assisted / automatic vehicle data recording systems use even higher sampling frequencies. This difference in sampling frequency results in different data point densities, making it impossible to establish a one-to-one mapping relationship.

[0004] 3. Data integrity varies. Due to factors such as triggering mechanisms, storage strategies, and transmission conditions, the data integrity recorded by different data sources varies. Some data sources may contain a large number of missing values, with some periods completely devoid of data records. Furthermore, data with low sampling frequencies may fail to capture key event characteristics (such as the timing of braking or steering actions), making time alignment impossible through event feature points.

[0005] 4. Significant differences in time span. For example, automotive EDR only records data for about 5 seconds before and after a collision event, while T-BOX and vehicle monitoring platforms may continuously record data for hours or even days. Finding the corresponding location in the long-term data from the short-term data is a typical local matching problem.

[0006] Existing technologies for matching and fusing different vehicle data mainly include: 1. Direct Alignment Based on Absolute Timestamps. This method assumes that the timestamps from different data sources are accurate and synchronized, and directly looks up the corresponding relationships based on the timestamps. Chinese patent document CN 119625986 B provides a method for analyzing and visualizing traffic accident data of autonomous vehicles. First, it collects raw data related to vehicle traffic accidents and standardizes and integrates the raw data. Second, it cleans, verifies, timestamps, and synchronizes the standardized and integrated data to obtain processed data. Based on the speed and path of the accident vehicle, it obtains the collision time point and analyzes the driving mode at the time of collision, the vehicle's steering before the collision, the vehicle's pedals before the collision, the vehicle's accessories before the collision, the vehicle's malfunctions before the collision, the traffic signals and road markings before the collision, and the vehicle's state after the collision to obtain key time points. Finally, it visualizes the analysis results based on the processed data and the key time points. This method uses resampling based on the minimum sampling frequency of different data and uses the absolute timestamp information of the data for alignment, but it cannot fuse vehicle data that does not include absolute timestamp information (such as EDR data).

[0007] Chinese patent document CN116415202B discloses a multi-source data fusion method. First, the measurement data to be fused is aligned with the timestamps of existing valid fusion targets. Second, a weighted distance is obtained by weighted summation of the Mahalanobis distance and the Euclidean distance of the position and velocity of the measurement data and the aligned valid fusion targets. Based on this weighted distance, an association threshold range and association probability between the measurement data and the aligned valid fusion targets are calculated. Then, based on the association threshold range and association probability, it is determined whether the measurement data is associated with any valid fusion target, generating a matching pair. For each matching pair, the filtering algorithm in the valid fusion targets is updated using the measurement data. Finally, a weighted calculation is performed on the Mahalanobis distance of the position vector and the Euclidean distance of the velocity vector between the fusion target and the sensor measurements, and the joint association probability between the fusion target and the sensor measurements is calculated based on the weighted distance. This method uses direct alignment based on absolute timestamps, requiring strict synchronization of clocks from all data sources, and does not consider clock deviation issues.

[0008] 2. Data Alignment Based on Hardware and Software Synchronization. Chinese Patent Document CN 109547693 A provides a multi-sensor data synchronization system and image data synchronization method. The system includes a host and sensors. The host is used to collect data input from multiple sensors and image acquisition devices. The multiple sensors include a first type of sensor, a second type of sensor, and a third type of sensor. The first type of sensor is connected to the vehicle's CAN bus, the second type of sensor is connected to a proprietary CAN bus, and the third type of sensor includes a signal terminal and a data terminal. The signal terminal is connected to the vehicle's CAN bus and is used to output frame signals. The data terminal is connected to the host via a data line and outputs data to the host. The vehicle's CAN bus and the proprietary CAN bus are connected to the host through the same data converter to synchronize the data input to the host. This method uses the same data converter to synchronize data from different sensors, which cannot meet the requirements for matching and fusing collected vehicle data containing different time bases.

[0009] 3. Data Alignment Based on Feature Events. Chinese patent document CN115980391B provides a method for testing acceleration sensors in an event data recording system, involving the alignment of collision acceleration data. This method first applies a shock wave to an impact test bench, then reads and analyzes the first acceleration signal recorded by the event data recording system and the first moment T1 of airbag deployment. It then obtains the second acceleration signal from a second acceleration sensor installed on the impact test bench and the second moment T2 of the current signal appearing in the airbag deployment circuit of the event data recording system. Based on the fact that airbag deployment and the current signal appearing in the deployment return circuit are the same feature event (i.e., T1 and T2 are at the same moment), the first acceleration signal and the second acceleration signal are aligned. However, this method relies on obvious feature events, and for stable driving conditions or low sampling frequency data, it is often difficult to identify reliable feature points.

[0010] 4. Cross-correlation-based sliding window matching. This method uses a cross-correlation function to measure the similarity between two signals and searches for the position with the highest correlation using a sliding window as the matching point. However, this method requires the two signals to have the same length and aligned sampling points. For cases with inconsistent sampling frequencies or different signal lengths, resampling or truncation is required, which may introduce additional errors. Furthermore, cross-correlation is sensitive to differences in signal amplitude, requiring normalization processing.

[0011] 5. Although the traditional Dynamic Time Warping (DTW) method can handle time warping, it usually assumes that the two sequences are of similar length. When directly applied to matching long and short sequences, the computational complexity is extremely high and the effect is poor.

[0012] 6. Time series matching based on machine learning. In recent years, some studies have attempted to use deep learning methods (such as recurrent neural networks and attention mechanisms) for time series matching. However, these methods require a large amount of labeled data for training, and the models are complex and slow inference, making them unsuitable for real-time or near-real-time applications.

[0013] In summary, existing methods for time alignment and fusion of multi-source automotive data generally suffer from limited applicability, insufficient robustness, and low accuracy in the face of practical problems such as inconsistent time bases, inconsistent sampling frequencies, and differences in data integrity. Furthermore, the lack of a systematic implementation plan makes it difficult to meet the engineering requirements of real-world applications.

[0014] Therefore, there is an urgent need for a widely applicable, robust, and highly accurate method for time alignment and fusion of multi-source data, as well as a modular, configurable, and easily expandable system implementation scheme to meet the application needs of accident / case investigation, driving behavior analysis, vehicle performance evaluation, and other applications. Summary of the Invention

[0015] This invention aims to solve three core problems in existing automotive multi-source data alignment technologies: First, the difficulty in alignment caused by inconsistent time bases, especially the inability to directly correspond relative time to absolute time; second, the problem of handling arbitrary sampling frequency relationships, as existing methods usually assume specific frequency relationships and lack universality; and third, the problem of locating short-time data in long-time data and the problem of the integrity of fusion results. The invention aims to achieve high-precision and robust multi-source data time alignment and fusion, and to provide a modular, configurable, and easily expandable system implementation scheme.

[0016] To achieve the above objectives, the technical solution adopted by the present invention is as follows: To address the aforementioned technical problems, this invention provides a method for time alignment and fusion of multi-source automotive data. The core concept of this method is to use short-term, high-frequency data with inconsistent time bases as a reference template for localization. By searching through a sliding window in long-term, low-frequency data, matching and localizing based on the similarity of parameter values ​​rather than timestamps is performed. A precise time mapping relationship is then established, ultimately generating fused data covering the entire time range. Simultaneously, this application also discloses a system for time alignment and fusion of multi-source automotive data.

[0017] The method of the present invention comprises four main steps performed sequentially, as follows: Step S1 is data acquisition and preprocessing, which is the foundation for subsequent processing.

[0018] This step acquires first and second data that record the same physical parameters but have different time bases. The first data contains the segment to be matched and the time span T1 is smaller than the time span T2 of the second data. The same parameters in the two data within the segment to be matched show obvious time variation characteristics.

[0019] Convert the time format of the two sets of data into a unified numerical time format so that the time can be used for numerical calculations.

[0020] Missing values ​​for the parameters used in the second dataset for matching are filled using interpolation methods to ensure that parameter estimates are available at any given time. The output of this step is standardized data with a consistent format and controllable quality, providing the necessary conditions for subsequent matching.

[0021] Step S2 is sliding window matching based on dynamic time warping, which is the core of the method.

[0022] This step determines the sliding window size based on the time span T1 of the segment to be matched, and then slides through the second data according to the sampling interval or a set step size.

[0023] For each candidate position i, the following steps are performed sequentially: extract window data; determine whether the window time range completely covers T1; if not, skip the position. Based on the time interval (or sampling frequency) of the first data segment to be matched, resample the window data to the same time point as the segment to be matched. The resampling adaptively selects interpolation upsampling, downsampling, or time point alignment method according to the sampling frequency relationship.

[0024] For the second data within the window, the dynamic time warping algorithm is used to calculate the similarity distance between the resampled sequence and the sequence of segments to be matched for the same parameter values, and the position i and its corresponding similarity distance D are recorded. dtw (i). The distance calculation formula is: ; Where: X = {x1, x2, ..., x} m} represents the parameter sequence of the first data, with a length of m, Y={y1,y2,…,y n} represents the parameter sequence of the second data after resampling, with a length of n, W={w1,w2,…,w K} represents the alignment path, which is a sequence of index pairs, where w k =(i k ,j k ) represents the i-th digit of X. k The element and the j-th element of Y k The elements are aligned, K represents the length of the alignment path, and d(∙,∙) is the distance function between points, usually using Euclidean distance.

[0025] After traversal, select the position i with the smallest similarity distance. best As the optimal matching position. This step achieves a global search through a sliding window, handles arbitrary sampling frequency relationships through adaptive resampling, and implements a robust similarity measure through a dynamic time warping algorithm. Its output is the sequence of optimal matching positions of the segment to be matched in the second data [i best, i best +1,…,i best +m].

[0026] Step S3 establishes the time alignment relationship, and establishes an accurate time mapping based on the matching results of step S2.

[0027] This step will determine the best matching position i in the second data. best The corresponding time t best As a baseline, for each time t in the first data segment to be matched 1,j (where j is the position index of the first data), calculate its time offset (t) relative to the start time of the segment. 1,j -t 1,0The reference time is added to the offset to determine the corresponding time in the second data. ; Find the closest actual sampling time to the corresponding time in the second set of data, and obtain the corresponding index k. j This establishes a one-to-one time mapping relationship between the segment to be matched and the second data.

[0028] The output of this step is a mapping table containing index correspondence, time correspondence, and time error, providing an accurate time reference for data fusion.

[0029] Step S4 is data fusion, which generates complete fused data based on the time mapping in step S3.

[0030] This step first compares the sampling frequencies of the first and second data within the matching window, and selects the time point of the data source with the higher sampling frequency as the time reference for the fused data.

[0031] Interpolate and upsample the data source with a low sampling frequency to align it with the time points of the high-frequency data source. Use linear interpolation, spline interpolation, or other interpolation methods to calculate estimates of the low-frequency data at high-frequency time points.

[0032] Identify identical parameters (parameters recorded simultaneously) and dissimilar parameters (parameters recorded only in one data set) in the first and second data sets.

[0033] Then, a fused data framework is created that covers the full (or partial) time range of the second data, where high-frequency time series are used within the matching window.

[0034] For each time k in the fused data, determine whether that time is within the matching window: If within a window, calculate the fusion value for parameters with the same name according to the fusion strategy (average fusion, weighted fusion, priority selection, or Kalman filter fusion), and fill in the corresponding data source value for parameters with different names; If not within the window, the values ​​of the second data are used for all parameters. The output of this step is a fused dataset containing the complete time series and the fusion results of all parameters.

[0035] The above four steps form a complete processing chain: the output of step S1 is the input of step S2, the output of step S2 is the input of step S3, and the output of step S3 is the input of step S4. There are strict dependencies between the steps, and they must be executed in sequence.

[0036] This invention also provides a system for time alignment and fusion of multi-source automotive data. This system corresponds to the method described above and includes a data acquisition and preprocessing module, a sliding window matching module based on dynamic time warping, a time alignment relationship establishment module, a data fusion module, and a data output module. Each module implements the corresponding step in the method and communicates through a standardized data interface to form a complete processing flow.

[0037] Step S1 of the data acquisition and preprocessing module includes a data input unit, a time standardization unit, and a missing value processing unit.

[0038] The data input unit is responsible for reading vehicle data from different data sources (local files, databases, network interfaces, etc.) and performing format verification.

[0039] The time standardization unit includes a time format recognition subunit and a time conversion subunit. The time format recognition subunit detects the type of time data (numerical, datetime object, timestamp string, etc.), and the time conversion subunit adopts the corresponding conversion strategy based on the recognition result to convert the time into a unified numerical time.

[0040] The missing value processing unit detects missing values ​​of the parameters used for matching in the second data and fills them using methods such as linear interpolation and spline interpolation.

[0041] The output interface of this module provides standardized time series and parameter sequences for use by subsequent modules.

[0042] Step S2 of the sliding window matching module corresponding method based on dynamic time warping includes a window parameter setting unit, a sliding window search unit, and an optimal position selection unit.

[0043] The window parameter setting unit sets the sliding window size according to the time span of the segment to be matched, determines the start and end positions of the traversal according to the effective time range of the second data, and calculates the total number of candidate positions.

[0044] The sliding window search unit is the core of this module, including a window positioning subunit, a data extraction subunit, a time coverage judgment subunit, a resampling subunit, a similarity distance calculation subunit, and a distance recording subunit. The resampling subunit includes a sampling frequency relationship judgment subunit and an adaptive interpolation subunit, which adaptively selects interpolation upsampling, downsampling, or time point alignment methods based on the sampling frequency relationship between the second data and the segment to be matched.

[0045] The similarity distance calculation subunit adopts the dynamic time warping algorithm and includes a distance matrix construction subunit, a cumulative distance calculation subunit, a path backtracking subunit, and a total distance output subunit.

[0046] Optionally, the sliding window search unit also includes a data sufficiency judgment unit, which is used to determine whether the number of data points in the window meets the preset minimum requirement. If not, the position is skipped to improve processing efficiency.

[0047] The optimal location selection unit chooses the location with the smallest similarity distance from the distance records as the best matching location. The output interface of this module provides the index of the best matching location, the minimum similarity distance value, and the distance records of all candidate locations.

[0048] Step S3 of the method corresponding to the time alignment relationship establishment module is to establish a time mapping relationship between the segment to be matched and the second data based on the best matching position.

[0049] This module uses the time corresponding to the best matching position as a benchmark, calculates the time offset for each time in the segment to be matched, and determines its corresponding time in the second data.

[0050] The output interface of this module provides a mapping table containing information such as the first data index, the first data time, the second data index, the second data time, the target time, and the time error.

[0051] Step S4 of the data fusion module method includes a parameter identification unit, a fusion framework construction unit, and a data filling unit.

[0052] The parameter identification unit identifies parameters with the same name and parameters with different names by comparing the column names of the two datasets.

[0053] The fusion framework building block creates a fused data framework that covers the second complete data time range and initializes all data columns.

[0054] The data filling unit determines whether each time point is within the matching window based on the time mapping table. Within the window, it calculates the fusion value for parameters with the same name according to the fusion strategy (average fusion, weighted fusion, priority selection, or Kalman filter fusion), fills the corresponding data source value for parameters with different names, and uses the value of the second data outside the window.

[0055] The module's output interface provides a complete fusion dataset and fusion statistics.

[0056] The data output module is responsible for outputting the processing results in a suitable format.

[0057] This module receives the fused dataset, the time mapping table, and the matching quality assessment information (such as minimum similarity distance, average time error, fusion ratio, etc.), and supports output in multiple formats such as CSV file, Excel file, database table, and JSON format, which is convenient for subsequent applications.

[0058] The system adopts a modular design, with each functional module communicating through standardized data interfaces. The inputs and outputs of each module follow a unified data structure specification, resulting in low coupling between modules and facilitating independent development, testing, and maintenance.

[0059] Compared with the prior art, the present invention has the following advantages: (1) High versatility. Through time standardization processing, this invention can automatically identify and convert various time formats (numerical, datetime objects, timestamp strings, etc.) without manual intervention, avoiding processing failures caused by inconsistent time formats in existing technologies. Through an adaptive resampling strategy, this invention can handle arbitrary sampling frequency relationships. Regardless of whether the first data frequency is higher than, equal to, or lower than the second data frequency, it can automatically select an appropriate resampling method, overcoming the limitations of assuming a specific frequency relationship in existing technologies. This versatility makes this invention applicable to various data source combinations such as EDR and monitoring platform data, T-BOX and driving recorder, ADAS data and vehicle operation log data.

[0060] (2) Good robustness. Global search is achieved through sliding window traversal, avoiding local optima and ensuring the finding of the true optimal matching position. Incomplete windows are excluded by time coverage judgment, avoiding incorrect matching caused by boundary effects. Similarity is calculated using a dynamic time warping algorithm, which can handle time distortion and local perturbations, and has good robustness to slight noise and uneven sampling. Compared with simple Euclidean distance or correlation coefficient, the matching accuracy is significantly improved. Experiments show that on real data containing noise, the matching accuracy of the method of this invention reaches over 95%, while the accuracy of the method based on correlation coefficient is only about 70%.

[0061] (3) Complete Fusion Results. Unlike existing technologies that only fuse data within a matching window, this invention generates a fusion dataset covering the entire time range of the second data set, including fusion results for all parameters with the same name and retained results for all parameters with different names. This completeness preserves the temporal context information of the data, providing a more comprehensive data foundation for applications such as accident analysis and driving behavior assessment. For example, in accident analysis, not only can the detailed state of the 5 seconds before the collision be seen (from EDR), but also the driving trajectory and vehicle status of the complete 40 minutes before and after the accident (from platform data), which helps to fully reconstruct the accident process.

[0062] (4) High processing efficiency. By using optional data sufficiency judgment, sparse windows are filtered in advance, avoiding meaningless interpolation and DTW calculations on invalid windows, thus reducing computational resource consumption. When there are many sparse regions in the second data, the processing speed can be improved by more than 30%. Through modular design and standardized interfaces, the system supports parallel processing, and can process multiple candidate window positions simultaneously on multi-core processors, further improving the processing speed.

[0063] (5) High scalability and adaptability. The modular system design makes each functional module relatively independent, which is convenient for maintenance, upgrades and functional expansion. It supports multiple fusion strategies (averaging, weighting, priority selection, Kalman filtering), and users can flexibly choose according to data characteristics and application requirements. The system is adaptable to different deployment environments and performance requirements. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention. Figure 2 This is a diagram illustrating a sliding window search. Figure 3 This is a block diagram of the overall structure of the system of the present invention; Figure 4 This is a data illustration for an example. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0067] Example 1: Alignment and Fusion of EDR Data and Vehicle Monitoring Platform Data (Method Example) In the analysis of a traffic accident, it is necessary to align and fuse the vehicle's EDR data with the data from the onboard monitoring platform to reconstruct the complete vehicle state before and after the accident. The EDR records detailed vehicle state information from 5 seconds before the collision to the moment of impact, but using relative time cannot determine its position on the actual timeline. The onboard monitoring platform continuously records long-term vehicle data, but the sampling frequency is low. Therefore, the method of this invention is needed to align and fuse the two.

[0068] Data preparation The characteristics of the first data (EDR data) are as follows: • Recording time: 5 seconds before the event to 0 seconds after the event; • Time format: relative time, floating-point number in seconds; • Sampling frequency: 2Hz, meaning sampling once every 0.5 seconds; • Number of data points: 11 points (time points are -5.0, -4.5, -4.0, -3.5, -3.0, -2.5, -2.0, -1.5, -1.0, -0.5, 0.0); • Recording parameters: Time (s), Speed ​​(km / h); The data is shown in Table 1 below.

[0069] Table 1: Data Content

[0070] The characteristics of the second type of data (vehicle monitoring platform data) are as follows: • Recording time: from 14:30:00 to 15:09:59, lasting 40 minutes; • Time format: Absolute time string "HH:MM:SS"; • Sampling frequency: 1Hz, sampled once per second; • Number of data points: 2400; • Record parameters: Time, Speed ​​(km / h), Latitude, Longitude; The key data segments are shown in Table 2 below (the incident occurred at 14:33:25, corresponding to index 205).

[0071] Table 2: Key Data Segments

[0072] According to the accident record, the collision occurred at 14:33:25. Therefore, the 0.0 second of EDR corresponds to 14:33:25 in the platform data, and the -5.0 second of EDR corresponds to 14:33:20 in the platform data.

[0073] Step S1: Data Acquisition and Preprocessing (1) Data acquisition: Data is acquired from the EDR device and the vehicle platform respectively and loaded into a data table format.

[0074] (2) Data verification: Check whether the two sets of data meet the basic conditions. Both the EDR and platform data record the vehicle speed (data item speed), a physical parameter. The EDR uses relative time, while the platform uses absolute time (the time bases are not unified). The time span T1 of the EDR is equal to 5 seconds, which is much less than the time span T2 of the platform data, which is equal to 2400 seconds (satisfying T1 < T2). The vehicle speed in the EDR changes from 95.2 km / h to 85.0 km / h (showing significant time-varying characteristics). All conditions are met.

[0075] (3) Time normalization processing: EDR data time: Detect the first sample -5.0, identify it as a floating-point type, and determine it as a numerical time. The unit is already seconds and no conversion is required. The normalized time remains: -5.0, -4.5, -4.0,..., 0.0.

[0076] Platform data time: Detect the first sample "14:30:00", identify it as a string type. Try to match the time string format and identify it as the "HH:MM:SS" format. Parse the first time "14:30:00" into a datetime object and use it as the reference time (corresponding to 0 seconds). Parse each time string into a datetime object and calculate the number of seconds relative to the reference time. For example, "14:30:00" corresponds to 0 seconds, "14:30:01" corresponds to 1 second, "14:33:20" corresponds to 200 seconds, and "15:09:59" corresponds to 2399 seconds. The normalized platform time is: 0, 1, 2,..., 2399.

[0077] (4) Missing value handling: Check the speed column of the platform data and no missing values are found, so no interpolation filling is required. If there are missing values, linear interpolation method will be used for filling.

[0078] Step S2: Sliding window matching based on dynamic time warping, see Figure 2 .

[0079] (1) Determine the sliding window size: According to the time span T1 of the segment to be matched (the entire EDR data) which is equal to 5 seconds, set the sliding window size to 5 seconds.

[0080] (2) Determine the traversal range: The time range of the platform data is from 0 to 2399 seconds. The maximum starting position of the window is 2399 minus 5, which is 2394 seconds, otherwise the end time of the window will exceed the data range. Therefore, the traversal range is from index 0 (time 0 seconds) to index 2394 (time 2394 seconds), a total of 2395 candidate positions.

[0081] (3) Sliding window search: Perform the following operations for each candidate position i (from 0 to 2394): Taking position i equal to 200 as an example (corresponding to time 200 seconds, i.e., 14:33:20): Window positioning: start_time equals 200 seconds, end_time equals 200 plus 5 equals 205 seconds. The window time range is [200, 205] seconds.

[0082] Data Extraction: Extract data points with times ranging from [200, 205] seconds from the platform data. Since the platform data sampling frequency is 1Hz, this range contains 6 data points with indices from 200 to 205, as shown in Table 3 below.

[0083] Table 3: Data Points

[0084] Time coverage judgment: The actual time span of the window is 205 minus 200 equals 5 seconds, which equals T1, satisfying the coverage condition (greater than or equal to 0.95 × 5 equals 4.75 seconds), and the judgment is passed.

[0085] Resampling: The offsets of the 11 time points (-5.0, -4.5, ..., 0.0) of the EDR data relative to their starting time -5.0 are (0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0). These time points correspond to positions 200 plus an offset in the windowed time system, i.e., (200.0, 200.5, 201.0, 201.5, 202.0, 202.5, 203.0, 203.5, 204.0, 204.5, 205.0). A linear interpolation function is constructed using 6 data points from the window to perform interpolation calculations on the 11 target time points. For example: target time 200.0 corresponds to a vehicle speed of 95.0 at window index 0; target time 200.5 needs to be interpolated between window index 0 (time 200, vehicle speed 95.0) and index 1 (time 201, vehicle speed 93.5), the result is 95.0 plus (93.5 minus 95.0) multiplied by (200.5 minus 200) divided by (201 minus 200) equals 94.25; target time 201.0 corresponds to a vehicle speed of 93.5 at window index 1; and so on. The resampled vehicle speed sequence is: [95.0, 94.25, 93.5, 92.75, 92.0, 91.1, 90.2, 89.0, 87.8, 86.65, 85.5].

[0086] See Figure 4 The resampling operation densifies the window data (6 points) with a sampling frequency of 1Hz to the target time point (11 points) with a sampling frequency of 2Hz, thus achieving upsampling.

[0087] Dynamic Time Warping (DTW) calculation: Construct an 11×11 distance matrix between the EDR speed sequence [95.2, 94.8, 94.5, 93.9, 93.2, 92.1, 90.8, 89.2, 87.5, 86.1, 85.0] and the resampled sequence [95.0, 94.25, 93.5, 92.75, 92.0, 91.1, 90.2, 89.0, 87.8, 86.65, 85.5]. Use dynamic programming to calculate the cumulative distance matrix, backtrack to find the optimal path, and calculate the total distance. In this example, because the two sequences have highly consistent trends, the DTW distance is small, and the calculated result is approximately 5.2.

[0088] Record distance: Record position 200 and its distance 5.2: distances

[200] = 5.2.

[0089] Perform the same operation on other candidate positions (e.g., i equals 100, 150, 300, etc.). Because the window data at these positions does not follow the same trend as the EDR data, the DTW distance will increase significantly. For example, position 100 corresponds to 100 seconds (14:31:40), at which point the vehicle speed trend differs from the EDR, and the DTW distance may exceed 50.

[0090] (4) Select the best matching position: After traversal, find the position with the smallest distance from the distances dictionary. In this example, the distance of position 200 is 5.2, which is the smallest among all candidate positions, therefore i best Equals 200, minD dtw(i) It equals 5.2, corresponding to time t. best It equals 200 seconds (i.e., 14:33:20).

[0091] See Figure 2 and Figure 4 The sliding window iterates through the platform data, and the DTW distance reaches its minimum at position 200, indicating that this position is most similar to the EDR data.

[0092] Step S3: Establishing Time Alignment Relationship (1) Determine the reference time: t best This equals 200 seconds, corresponding to 14:33:20 in the platform data. This moment corresponds to the -5.0 seconds of EDR in the platform data.

[0093] (2) Establish time mapping: For the 11 time points of EDR, establish mapping relationships with platform data respectively, as shown in Table 4 below.

[0094] Table 4: Mapping Relationships

[0095] For target times that are not integers in seconds (e.g., 200.5 seconds), the closest actual sampling time is found in the platform data. 200.5 is 0.5 seconds away from both 200 and 201, so we can choose 200 (take the nearest) or 201 (take the nearest). In this example, we choose the nearest, so the target time 200.5 corresponds to platform index 200, with a time error of 0.5 seconds.

[0096] (3) Mapping statistics: The average time error is (0.0+0.5+0.0+0.5+0.0+0.5+0.0+0.5+0.0+0.5+0.0) / 11=0.227 seconds. The maximum time error is 0.5 seconds. The error mainly comes from the sampling interval (1 second) of the platform data.

[0097] Verification: The -5.0 seconds of EDR were mapped to 14:33:20 on the platform, and the 0.0 seconds of EDR were mapped to 14:33:25 on the platform. This perfectly matches the known real situation (the collision occurred at 14:33:25), verifying the correctness of the alignment.

[0098] Step S4: Data Fusion (1) Parameter identification. Same-name parameter: Speed ​​(available in both EDR and platform data), EDR-specific parameter: none, platform data-specific parameters: Latitude, Longitude.

[0099] (2) Create a merged data framework. The number of rows in the merged data is equal to the number of rows in the platform data, which is 2400. The time column is copied from the time column of the platform data (keeping the original format "HH:MM:SS" or using the normalized number of seconds). The columns include: Time, Speed, Speed_Source (data source, optional), Latitude, and Longitude.

[0100] (3) Data population: Populate each row of the fused data (corresponding to the index k of the platform data, from 0 to 2399): For the case where k is within the matching window (k=[200, 201, 202, 203, 204, 205]), taking k=202 as an example: Within the window: k=202 appears in the mapping table (corresponding to EDR indices 2 and 4), confirming it's within the matching window. For the parameter Speed ​​with the same name: obtain the closest value from EDR. The target time 202.0 seconds corresponds to EDR index 2 (EDR time - 4.0 seconds), and the vehicle speed in EDR is 94.5 km / h. The platform data shows a vehicle speed of 92.0 km / h for k=202. Using an average fusion strategy, the fused value is (94.5 + 92.0) / 2 = 93.25 km / h. The Speed_Source item is marked as "fused". For platform-specific parameters Latitude and Longitude: retain the platform data values, which are 31.2323 and 121.5323 respectively.

[0101] If k is not within the matching window (e.g., k=100 or k=300): The condition is that k does not appear in the mapping table. Platform data values ​​are used for all parameters. Speed ​​directly uses the platform's vehicle speed value, with the data source marked as "Data2". Latitude and Longitude use platform data values.

[0102] (4) Generate a fused dataset: Examples of the fused data are shown in Table 5 below.

[0103] Table 5: Examples of merged data

[0104] Results Analysis: By comparing the speed change trends of EDR and platform data within the matching window (14:33:20 to 14:33:25), the two data were highly consistent: EDR decreased from 95.2 to 85.0 km / h, and platform data decreased from 95.0 to 85.5 km / h, with the same magnitude and trend, verifying the accuracy of the alignment. The fused data not only retained detailed vehicle speed information from EDR in the 5 seconds before the collision (2Hz sampling), but also fully preserved 40 minutes of vehicle speed records and GPS trajectories from the platform data. Accident investigators can use the fused data to analyze the detailed vehicle status before the collision (high-frequency data from EDR), while also understanding driving conditions and geographical locations over a longer period before and after the accident (from platform data), providing comprehensive evidence for real-time accident reconstruction and liability determination.

[0105] Example 2: System-based data processing flow See Figure 3 This embodiment demonstrates the workflow of the system of the present invention in a practical application.

[0106] The system is deployed on the server of the accident analysis center and is implemented using software. The system configuration parameters are as follows: • Input data source: Local CSV file • Target time unit: seconds (s) • Missing value interpolation method: linear interpolation • Sliding step size: 1 second (point-by-point traversal) • Minimum requirement for data sufficiency: 3 data points • Fusion Strategy: Average Fusion Output format: Excel file System workflow: (1) Start the data acquisition and preprocessing module The data input unit reads the EDR data file (edr_data.csv) and the platform data file (platform_data.csv) from the specified path and performs format validation. After successful validation, the data is passed to the time standardization unit.

[0107] The time format recognition subunit of the time standardization unit detects that the EDR time is a floating-point number and determines it as a numeric time; it detects that the platform time is a string type "HH:MM:SS" and determines it as a time string. The time conversion subunit directly uses the EDR time and parses the platform time into a datetime object before converting it to relative seconds.

[0108] The missing value processing unit checks the vehicle speed column of the platform data and does not detect any missing or invalid (NaN) values. It outputs the message "No missing value found, skip interpolation processing".

[0109] The output interface of this module provides standardized datasets, including EDR time series (11 points), EDR vehicle speed series, platform time series (2400 points), platform vehicle speed series, and platform latitude and longitude series.

[0110] (2) Startup of the sliding window matching module based on dynamic time warping Window parameter setting unit calculation: window size = 5 seconds, traversal start position = index 0, traversal end position = index 2394, total number of candidate positions = 2395.

[0111] The sliding window search unit begins traversing, and for each candidate position: the window positioning subunit determines the window time range, and the data extraction subunit extracts window data from the platform data; the data sufficiency judgment unit checks the number of window data points (approximately 6 points), which is greater than the minimum requirement of 3 points, and passes the judgment; the time coverage judgment subunit checks the actual span of the window (5 seconds), which is equal to the required span, and passes the judgment; the sampling frequency relationship judgment subunit of the resampling subunit determines that the platform sampling interval (1 second) is greater than the EDR sampling interval (0.5 seconds), and the adaptive interpolation subunit selects the interpolation upsampling method to interpolate the 6 points to 11 target time points; the similarity distance calculation subunit calls the dynamic time warping algorithm to calculate the DTW distance; and the distance recording subunit records the position index and distance value in the dictionary.

[0112] After the traversal is complete, the optimal position selection unit finds the minimum value from the distance records and outputs "Optimal matching position: index 200, DTW distance 5.2". The output interface of this module provides the optimal matching position index 200, the minimum distance 5.2, and the distance records of all candidate positions.

[0113] (3) Startup of the time alignment relationship establishment module This module receives the best matching position index 200 and determines the base time 200 seconds. For each of the 11 time points of the EDR, it calculates the time offset and determines the corresponding time in the platform data.

[0114] The module's output interface provides a mapping table (11 rows × 6 columns), containing the first data index, first data time, second data index, second data time, target time, and time error. The output indicates "Average time error 0.227 seconds, maximum time error 0.5 seconds".

[0115] (4) Start the data fusion module The parameter recognition unit identifies the parameter Speed ​​(with the same name), the platform-specific parameters Latitude and Longitude, and outputs the message "1 parameter with the same name, 2 platform-specific parameters".

[0116] The fusion framework building block creates a fusion data framework with 2400 rows and columns including Time, Speed, Speed_Source, Latitude, and Longitude.

[0117] The data filling unit traverses the 2400 rows of fused data, determining whether each row is within the matching window based on the mapping table. For indices 200 to 205 (within the matching window), average fusion is applied to Speed, and the data source (Speed_Source) is marked as "fused"; for other indices, platform data values ​​are used and marked as "Data2".

[0118] The module’s output interface provides a complete fused dataset (2400 rows × 5 columns).

[0119] (5) Start the data output module This module receives the fused dataset, mapping table, and matching quality assessment information. It generates an Excel file containing three worksheets: fused data, mapping table, and matching quality assessment. The output is sent to a specified path (fused_data.xlsx).

[0120] The system outputs a fused data file containing 2400 rows of data. Each row includes the time, fused vehicle speed, data source marker, and latitude / longitude information. The mapping table contains 11 rows of data, clearly showing the time correspondence between EDR and platform data. The matching quality evaluation table includes the best matching location, minimum DTW distance, average time error, and fusion statistics.

[0121] Accident analysts using this fused data can clearly see detailed changes in vehicle speed in the 5 seconds before the collision (2Hz sampling from EDR), while also understanding the complete driving process and geographical location in the 40 minutes before and after the accident (1Hz sampling from platform data and GPS information), providing comprehensive data support for accident liability determination.

[0122] The embodiments described above are merely preferred embodiments of the present invention and are not intended to define the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made by those skilled in the art to the technical solutions of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for time alignment and fusion of multi-source automotive data, characterized in that, It includes the following steps: S1: Data Acquisition and Preprocessing S11: Obtain first data and second data. Both data record the values ​​of one or more physical parameters as they change over time. Both data have at least one data item corresponding to the same parameter, but the time base is not consistent. The first data contains the segment to be matched. S12: Convert the time of both into a unified numerical time; S13: Preprocess the second data. If there are missing values ​​in the parameters used for matching, perform interpolation to obtain the interpolated second parameter sequence. S14: Determine the valid data range of the second data, and record the start position, end position, start time, end time, start parameter value, and end parameter value of the valid data range; S2: Sliding window matching based on dynamic time warping: S21: Select the segment to be matched in the first data, determine the sliding window size based on its time span T1, and perform the following operations in the second data: S211: Extract window data for each candidate position i, and determine whether the window time range completely covers T1. If it does not cover, skip the position. S212: Resample the second data within the window time range to the same time point as the segment to be matched in the first data; S213: Calculate the similarity distance between the resampled second data within the sliding window and the first data segment to be matched for the same parameter value using the dynamic time warping algorithm, and record the position i and its corresponding similarity distance D. dtw(i) ; S22: After traversal, select the similarity distance D. dtw(i) Minimum position i best As the best matching position; S3: Establishing time alignment relationships: S31: Select the best matching position i best The second data corresponds to time t best As a benchmark; S32: Treat each time t in the matching segment 1,j Calculate its corresponding time t in the second data. 2,j : ； Among them, t 1,0 is the start time of the first data segment to be matched; j is the position index of the first data segment; S33: In the time series of the second data, find the relationship with t. 2,j The closest moment, obtain the corresponding index k. j Establish the first data index j and the second data index k j The mapping relationship; S34: Repeat S32-S33 until the mapping of all moments of the first data segment to be matched is completed; S4: Data fusion, generating complete fused data based on the time mapping in step S3; S41: Compare the sampling frequencies of the first and second data within the matching window, and select the time point of the data source with the higher sampling frequency as the time reference for the fused data. S42: Upsample the data source with a low sampling frequency to align it with the time points of the high-frequency data source; use linear interpolation, spline interpolation or other interpolation methods to calculate the estimated values ​​of the low-frequency data at the high-frequency time points; S43: Identify identical and dissimilar parameters in the first and second data; The same parameters are those recorded simultaneously; the different parameters are those recorded only in one data entry. S44: Create a fused data framework that covers the full time or part of the second data, where high-frequency time series are used within the matching window; S45: Perform judgment and filling for each time point in the fused data; if the time point is within the matching window, calculate the fused value for parameters with the same name according to the fusion strategy, and fill the corresponding data source value for parameters with different names, using the original value or the interpolated value for filling; if the time point is not within the matching window, use the value of the second data for all parameters; S46: Generate a fused dataset, which includes: time series, fusion results of all identical parameters, and results of all dissimilar parameters that are retained or filled.

2. The method for time alignment and fusion of multi-source automotive data according to claim 1, characterized in that: In step S211, after extracting window data, a data sufficiency judgment is added to determine whether the number of data points in the window meets the preset minimum requirement. If the condition is not met, skip that position. If the conditions are met, proceed with step S212.

3. The method for time alignment and fusion of multi-source automotive data according to claim 1, characterized in that: The resampling in step S212 is based on the sampling frequency relationship between the second data and the segment to be matched in the first data. The resampling method includes, but is not limited to: linear interpolation, spline interpolation, nearest neighbor interpolation, piecewise constant interpolation, or any other method that can convert the second data to the time point of the segment to be matched.

4. The method for time alignment and fusion of multi-source automotive data according to claim 1, characterized in that: In step S213, the dynamic time warping algorithm is used to calculate the similarity distance D between the second data Y after resampling within the window and the sequence X with the same parameter in the first data segment to be matched. dtw (X,Y), the distance calculation formula of the dynamic time warping algorithm is: ； Where X = {x1, x2, ..., x} m }: Represents the parameter sequence of the first data, with a length of m, Y={y1,y2,…,y n } represents the parameter sequence of the second data after resampling, with a length of n, W={w1,w2,…,w K } represents the alignment path, which is a sequence of index pairs, where W k =(i k ,j k ) represents the i-th digit of X. k The element and the j-th element of Y k The elements are aligned, K represents the length of the alignment path, and d(∙,∙) is the distance function between points.

5. The method for time alignment and fusion of multi-source automotive data according to claim 1, characterized in that: The fusion rules in step S432 include average fusion, weighted fusion, priority selection, or Kalman filter fusion.

6. The method for time alignment and fusion of multi-source automotive data according to claim 1, characterized in that: The segment to be matched includes: all or a segment of the first data; When the first data contains multiple segments to be matched, steps S2 to S4 are performed on each segment.

7. A system for time alignment and fusion of multi-source automotive data, characterized in that, It includes: The system includes a data acquisition and preprocessing module, a sliding window matching module based on dynamic time warping, a time alignment relationship establishment module, a data fusion module, and a data output module. The data acquisition and preprocessing module includes: a data input unit, a time standardization unit, and a missing value processing unit; the data acquisition and preprocessing module is used to acquire first data and second data, identify and convert the time format into a unified numerical time, and fill in the missing values ​​of the second data, including interpolation. The sliding window matching module based on dynamic time warping includes: a window parameter setting unit, a sliding window search unit, and an optimal position selection unit; the sliding window matching module based on dynamic time warping is used to set the window size according to the time span of the segment to be matched, extract window data for each candidate position, determine the time coverage, perform resampling and calculate the dynamic time warping similarity distance, and select the position with the smallest distance as the optimal matching position; The time alignment relationship establishment module is used to establish a time mapping relationship between the segment to be matched and the second data based on the best matching position; The data fusion module includes: a parameter data item identification unit, a fusion framework construction unit, and a data filling unit; the data fusion module is used to identify parameters with the same name and parameters with different names, create a fusion data framework with time data items based on the second data time series value, and perform data fusion using different fusion rules based on whether each time moment is within the matching window to generate a complete fusion dataset; The data output module is used to output the fused dataset, time mapping relationship, and matching quality assessment information.

8. The automotive multi-source data time alignment and fusion system according to claim 7, characterized in that: The resampling operation in the sliding window search unit includes: a sampling frequency relationship judgment subunit and an adaptive interpolation subunit, which selects a resampling method based on the sampling frequency relationship between the second data and the segment to be matched.

9. The automotive multi-source data time alignment and fusion system according to claim 7, characterized in that: The sliding window search unit also includes a data sufficiency judgment unit, which is used to determine whether the number of data points in the window meets the preset minimum requirement, and skips the position if it does not meet the minimum requirement.

10. The automotive multi-source data time alignment and fusion system according to claim 7, characterized in that: The various functional modules in the system communicate through standardized data interfaces.

Images