FPGA-based vehicle-mounted sensor data back-annotation time synchronization method

Abstract

The application belongs to the field of automobile electronics and relates to a vehicle-mounted sensor data back-annotation time synchronization method based on FPGA, which comprises FPGA master cards and sub-cards working in cooperation; when the system is initially operated, each sub-card sends the timestamp of the first data packet to the master card, the master card collects the timestamp information and uses the earliest timestamp as the starting sending time of each sub-card; the master card determines the actual clock frequency of the sub-cards and adjusts the timing length of the timer; the master card and the sub-cards realize clock synchronization calibration through multi-stage calibration and regular periodical calibration; the method utilizes the parallel processing and accurate calibration mechanism of FPGA to realize high-precision time synchronization of multi-sensor data, is suitable for vehicle-mounted sensor systems with high time precision requirements, and significantly improves the performance and reliability of the data back-annotation system.

Description

Technical Field

[0001] The present invention belongs to the field of automotive electronics, and in particular relates to a vehicle-mounted sensor data re-injection time synchronization method based on FPGA. Background Art

[0002] Before achieving intelligent driving, comprehensive vehicle safety testing and simulation are crucial. A key step is utilizing onboard sensor back-injection equipment to test and simulate vehicles. These devices capture a wide range of vehicle operating data and transmit it back to a test center or server for analysis and evaluation. Therefore, developing an efficient and reliable vehicle data back-injection system is crucial.

[0003] During the data collection process, both offline data collected on-road and hardware-in-the-loop (HIL) simulation data are generated by different sensors or simulation algorithms, resulting in significant temporal heterogeneity. Data from camera video, ultrasonic radar, lidar, and other sources are often collected based on different clock sources. This temporal heterogeneity poses significant challenges to data synchronization during vehicle safety testing and intelligent driving simulation.

[0004] To achieve accurate results during testing and simulation, time synchronization is required for data streams from different sensors. The accuracy of time synchronization directly determines the performance of testing and simulation during back-injection. Inaccurate time synchronization can lead to inconsistent data, compromising the reliability of test results. Therefore, extracting timestamps from each channel's data stream and achieving high-precision time synchronization are crucial. Summary of the Invention

[0005] To address the problems of the prior art, the present invention proposes an FPGA-based vehicle-mounted sensor data injection time synchronization method. The method comprises: constructing a vehicle-mounted sensor injection system, which includes an FPGA daughter card and an FPGA master card; the FPGA daughter card implements sensor data reception, timing calculation, sensor data storage, and data timing transmission functions; the FPGA master card completes system initial time determination, global reference clock maintenance, and daughter card time calibration functions;

[0006] When the system is running, each FPGA sub-card sends the timestamp of the first data packet it receives to the FPGA main card. The main card aggregates the timestamps of each sub-card and uses the earliest timestamp among all sub-cards as the initial sending time and returns it to each sub-card.

[0007] Determine the actual frequency of the daughter card clock through the clock comparison method and adjust the timing length of the daughter card timer;

[0008] The daughter card receives sensor data packets in a fixed frame format, calculates the timestamp difference between adjacent data packets, and determines whether the timestamp difference is incorrect. If the timestamp difference is incorrect, the data packet sequence is disordered and the data packet is discarded; if the timestamp difference is correct, the data packet is sent to the main card on a regular basis;

[0009] Multi-level calibration is used between the main card and the sub-card to achieve clock calibration.

[0010] Beneficial effects of the present invention:

[0011] This method leverages the parallel processing capabilities of FPGAs and their high-precision clock synchronization mechanism to ensure high-precision time synchronization of multi-sensor data, improving the performance and reliability of the data injection system. This method is particularly suitable for intelligent driving testing and simulation environments, effectively improving the overall efficiency of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a flow chart of the FPGA-based vehicle-mounted sensor data injection time synchronization method of the present invention;

[0013] Figure 2 This is a schematic diagram of the sensor data packet frame structure of the present invention. DETAILED DESCRIPTION

[0014] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0015] A method for FPGA-based on-vehicle sensor data injection time synchronization, such as Figure 1 As shown, the method includes: the FPGA daughter card realizes the functions of sensor data reception, timing time calculation, sensor data storage and data timing transmission; the FPGA main card completes the system initial time determination, maintenance of the global reference clock, and daughter card time calibration;

[0016] The main card uses a weighted average algorithm to determine the system initial time. The sub-card receives sensor data packets and extracts the timestamp information in the sensor data packets. The sub-card extracts the timestamp of the first received data packet and transmits the extracted timestamp information to the main card. Different sensors have different weights, and the main card determines the system initial time through weighted average.

[0017] Multi-level calibration is used between the master and daughter cards to calibrate the clocks. When a sensor daughter card is first activated, the master card performs a global calibration. The master card also performs periodic calibration at preset intervals. For sensors with high time accuracy, the master card monitors the time differences between sensors in real time. If a sensor deviates from the global reference time, a local calibration is immediately triggered to resynchronize the sensor's data time.

[0018] On a single daughter card, each card receives sensor data packets in a fixed frame format and calculates the timestamp differences between adjacent packets to determine whether the timestamp differences are within a reasonable range. If the timestamp of a packet differs significantly from the timestamps of the preceding and following packets, the daughter card deems the packets out of order or has an incorrect timestamp and proceeds with packet processing.

[0019] In this embodiment, the frame header identifier is used to identify the start of the frame; the frame check code is used to check the integrity and correctness of the data frame; the sensor type is used to identify the type of sensor, such as camera, ultrasonic radar, laser radar, etc.; the sensor configuration is used to identify the specific configuration parameters of the sensor; the sensor channel number is used to distinguish different sensor channels; the data packet timestamp is used to record the time point when the data packet is generated; the sensor data is used to record the data generated by the sensor path sampling or simulation; the frame end identifier is used to identify the end of the frame. Figure 2 shown.

[0020] In this embodiment, the data packet receiving and processing steps are as follows: different subcards receive data packets serially, detect the frame header and frame tail identifiers, extract and cache the sensor type Stype, sensor configuration information Smess, data packet timestamp information T_send, and channel number N_ID in the data packet according to the frame structure.

[0021] Furthermore, the system initial time synchronization step is specifically as follows: the FPGA main card summarizes the earliest timestamp information between each sub-card as the initial sending time, and the sub-card sensor uses relative time for regular transmission.

[0022] The clock frequency of the main card is f a , determine the actual frequency f of the daughter card clock by clock comparison method b In a fixed time interval T, the master card's clock count is N a , the daughter card's clock count is N b , then the actual frequency of the daughter card clock is f b It can be calculated by the following formula:

[0023]

[0024] By using this calculation method, the actual frequency of the daughter card clock can be accurately determined and used to calculate the number of timing times.

[0025] Furthermore, clock synchronization is achieved between the main card and the sub-card through multi-level calibration, and periodic calibration is performed regularly during system operation.

[0026] S1: When the system is initially working, the sub-card calculates the weighted average timestamp based on the main card and performs a global clock calibration.

[0027] S2: The master card sends a calibration signal to each sub-card at a preset interval. The calibration cycle is short for video sensors and radar sensors with high time accuracy, and long for ultrasonic sensors with low time accuracy.

[0028] S3: The master card monitors the time differences between sensors in real time. Once it detects that a sensor's timer timestamp begins to deviate from the global reference time, exceeding a set threshold, it immediately triggers a local calibration to resynchronize the sensor's data time.

[0029] Specifically, the daughter card sends multiple sensor channel data in parallel. Each daughter card opens a buffer BRAM for each channel. Each buffer BRAM can cache a maximum of 150 data packets of the channel, and sets a sending timer for each channel.

[0030] Each channel data packet calculates whether the time is reasonable based on the timestamp. The data packet after timing calculation is stored in the BRAM corresponding to each sensor, including the timing duration, sensor data, and the channel wr_index of the number of data packets written. When a new data packet is written to each channel, the wr_index corresponding to the channel is automatically incremented.

[0031] The sending data packet position rd_index is incremented each time the data packet is sent. Each channel uses a first-in-first-out mechanism to send data packets.

[0032] Each sensor channel sends data packets according to the first-in-first-out order. Specifically, the wr_index and rd_index increment range is 0-149. When wr_index and rd_index increase to 149, after the new data packet is written, wr_inedx and rd_index become 0; the number of data packets to be read in the BRAM is calculated. When there are more than or equal to 148 data packets in the buffer BRAM, the data packet processing at the front end of the data stream is stopped, that is, the sending of data to the BRAM of each sensor channel is stopped; when wr_index is greater than rd_index, the number of data packets to be read in the BRAM is: numpack = wr_index - rd_index; when wr_index is less than rd_index, the number of data packets to be read in the BRAM is: numpack = wr_index + 150 - rd_index.

[0033] Determine whether the timestamp of the data packet of each channel is reasonable. For ultrasonic sensors that are not sensitive to time accuracy, perform time error correction and record the difference between the last 10 time increments. For incorrect timestamps, modify them to the last sending time plus the increment time. Specifically:

[0034] S1: Record the incremental difference ΔT of the last 10 timestamps i =T i -T i-1 ,i=1,2,3,…,10;

[0035] S2: Calculate the average of the time difference

[0036] S3: The timestamp T of a data packet detected err In case of error, the timestamp is modified to the last sending time T 10 Add average increment time ΔT avg , T err_correct =T 10 +ΔT avg ;

[0037] S4: The timestamp of the next data packet with an incorrect timestamp is not modified, and it is sent according to the current timestamp, and the average of the time differences is recalculated.

[0038] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation plans of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A vehicle-mounted sensor data injection time synchronization method based on FPGA, characterized in that: include: Build an on-board sensor back-injection system, which includes an FPGA daughter card and an FPGA main card. The FPGA daughter card implements sensor data reception, timing calculation, sensor data storage, and data timing transmission. The FPGA main card completes system initial time determination, maintains the global reference clock, and performs daughter card time calibration. When the system is running, each FPGA sub-card sends the timestamp of the first data packet it receives to the FPGA main card. The main card aggregates the timestamps of each sub-card and uses the earliest timestamp among all sub-cards as the initial sending time and returns it to each sub-card. Determine the actual frequency of the daughter card clock through the clock comparison method and adjust the timing length of the daughter card timer; The daughter card receives sensor data packets in a fixed frame format, calculates the timestamp difference between adjacent data packets, and determines whether the timestamp difference is incorrect. If the timestamp difference is incorrect, the data packet sequence is disordered and the data packet is discarded. If the timestamp difference is correct, the data packet is sent to the master card on time; Multi-level calibration is used between the main card and the sub-card to achieve clock calibration.

2. The FPGA-based vehicle-mounted sensor data injection time synchronization method according to claim 1 is characterized in that: Determining the actual frequency of the daughter card clock by clock comparison method includes: obtaining the clock frequency f of the main card a , obtain the master card's clock count N within a fixed time interval T a and the daughter card's clock count is N b ; Calculate the actual frequency of the daughter card clock based on the main card clock frequency, the main card clock count, and the daughter card clock count.

3. The FPGA-based vehicle-mounted sensor data injection time synchronization method according to claim 1, characterized in that: The timestamp information in the sensor data packet includes: detecting the frame header and frame tail identifiers of the sensor data packet, extracting and caching the sensor type, sensor configuration information, data packet timestamp information T_send, and channel number N_ID in the data packet according to the frame structure.

4. The FPGA-based vehicle-mounted sensor data injection time synchronization method according to claim 1, characterized in that: Determining whether the timestamp difference is incorrect includes: S1, record the timestamp T of the last data packet of channel i i,last_send ; S2. Get the timestamp T of the data packet sent by the current channel i i,now_send , judge T i,now_send Is it greater than T i,last_send ,If , ,then ,the data packet is sent, otherwise it is considered that the sensor data packet of channel i in ,the data packet does not need to be sent; S3. Calculate the timestamp difference ΔT = T i,now_send -T i,last_send ; S4. Set a threshold and determine whether the timestamp difference ΔT is greater than the threshold. If the data packet is out of order or the timestamp is wrong, the subcard marks the data packet as abnormal and processes the abnormal data packet.

5. The FPGA-based vehicle-mounted sensor data injection time synchronization method according to claim 1, characterized in that: Processing of abnormal data packets includes: Step 1: Record the incremental difference ΔT of the last 10 timestamps i =T i -T i-1 ,i=1,2,3,…,10; Step 2: Calculate the average of the time difference Step 3: When the timestamp T of the data packet is detected err In case of error, the timestamp is modified to the last sending time T 10 Add average increment time ΔT avg , T err_correct =T 10 +ΔT avg ; Step 4: For the next data packet with an incorrect timestamp, send it according to the current timestamp and start recalculating the mean of the time difference.

6. The FPGA-based vehicle-mounted sensor data injection time synchronization method according to claim 1, characterized in that: Each sensor channel of the daughter card uses the BRAM resources of the FPGA to cache the data packets to be sent and adopts the FIFO mechanism to send the data packets.

7. The FPGA-based vehicle-mounted sensor data injection time synchronization method according to claim 1, characterized in that: Multi-level calibration includes: global calibration, periodic calibration and local calibration.

8. The FPGA-based vehicle-mounted sensor data injection time synchronization method according to claim 7, characterized in that: The specific steps for multi-level calibration are: Step 1: When the system starts working, the sub-card calculates the initial timing timestamp based on the main card and performs a global clock calibration; Step 2: The master card sends a calibration signal to each sub-card at a preset interval; Step 3: The master card monitors the time differences between sensors in real time. When the sensor's timer timestamp deviates from the global reference time, that is, exceeds the set threshold, a local calibration is performed to resynchronize the sensor's data time.

9. The FPGA-based vehicle-mounted sensor data injection time synchronization method according to claim 7, characterized in that: The periodic calibration cycle is: judge the time accuracy of the sensor. When the time accuracy of the sensor is high, set the calibration cycle to T1. When the time accuracy of the sensor is low, set the calibration cycle to T2, and T1 <T2。

Images