Timestamp capturing method, system, device and equipment based on system on chip

Abstract

The invention discloses a timestamp capturing method, system, device and equipment based on a system on chip, and relates to the field of intelligent driving, and the method comprises the steps: carrying out the time synchronization of a first network card and a second network card; in response to completion of transmission of a first data frame corresponding to the first sensor by the first communication controller, capturing a first timestamp corresponding to the first data frame through the first network card; and in response to a second data frame generated by the second sensor, capturing a second timestamp corresponding to the second data frame through the second network card. According to the technical scheme, the hardware timestamps of the first data frame and the second data frame can be captured, the scheduling delay of an application layer is avoided, the problems of inconsistent timestamp reference, transmission delay, scheduling delay and the like are solved, the timestamp error of the first sensor and the second sensor is reduced to the microsecond level from the millisecond level, and the data transmission efficiency is improved. While the time sequence alignment precision of multi-sensor fusion and the system reliability are improved, the sensing effect and the safety of intelligent driving are enhanced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of timestamp capture in intelligent driving, and in particular to a timestamp capture method, system, device, and apparatus based on a system-on-chip. Background Art

[0002] Intelligent driving systems require real-time acquisition and fusion of heterogeneous data from multiple sources, including inertial measurement units (IMUs), global navigation satellite system (GNSS) sensors, chassis, millimeter-wave radar, and vision sensors, to achieve high-precision vehicle pose. However, due to the varying sampling periods and transmission delays of various sensors, time deviations between these multi-source data sources occur. Insufficient sensor timestamp accuracy and consistency will directly impact the fusion accuracy of the perception algorithm, reducing the reliability of pose estimation and even compromising the real-time and safety of decision-making and control. Therefore, improving timestamp accuracy and synchronization is crucial to the perception performance of intelligent driving systems.

[0003] IMU usually uses a serial peripheral interface (SPI) to transmit data, but because the standard SPI hardware and software drivers do not have the ability to capture timestamps, the IMU cannot directly provide accurate timestamps. Although the GNSS data output by the GNSS sensor comes with a satellite timestamp, the outdoor GNSS signal is unstable and cannot meet the requirements of a high-security intelligent driving system, so the satellite timestamp cannot be used directly. Based on this, the current method mainly obtains IMU data and GNSS sensor data at the application layer, and uses the local system time when the IMU data and GNSS sensor data are obtained as the data timestamp. This method of obtaining timestamps has problems such as inconsistent timestamp references, transmission delays, and scheduling delays, which can lead to low timestamp precision and accuracy, thereby affecting the perception effect and the safety of intelligent driving. Summary of the Invention

[0004] Typically, when the application layer obtains IMU data and GNSS sensor data, the local system time at that time is used as the data timestamp, which leads to problems such as inconsistent timestamp references, transmission delays, and scheduling delays.

[0005] To solve the above technical problems, a first embodiment of the present disclosure provides a timestamp capture method based on a system-on-chip, wherein the system-on-chip includes a processor, a first network interface card (NIC) corresponding to a first sensor and a first communication controller, and a second network interface card (NIC) corresponding to a second sensor. The method includes:

[0006] Performing time synchronization on the first network card and the second network card;

[0007] In response to the first communication controller completing the transmission of the first data frame corresponding to the first sensor, capturing a first timestamp corresponding to the first data frame through the first network card;

[0008] In response to the second sensor generating a second data frame, a second timestamp corresponding to the second data frame is captured through the second network card.

[0009] A second aspect of the present disclosure provides a system on chip, the system on chip including a processor, a first network card corresponding to a first sensor and a first communication controller, and a second network card corresponding to a second sensor;

[0010] a processor, configured to synchronize time between the first network card and the second network card;

[0011] a first communication controller, configured to transmit a first pulse signal to the first network card in response to transmission of a first data frame corresponding to the first sensor;

[0012] The first network card is configured to determine a first timestamp corresponding to the first data frame in response to the first pulse signal;

[0013] The second network card is used to determine a second timestamp corresponding to a second data frame in response to a second pulse signal transmitted by a second sensor; wherein the second data frame corresponds to the second sensor; and the second pulse signal is a signal generated by the second sensor when generating the second data frame.

[0014] A third embodiment of the present disclosure provides a timestamp capture device based on a system on a chip, the system on a chip including a processor, a first network card corresponding to a first sensor and a first communication controller, and a second network card corresponding to a second sensor, the device including:

[0015] A time synchronization module, used for synchronizing the time between the first network card and the second network card;

[0016] a first determining module, configured to capture, through the first network card, a first timestamp corresponding to the first data frame in response to the first communication controller completing the transmission of the first data frame corresponding to the first sensor;

[0017] The second determining module is configured to capture a second timestamp corresponding to the second data frame through a second network card in response to the second sensor generating the second data frame.

[0018] A fourth embodiment of the present disclosure provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it is used to execute the system-on-chip timestamp capture method provided in the first aspect.

[0019] The fifth aspect embodiment of the present disclosure provides an electronic device, which includes: a processor, a memory for storing processor-executable instructions; a processor, for reading executable instructions from the memory and executing the instructions to implement the timestamp capture method based on the system-on-chip provided in the first aspect above.

[0020] A sixth embodiment of the present disclosure provides a computer program product. When instructions in the computer program product are executed by a processor, the system-on-chip timestamp capture method provided in the first aspect is executed.

[0021] The timestamp capture method based on the system on chip provided by the embodiment of the present disclosure pre-synchronizes the time base of the first network card and the second network card, and captures the physical transmission moment of the first data frame corresponding to the first sensor through the first network card, and captures the generation moment of the second data frame corresponding to the second sensor based on the second network card. It realizes the hardware timestamp capture of the first data frame and the second data frame, avoids the scheduling delay of the application layer, and can eliminate problems such as inconsistent timestamp base, transmission delay and scheduling delay, so that the timestamp error of the first sensor and the second sensor is reduced from millisecond level to microsecond level, while improving the timing alignment accuracy and system reliability of multi-sensor fusion, and enhancing the perception effect and safety of intelligent driving. And there is no need to modify the system on chip hardware, operating system kernel and driver, etc., which can effectively avoid hardware modification and reduce costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a structural diagram of a timestamp capture system provided by an exemplary embodiment of the present disclosure.

[0023] Figure 2 It is a structural diagram of another timestamp capture system provided by an exemplary embodiment of the present disclosure.

[0024] Figure 3 The figure is a flowchart of a timestamp capture method based on a system on a chip provided by an exemplary embodiment of the present disclosure.

[0025] Figure 4 The figure is a flowchart of another system-on-chip-based timestamp capture method provided by an exemplary embodiment of the present disclosure.

[0026] Figure 5 This is a flowchart of another system-on-chip-based timestamp capture method provided by an exemplary embodiment of the present disclosure.

[0027] Figure 6 This is a flowchart of another system-on-chip-based timestamp capture method provided by an exemplary embodiment of the present disclosure.

[0028] Figure 7 This is a flowchart of another system-on-chip-based timestamp capture method provided by an exemplary embodiment of the present disclosure.

[0029] Figure 8 This is a flowchart of another method for capturing a timestamp based on a system on a chip provided by an exemplary embodiment of the present disclosure.

[0030] Figure 9 FIG. 1 is a schematic structural diagram of a system on a chip provided by an exemplary embodiment of the present disclosure.

[0031] Figure 10 It is a structural diagram of a timestamp capture device based on a system on a chip provided by an exemplary embodiment of the present disclosure.

[0032] Figure 11 It is a structural diagram of another system-on-chip-based timestamp capture device provided by an exemplary embodiment of the present disclosure.

[0033] Figure 12 It is a structural diagram of another system-on-chip-based timestamp capture device provided by an exemplary embodiment of the present disclosure.

[0034] Figure 13 It is a structural diagram of another system-on-chip-based timestamp capture device provided by an exemplary embodiment of the present disclosure.

[0035] Figure 14 It is a structural diagram of another system-on-chip-based timestamp capture device provided by an exemplary embodiment of the present disclosure.

[0036] Figure 15 It is a structural diagram of another system-on-chip-based timestamp capture device provided by an exemplary embodiment of the present disclosure.

[0037] Figure 16 It is a schematic diagram of the composition structure of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0039] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0040] Application Overview

[0041] In intelligent driving systems, multi-sensor data fusion is key to achieving high-precision vehicle pose estimation. The system requires real-time acquisition and fusion of heterogeneous data from multiple sources, including IMUs, GNSS sensors, chassis, millimeter-wave radar, and vision sensors. However, due to the varying sampling periods and transmission delays of each sensor, time skews can occur between these multi-source data. Inadequate sensor timestamp accuracy and consistency directly impact the fusion accuracy of the perception algorithm, reducing the reliability of pose estimation and even compromising the real-time and safety of decision-making and control. Therefore, improving timestamp accuracy and synchronization is crucial to the perception performance of intelligent driving systems.

[0042] IMUs typically use SPI to transmit data, but because standard SPI hardware and software drivers lack timestamp capture capabilities, IMUs cannot directly provide accurate timestamps. Existing solutions can only use the local system time when the application layer receives IMU data as the timestamp for the IMU data. However, due to factors such as operating system scheduling delays, fluctuations in interrupt response times, and changes in CPU load, the timestamp error can reach over 5ms (milliseconds) in high-load scenarios. This large timestamp error can lead to timing misalignment during multi-sensor data fusion, which can cause perception algorithm anomalies and ultimately impact the real-time performance and reliability of the intelligent driving system.

[0043] Although GNSS data output by GNSS sensors includes satellite timestamps, outdoor GNSS signals can experience momentary interruptions or jumps due to factors such as tunnel obstruction and multipath effects. This instability makes this timestamp instability unsuitable for high-security intelligent driving systems. Therefore, the system still needs to obtain the local system time when the application layer receives the GNSS data as the timestamp for the GNSS data. This presents the same issues as for IMU data.

[0044] To address the above technical issues, embodiments of the present disclosure provide a system-on-chip (SoC) timestamp capture method. This method pre-synchronizes a first network interface card (NIC) corresponding to an IMU (Integrated Multi-Unit) (IMU) and a second network interface card (NIC) corresponding to a GNSS receiver. Upon completion of an SPI controller transmission of a frame of IMU data, the first NIC captures the first timestamp corresponding to the frame of IMU data; upon generation of a frame of GNSS data by the GNSS receiver, the second NIC captures the second timestamp of the frame of GNSS data. By pre-synchronizing the time bases of the first and second NICs, capturing the physical transmission moment of the IMU data frame by the first NIC, and capturing the generation moment of the GNSS data frame by the second NIC, hardware timestamp capture of the IMU and GNSS data is achieved. This hardware timestamp capture avoids application-layer scheduling delays, eliminates issues such as inconsistent timestamp bases, transmission delays, and scheduling delays, and reduces the timestamp error between the first and second sensors from milliseconds to microseconds. This improves the timing alignment accuracy and system reliability of multi-sensor fusion, while enhancing perception and the safety of intelligent driving. Furthermore, this method eliminates the need to modify the SoC hardware, operating system kernel, and drivers, effectively avoiding hardware modifications and reducing costs.

[0045] Exemplary Systems

[0046] Figure 1 FIG. 1 is a structural diagram of a timestamp capture system provided by an exemplary embodiment of the present disclosure. Figure 1 As shown, the timestamp capture system includes a system on chip 11, an IMU 12, and a GNSS receiver 13. The system on chip 11 includes a first processor 111 (corresponding to the core computing domain), an SPI controller 112, a first network card 113, a second network card 114, and a universal asynchronous receiver / transmitter (UART controller) 115.

[0047] The IMU 12 and the SPI controller 112 are electrically connected via an SPI bus. The SPI bus includes an SCLK (Serial Clock) signal line, a MOSI (Master Out Slave In) signal line, a MISO (Master In Slave Out) signal line, and a CS (Slave Select / Chip Select) signal line. The SCLK, MOSI, and MISO signal lines are used to transmit IMU data (not indicated in the disclosed embodiment), and the CS signal line is used to indicate that the IMU 12 has completed sending data to the SPI controller 112. The PPS pin of the first network interface card 113 is electrically connected to the CS signal line. The SPI controller 112 and the first network interface card 113 are both electrically connected to the first processor 111 via a high-speed bus. In some examples, the high-speed bus may include a high-speed Peripheral Component Interconnect Express (PCIe) bus. The PPS pin of the GNSS receiver 13 is electrically connected to the PPS pin of the second network interface card 114; the GNSS receiver 13 is electrically connected to the UART controller 115 via a UART bus. The second network card 114 and the UART controller 115 are both electrically connected to the first processor 111 via a high-speed bus. The first network card 113 and the second network card 114 are connected via a network that shares a clock domain. Figure 1 As shown, the first processor 111 , the SPI controller 112 , the first network card 113 , the second network card 114 and the UART controller 115 may all be in the application core computing domain (Acore computing domain).

[0048] Figure 2 FIG. 1 is a schematic diagram of another time stamp capture system provided by an exemplary embodiment of the present disclosure. Figure 2 As shown, in Figure 1 Based on the timestamp capture system shown, the system on chip 11 also includes a second processor 116 (corresponding to the microcontroller unit control domain (MCU control domain)) and a system time base management module (STBM) module 117. The STBM module 117 is electrically connected to the second processor 116 via a high-speed bus. The STBM module 117 is connected to the second network card 114 via Ethernet.

[0049] In some embodiments of the present disclosure, the system on chip 11 further includes a CAN controller and a third network card, etc. The embodiments of the present disclosure do not limit the specific hardware included in the system on chip 11.

[0050] Exemplary Methods

[0051] Figure 3 This is a flow chart of a method for capturing a timestamp based on a system on a chip provided by an exemplary embodiment of the present disclosure. The method for capturing a timestamp based on a system on a chip can be applied to Figure 1 The timestamp capture system shown in the system-on-chip 11 may include the following steps 301 to 303 .

[0052] Step 301: synchronize the time of the first network card and the second network card.

[0053] refer to Figure 1 or Figure 2 As shown, the first network card may correspond to the first network card 113, and the second network card may correspond to the second network card 114. Taking the Linux operating system as an example, the system on chip 11 can achieve high-precision time synchronization between the first network card 113 and the second network card 114 through the standard phc2sys tool.

[0054] For example, during the system-on-chip startup phase, the phc2sys tool can automatically perform the following tasks through a background daemon process: read the Precision Time Protocol (PTP) hardware clocks of the first network card 113 and the second network card 114 at the same time at a fixed period, calculate the time difference between the two, and correct the time of the first network card 113 in real time based on the time difference, so that the time of the first network card 113 is consistent with the time of the second network card 114.

[0055] In some examples, the clock frequency and phase of the first network interface card 113 can be dynamically adjusted based on the time difference to gradually eliminate the time deviation. In other examples, the PTP hardware clock (PHC) register value of the first network interface card 113 can be directly modified using the ioctl system call to achieve rapid time correction. The embodiments of the present disclosure do not limit the implementation method of correcting the time of the first network interface card 113. The embodiments of the present disclosure use the example of directly modifying the PHC register value of the first network interface card 113 using the ioctl system call as an example for illustrative description.

[0056] Step 302 : In response to the first communication controller completing the transmission of the first data frame corresponding to the first sensor, a first timestamp corresponding to the first data frame is captured through the first network card.

[0057] refer to Figure 1 or Figure 2 As shown, the first sensor may correspond to the IMU 12 , the first data frame may be a complete frame of IMU data, and the first communication controller may correspond to the SPI controller 112 .

[0058] Taking the Linux operating system as an example, the upper-layer service software of the Linux operating system can call the Linux SPI interface to read IMU data. When the SPI controller 112 completes receiving a single frame of IMU data sent by the IMU 12 via the MOSI, it indicates that the SPI controller has completed the transmission of the single frame of IMU data. At this time, the first network interface card 113 captures the current hardware timestamp and uses this timestamp as the first timestamp corresponding to the frame of IMU data (the first data frame).

[0059] The process of the upper layer business software of the Linux operating system calling the Linux SPI interface to read the IMU data may include: the processor (such as Figure 1 or Figure 2 The first processor 111 in the SPI controller generates an IMU data read command and writes the IMU data read command into the Tx register of the SPI controller; the SPI controller automatically loads the data read command in the Tx register into the shift register; driven by the SCLK clock, the data read command in the shift register is sent to the IMU through the MOSI line; the IMU parses the data read command, returns the requested IMU data to the SPI controller through the MISO line, and stores the IMU data in the Rx register; the processor reads the IMU data in the Rx register.

[0060] In some examples, when the IMU sends a complete frame of IMU data to the SPI controller through the SPI interface, and the frame data is completely stored in the Rx register of the SPI controller, it means that the SPI controller completes the transmission of the first data frame corresponding to the first sensor.

[0061] Step 303: In response to the second sensor generating a second data frame, a second timestamp corresponding to the second data frame is captured by a second network card.

[0062] refer to Figure 1 or Figure 2 As shown, the second sensor may correspond to the GNSS receiver 13 , and the second data frame may be a single frame of GNSS data.

[0063] Take the default working mode of the GNSS receiver 13 as an example. Figure 1 or Figure 2 As shown, the GNSS receiver 13 may automatically enter the active continuous output mode after power-on, and continuously send GNSS data to the UART controller 115 via the UART interface.

[0064] In some examples, after generating a single frame of GNSS data, the GNSS receiver 13 can immediately transmit the GNSS data to the UART controller via the UART interface. At the start of the GNSS data transmission, the PHC of the second network card 114 can capture an accurate hardware timestamp, and use the captured hardware timestamp as the second timestamp corresponding to the current GNSS data frame (the second data frame).

[0065] The timestamp capture method based on the system on chip provided by the embodiment of the present disclosure pre-synchronizes the time base of the first network card and the second network card, and captures the physical transmission moment of the first data frame corresponding to the first sensor through the first network card, and captures the generation moment of the second data frame corresponding to the second sensor based on the second network card. The hardware timestamp capture of the first data frame and the second data frame is realized, which avoids the scheduling delay of the application layer, and can eliminate the problems of inconsistent timestamp base, transmission delay and scheduling delay, so that the timestamp error of the first sensor and the second sensor is reduced from millisecond level to microsecond level. While improving the timing alignment accuracy and system reliability of multi-sensor fusion, it enhances the perception effect and the safety of intelligent driving. And there is no need to modify the system on chip hardware, operating system kernel and driver, etc., which can effectively avoid hardware modification and reduce costs.

[0066] like Figure 4 As shown in the above Figure 3 Based on the illustrated embodiment, step 302 , in response to the first communication controller completing the transmission of the first data frame corresponding to the first sensor, capturing the first timestamp corresponding to the first data frame through the first network card, may include the following steps 3021 and 3022 .

[0067] Step 3021: In response to the first communication controller completing the transmission of the first data frame, a first pulse signal is transmitted to the first network card through the first communication controller.

[0068] For example, the first pulse signal may be a precise pulse per second (PPS) signal formed by switching the CS signal line of the SPI controller between a low level and a high level.

[0069] refer to Figure 1 or Figure 2As shown, when the SPI controller 112 begins receiving a single frame of IMU data sent by the IMU 12 via MOSI, the CS signal line is pulled low. When the single frame of IMU data is received, the CS signal line jumps from low to high. In this way, using the rising edge transition of the CS signal as a hardware trigger condition, a precise PPS signal is generated on the CS pin of the SPI controller 112. Furthermore, by connecting the CS pin to the PPS pin of the first network interface card 113, the generated PPS signal can be transmitted to the PPS pin of the first network interface card 113 for hardware timestamp capture.

[0070] Step 3022: The first network card obtains and saves a first timestamp corresponding to the first data frame in response to the first pulse signal.

[0071] refer to Figure 1 or Figure 2 As shown, when the PPS pin of the first network interface card 113 detects a rising edge of the PPS signal, it automatically triggers the hardware timestamp capture function to record the timestamp of the current moment. In some examples, the first network interface card 113 can automatically write the current value ts1 of the hardware clock (PHC) of the first network interface card 113 directly into a designated timestamp register at the moment of detecting the rising edge.

[0072] The system-on-chip (SoC)-based timestamp capture method provided by the disclosed embodiment transmits a first pulse signal to the first network card when the first communication controller completes the transmission of the first data frame, and obtains and saves the first timestamp through the first network card in response to the first pulse signal. In this way, hardware timestamp capture of the physical transmission moment of the first data frame can be achieved, avoiding scheduling delays at the application layer, eliminating problems such as inconsistent timestamp references, transmission delays, and scheduling delays, and reducing the timestamp error of the first sensor from milliseconds to microseconds. Furthermore, there is no need to modify the SoC hardware, operating system kernel, and drivers, which can effectively avoid hardware modifications and reduce costs.

[0073] In some embodiments of the present disclosure, when a processor acquires an invalid first data frame, if it still reads the first timestamp corresponding to the first data frame from the first network interface card, the timestamp of the invalid data may interfere with timing alignment, causing time series analysis errors, wasting resources and reducing overall system efficiency. To address this technical issue, the processor can determine the validity of the first data frame upon acquiring it, and only read the first timestamp corresponding to the first data frame from the first network interface card if it determines that the first data frame is valid.

[0074] like Figure 5 As shown in the above Figure 3Based on the illustrated embodiment, in step 302, in response to the first communication controller completing the transmission of the first data frame corresponding to the first sensor, after capturing the first timestamp corresponding to the first data frame through the first network card, the timestamp capture method based on the system on chip may also include the following steps 304 and 305.

[0075] Step 304 : In response to the processor acquiring the first data frame, determine the validity of the first data frame.

[0076] refer to Figure 1 or Figure 2 As shown, the processor may correspond to the first processor 111. The upper-layer service software may read the first data frame through the first processor 111, and after reading the first data frame, perform validity check on the first data frame to obtain a first check result for identifying whether the first data frame is valid.

[0077] Exemplarily, the validity check may include one or more of an integrity check, a physical quantity rationality check, and a sensor state check. The integrity check may include a data format check and a cyclic redundancy check (CRC). The physical quantity rationality check may include a value range check and a change rate continuity check. The embodiments of the present disclosure do not limit the types of validity checks. The embodiments of the present disclosure use the example of validity checks including integrity checks, physical quantity rationality checks, and sensor state checks as examples for illustrative purposes.

[0078] In some examples, the first check result may include the first data frame being valid and the first data frame being invalid.

[0079] Step 305: In response to determining that the first data frame is valid, the processor reads a first timestamp corresponding to the first data frame from the first network card.

[0080] refer to Figure 1 or Figure 2 As shown, the upper-layer business software can call the ioctl interface of the first network card 113 when the first check result is that the first data frame is valid, read the PHC current value ts1 from the timestamp register specified by the first network card 113 through the processor 11, and use the PHC current value ts1 as the first timestamp.

[0081] The system-on-chip timestamp capture method provided by the embodiment of the present disclosure determines the validity of the first data frame when the processor obtains the first data frame, and only reads the first timestamp corresponding to the first data frame from the first network card when the first data frame is determined to be valid. This can avoid unnecessary operations to a certain extent, save resource overhead, and improve the overall efficiency of the system.

[0082] In some embodiments of the present disclosure, the upper-layer service software does not process the received first data frame before receiving a time synchronization indication signal. The upper-layer service software initiates the processing of the first data frame only after receiving a time synchronization indication signal indicating that the clocks of the first network interface card and the second network interface card have been aligned. Specifically, only after receiving the time synchronization indication signal is the validity check performed on the first data frame, and if the first data frame is determined to be valid, the first timestamp corresponding to the first data frame is read from the first network interface card.

[0083] like Figure 6 As shown in the above Figure 3 Based on the illustrated embodiment, step 303 of capturing a second timestamp corresponding to the second data frame through a second network card in response to the second sensor generating the second data frame may include the following steps 3031 and 3032 .

[0084] Step 3031: In response to the second sensor generating a second data frame, a second pulse signal is transmitted to the second network card through the second sensor.

[0085] For example, the second pulse signal may be a PPS signal of a GNSS receiver. The second pulse signal may be different from or the same as the first pulse signal. The embodiments of the present disclosure do not limit the relationship between the second pulse signal and the first pulse signal. The embodiments of the present disclosure use the example of the second pulse signal being different from the first pulse signal as an example for illustration.

[0086] refer to Figure 1 or Figure 2 As shown, the GNSS receiver 13 can support the PPS trigger output mode. That is, the GNSS receiver 13 triggers the transmission of a single-frame GNSS signal to the UART controller 115 upon the rising edge of the PPS signal. The PPS signal is a hardware signal that is strictly aligned with the Coordinated Universal Time (UTC) second. Therefore, it can be assumed that the time when the GNSS receiver 13 generates and starts transmitting a single-frame GNSS data is also strictly aligned with the UTC second.

[0087] Continue to refer Figure 1 or Figure 2 As shown, the PPS pin of the GNSS receiver 13 is connected to the PPS pin of the second network card 114, and the PPS signal generated by the GNSS receiver 13 can be transmitted to the second network card 114 through the connection. Therefore, the second network card 114 can respond to the PPS signal to perform hardware timestamp capture.

[0088] Step 3032: The second network card obtains and saves a second timestamp corresponding to the second data frame in response to the second pulse signal.

[0089] refer to Figure 1 or Figure 2 As shown, upon detecting a rising edge of the PPS signal, the PPS pin of the second network card 114 automatically triggers the hardware timestamp capture function to record the current timestamp. In some examples, the second network card 114 can automatically write the current PHC value ts2 of the second network card 114 to a designated timestamp register upon detecting the rising edge.

[0090] The system-on-chip (SoC)-based timestamp capture method provided by the disclosed embodiment transmits a second pulse signal to the second network card when the second sensor generates a second data frame, and obtains and saves the second timestamp through the second network card in response to the second pulse signal. In this way, hardware timestamp capture of the physical transmission moment of the second data frame can be achieved, avoiding scheduling delays at the application layer, eliminating problems such as inconsistent timestamp references, transmission delays, and scheduling delays, and reducing the timestamp error of the second sensor from milliseconds to microseconds. Furthermore, there is no need to modify the SoC hardware, operating system kernel, and drivers, effectively avoiding hardware modifications and reducing costs.

[0091] In some embodiments of the present disclosure, when a processor acquires an invalid second data frame, if it still reads the second timestamp corresponding to the second data frame from the second network interface card, the invalid timestamp may interfere with timing alignment, causing errors in time series analysis, wasting resources and reducing overall system efficiency. To address this technical issue, the processor can first determine the validity of the second data frame upon acquiring it, and only read the second timestamp corresponding to the second data frame from the second network interface card if the second data frame is determined to be valid.

[0092] like Figure 7 As shown in the above Figure 3 Based on the illustrated embodiment, after generating a second data frame in response to the second sensor in step 303 and capturing a second timestamp corresponding to the second data frame through the second network card, the system-on-chip-based timestamp capture method further includes the following steps 306 and 307.

[0093] Step 306: In response to the processor acquiring the second data frame, determine the validity of the second data frame.

[0094] Taking the Linux operating system as an example, refer to Figure 1 or Figure 2 As shown, the upper-layer service software can call the Linux UART interface to read the second data frame through the first processor 111, and after reading the second data frame, perform validity verification on the second data frame to obtain a second verification result for identifying whether the second data frame is valid.

[0095] Similar to the first verification result, the second verification result may include that the second data frame is valid and that the second data frame is invalid.

[0096] Since the implementation method of performing validity check on the second data frame is similar to the implementation method of performing validity check on the first data frame, the embodiment of the present disclosure will not be described in detail here.

[0097] Step 307: In response to determining that the second data frame is valid, the processor reads a second timestamp corresponding to the second data frame from the second network card.

[0098] refer to Figure 1 or Figure 2 As shown, the upper-layer business software can call the ioctl interface of the second network card 114 when the second verification result is that the second data frame is valid, read the PHC current value ts2 from the timestamp register specified by the second network card 114 through the processor 11, and use the PHC current value ts2 as the second timestamp.

[0099] The system-on-chip timestamp capture method provided by the embodiment of the present disclosure determines the validity of the second data frame when the processor obtains the second data frame, and only reads the second timestamp corresponding to the second data frame from the second network card when it is determined that the second data frame is valid. This can avoid unnecessary operations to a certain extent, save resource overhead, and improve the overall efficiency of the system.

[0100] In some embodiments of the present disclosure, in a multi-sensor data fusion scenario, cross-domain time alignment between the Acore computing domain and the MCU control domain needs to be achieved. If the Acore computing domain includes a first sensor and a second sensor, and the MCU control domain includes a third sensor, it is necessary to ensure that the first sensor and the second sensor under the Acore computing domain, as well as the third sensor under the MCU control domain, all meet the time axis alignment. In some examples, the third sensor under the MCU control domain can be first time-aligned with the first sensor or the second sensor under the Acore computing domain, and then the first sensor and the second sensor are time-aligned within the domain. In other examples, the first sensor and the second sensor can be time-aligned first, and then the third sensor under the MCU control domain is time-aligned with the first sensor or the second sensor under the Acore computing domain. The embodiments of the present disclosure do not limit the time axis alignment method of the first sensor, the second sensor, and the third sensor. The embodiments of the present disclosure are illustrative in the following example: first, the third sensor under the MCU control domain is time-aligned with the first sensor or the second sensor under the Acore computing domain, and then the first sensor and the second sensor are time-aligned within the domain.

[0101] For example, the third sensor may be an on-board sensor that transmits data via a CAN bus. In some examples, the third sensor may include a vehicle chassis and a radar. The present disclosure does not limit the type of the third sensor; however, the present disclosure uses the example of a vehicle chassis and a radar as an example.

[0102] like Figure 8 As shown, in Figure 3 Based on the illustrated embodiment, the second network interface card is deployed in the application core computing domain of the system-on-chip (SoC); the SoC includes a system time reference management module corresponding to the third sensor; and the system time reference management module is deployed in the microcontroller unit (MCU) control domain of the SoC. Before synchronizing the time of the first and second NICs in step 301, the SoC-based timestamp capture method further includes the following step 308.

[0103] Step 308: synchronize the system time reference management module and the second network card.

[0104] like Figure 2 As shown, the system time reference management module corresponds to the STBM module 117 , the second network card 114 is deployed in the Acore computing domain of the system on chip 114 ; the STBM module 117 is deployed in the MCU control domain of the system on chip 11 .

[0105] For example, during the system-on-chip startup phase, a background daemon process can automatically execute a synchronization process for synchronizing the system time reference management module and the second network interface card. In some examples, the standard PTP protocol can be used to synchronize the local time of the STBM module 117 to the hardware clock of the second network interface card 114, achieving nanosecond-level precision alignment.

[0106] For example, synchronizing the local time of the STBM module 117 to the hardware clock of the second network interface card 114 using the standard PTP protocol may include three main steps: system architecture accuracy, master-slave clock distribution, and clock synchronization. The specific implementation of synchronizing the local time of the STBM module 117 to the hardware clock of the second network interface card 114 is not further described in detail in this disclosed embodiment.

[0107] The system-on-chip (SoC)-based timestamp capture method provided by the disclosed embodiment can align the time axis of the sensors under the MCU control domain with the second sensor corresponding to the second network card by synchronizing the time of the system time reference management module and the second network card. When the time axis of the second network card is aligned with the time axis of the first network card, timestamps on the same time axis can be collected by each sensor, which is beneficial to improving the precision and accuracy of the timestamp.

[0108] Exemplary Systems

[0109] Figure 9 FIG. 1 is a schematic diagram of a system-on-chip structure provided by an exemplary embodiment of the present disclosure. Figure 9 As shown, the system on chip 90 may include a processor 901 , a first network card 902 and a first communication controller 903 corresponding to the first sensor, and a second network card 904 corresponding to the second sensor.

[0110] The processor 901 is configured to synchronize time between the first network card 902 and the second network card 904;

[0111] The first communication controller 903 is configured to transmit a first pulse signal to the first network card 902 in response to the transmission of the first data frame detected by the first sensor;

[0112] The first network card 902 is configured to determine a first timestamp corresponding to the first data frame in response to the first pulse signal;

[0113] The second network card 904 is used to determine a second timestamp corresponding to the second data frame in response to a second pulse signal transmitted by the second sensor; wherein the second data frame corresponds to the second sensor, and the second pulse signal is a signal generated by the second sensor when generating the second data frame.

[0114] In some embodiments of the present disclosure, the first sensor includes an inertial measurement unit, the second sensor includes a satellite navigation receiver, and the third sensor includes a controller area network bus sensor; and the first communication controller is a serial peripheral interface controller.

[0115] For example, the system on chip 90 may correspond to Figure 1 or Figure 2 The processor 901, the first network card 902, the first communication controller 903 and the second network card 904 may correspond to Figure 1 or Figure 2 The first processor 111, the first network card 113, the SPI controller 112 and the second network card 114 in the embodiment.

[0116] Regarding a system on chip in the above embodiment, the specific manner in which each unit performs operations and the corresponding beneficial effects have been described in detail in the corresponding embodiment part of the timestamp capture method based on the system on chip in the above-mentioned embodiment. Please refer to the corresponding execution operation manner and beneficial technical effects of the above-mentioned exemplary method part, and no further details will be given here.

[0117] Exemplary devices

[0118] Figure 10 FIG is a schematic diagram of a timestamp capture device based on a system on a chip provided by an exemplary embodiment of the present disclosure. Figure 10As shown, the system-on-chip based timestamp capture device 100 may include a first time synchronization module 1001 , a first capture module 1002 and a second capture module 1003 .

[0119] The first time synchronization module 1001 is used to synchronize the time between the first network card and the second network card;

[0120] A first capturing module 1002 is configured to capture a first timestamp corresponding to the first data frame through a first network card in response to the first communication controller completing the transmission of the first data frame corresponding to the first sensor;

[0121] The second capturing module 1003 is configured to capture a second timestamp corresponding to the second data frame through a second network card in response to the second sensor generating the second data frame.

[0122] In some embodiments, as Figure 11 As shown in the above Figure 10 Based on the illustrated embodiment, the first capturing module 1002 may include a first transmitting unit 1101 and a first acquiring and saving unit 1102 .

[0123] The first transmission unit 1101 is configured to transmit a first pulse signal to the first network card via the first communication controller in response to the first communication controller completing transmission of the first data frame;

[0124] The first acquisition and storage unit 1102 is configured to enable the first network card to acquire and store a first timestamp corresponding to the first data frame in response to the first pulse signal.

[0125] In some embodiments, as Figure 12 As shown in the above Figure 10 Based on the illustrated embodiment, the system-on-chip-based timestamp capture device 100 may further include a first determining module 1004 and a first reading module 1005 .

[0126] The first determining module 1004 is configured to determine the validity of the first data frame in response to the processor acquiring the first data frame.

[0127] The first reading module 1005 is configured to, in response to determining that the first data frame is valid, enable the processor to read a first timestamp corresponding to the first data frame from the first network card.

[0128] In some embodiments, as Figure 13 As shown in the above Figure 10 Based on the illustrated embodiment, the second capturing module 1003 may include a second transmitting unit 1301 and a second acquiring and saving unit 1302 .

[0129] The second transmission unit 1301 is configured to generate a second data frame in response to the second sensor, and transmit a second pulse signal to the second network card through the second sensor.

[0130] The second acquiring and saving unit 1302 is configured to acquire and save a second timestamp corresponding to the second data frame by the second network card in response to the second pulse signal.

[0131] like Figure 14 As shown in the above Figure 10 Based on the illustrated embodiment, the system-on-chip-based timestamp capture device 100 may further include a second determination module 1006 and a second reading module 1007 .

[0132] The second determining module 1006 is configured to determine the validity of the second data frame in response to the processor acquiring the second data frame.

[0133] The second reading module 1007 is configured to, in response to determining that the second data frame is valid, enable the processor to read a second timestamp corresponding to the second data frame from the second network card.

[0134] like Figure 15 As shown, the second network card is deployed in the application core computing domain of the system on chip; the system on chip includes a system time reference management module corresponding to the third sensor; the system time reference management module is deployed in the microcontroller unit control domain of the system on chip. Figure 10 Based on the illustrated embodiment, the system-on-chip-based timestamp capture device 100 may further include a second time synchronization module 1008 .

[0135] The second time synchronization module 1008 is used to synchronize the system time reference management module and the second network card.

[0136] Regarding a timestamp capture device based on a system on chip in the above embodiment, the specific manner in which each module performs operations and the corresponding beneficial effects have been described in detail in the corresponding embodiment part of the time synchronization method part of the system on chip mentioned above. Please refer to the corresponding execution operation manner and beneficial technical effects of the above exemplary method part, and no further details will be given here.

[0137] Exemplary electronic devices

[0138] Figure 16 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure. Figure 16 As shown, the electronic device 160 includes one or more processors 1601 and a memory 1602 .

[0139] The processor 1601 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 160 to perform desired functions.

[0140] The memory 1602 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1601 may execute the program instructions to implement the above-mentioned system-on-chip timestamp capture method and / or other desired functions of the various embodiments of the present disclosure.

[0141] In one example, the electronic device 160 may further include an input device 1603 and an output device 1604 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0142] Of course, to simplify, Figure 16 Only some of the components related to the present disclosure in the electronic device 160 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, the electronic device 160 may further include any other appropriate components according to specific application scenarios.

[0143] Exemplary computer program products and computer-readable storage media

[0144] In addition to the above-mentioned methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the system-on-chip timestamp capture method of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0145] The computer program product may be written in any combination of one or more programming languages to implement the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, and conventional procedural programming languages such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0146] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor executes the steps of the system-on-chip timestamp capture method of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0147] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium is, for example, but not limited to, a system, device or component comprising electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0148] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be considered as essential to each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0149] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. A timestamp capture method based on a system-on-chip, the system-on-chip comprising a processor, a first network interface card (NIC) and a first communication controller (FCC) corresponding to a first sensor, and a second network interface card (NIC) corresponding to a second sensor, the method comprising: Performing time synchronization on the first network card and the second network card; In response to the first communication controller completing the transmission of the first data frame corresponding to the first sensor, capturing, through the first network card, a first timestamp corresponding to the first data frame; In response to the second sensor generating a second data frame, a second timestamp corresponding to the second data frame is captured by the second network card.

2. The method according to claim 1, wherein In response to the first communication controller completing the transmission of the first data frame corresponding to the first sensor, capturing, through the first network card, a first timestamp corresponding to the first data frame, includes: In response to the first communication controller completing the transmission of the first data frame, transmitting a first pulse signal to the first network card through the first communication controller; The first network card obtains and saves a first timestamp corresponding to the first data frame in response to the first pulse signal.

3. The method according to claim 1, further comprising: In response to the processor acquiring the first data frame, determining validity of the first data frame; In response to determining that the first data frame is valid, the processor reads a first timestamp corresponding to the first data frame from the first network card.

4. The method according to claim 1, wherein In response to the second sensor generating the second data frame, capturing, by the second network card, a second timestamp corresponding to the second data frame includes: In response to the second sensor generating a second data frame, transmitting a second pulse signal to the second network card through the second sensor; The second network card obtains and saves a second timestamp corresponding to the second data frame in response to the second pulse signal.

5. The method according to claim 4, further comprising: In response to the processor acquiring the second data frame, determining validity of the second data frame; In response to determining that the second data frame is valid, the processor reads a second timestamp corresponding to the second data frame from the second network card.

6. The method according to any one of claims 1 to 5, wherein the second network card is deployed in an application core computing domain of the system-on-chip; the system-on-chip includes a system time reference management module corresponding to the third sensor; the system time reference management module is deployed in a microcontroller unit control domain of the system-on-chip; and the method further comprises: Time synchronization is performed on the system time reference management module and the second network card.

7. A system on chip, comprising a processor, a first network card corresponding to a first sensor and a first communication controller, and a second network card corresponding to a second sensor; The processor is configured to perform time synchronization on the first network card and the second network card; The first communication controller is configured to transmit a first pulse signal to the first network card in response to transmission of a first data frame corresponding to the first sensor; The first network card is configured to determine a first timestamp corresponding to the first data frame in response to the first pulse signal; The second network card is used to determine a second timestamp corresponding to a second data frame in response to a second pulse signal transmitted by the second sensor; wherein the second data frame corresponds to the second sensor; and the second pulse signal is a signal generated by the second sensor when generating the second data frame.

8. The system-on-chip according to claim 7, wherein the second network card is deployed in the application core computing domain of the system-on-chip; the system-on-chip further comprises a system time reference management module corresponding to the third sensor; the system time reference management module is deployed in the microcontroller unit control domain of the system-on-chip; The processor is further configured to perform time synchronization on the system time reference management module and the second network card.

9. The system on chip according to claim 7 or 8, characterized in that: The first sensor includes an inertial measurement unit, the second sensor includes a satellite navigation receiver, and the third sensor includes a controller area network bus sensor; the first communication controller is a serial peripheral interface controller.

10. A timestamp capture device based on a system-on-chip, the system-on-chip comprising a processor, a first network interface card (NIC) and a first communication controller corresponding to a first sensor, and a second network interface card (NIC) corresponding to a second sensor, the device comprising: A time synchronization module, configured to synchronize the time between the first network card and the second network card; a first determining module, configured to capture, through the first network card, a first timestamp corresponding to the first data frame in response to the first communication controller completing the transmission of the first data frame corresponding to the first sensor; The second determining module is configured to capture, in response to the second sensor generating a second data frame, a second timestamp corresponding to the second data frame through the second network card.

11. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the computer program is used to execute the system-on-chip-based timestamp capture method according to any one of claims 1 to 6.

12. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the system-on-chip-based timestamp capture method according to any one of claims 1 to 6.

Images