Multi-sensor time synchronization method, electronic equipment, storage medium and program product

Abstract

The invention provides a multi-sensor time synchronization method, electronic equipment, a storage medium and a program product, which are applied to an intelligent driving system comprising a domain controller and a first-class sensor which does not support a GPTP protocol. The method comprises the following steps: receiving a second pulse signal from a first type of sensor and a data message carrying an internal timestamp by a domain controller; capturing receiving moments of at least two continuous second pulse signals under the domain controller time system, and calculating a second scale ratio between the first type sensor time system and the domain controller time system; based on the second scale ratio and a timestamp in the data message, performing conversion to obtain a mapping time point of a data generation moment of the first type of sensor in a domain controller time system; and determining the mapping time point as a data generation moment of the first type of sensor in the domain controller time system. The high-precision time synchronization between the first-class sensor and the domain controller which do not support the GPTP protocol is realized.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a multi-sensor time synchronization method, electronic equipment, storage medium, and program product. Background Art

[0002] With the rapid development of intelligent driving technology, intelligent driving systems typically include multiple types of sensors for vehicle environmental perception, path planning, and motion control. Based on the sensor's synchronization capabilities and its timing source, existing sensors can be roughly divided into three categories:

[0003] Type 1 sensor: For example, an independent module with GPS timing function (such as a GPS module or positioning box) implements internal GPS timing by connecting to a GPS antenna and outputs an absolute timestamp aligned with UTC (Coordinated Universal Time).

[0004] Type 2 sensors, such as some lidars and radars, typically align their internal clocks with the UTC time output by an external GPS module by receiving 1PPS pulse signals and GPRMC time messages. After alignment, the timestamp carried in the output data frame is the synchronized UTC absolute time.

[0005] Type III sensors, such as cameras, can receive external trigger signals to collect data but lack the ability to synchronize with an external time base. These sensors typically initiate data collection via a trigger signal from a domain control system. The sensor internally records data based on its own counting logic, but the output data only contains the time or frame number relative to its own counter and does not carry a unified absolute timestamp.

[0006] In intelligent driving systems, system time, as the foundation for the coordinated operation of various modules, must meet the requirements of both continuity (no jumps) and accuracy per unit time scale. Currently, some domain controllers use GPS timing to directly modify local time. However, when entering a GPS-enabled environment from a non-GPS environment (such as an underground garage), the system time can suddenly change, causing significant errors in timestamp synchronization in downstream modules, impacting system performance and safety.

[0007] To this end, an increasing number of devices are adopting the IEEE 1588-based GPTP protocol for high-precision time synchronization, with the domain controller acting as the master node to achieve a unified clock for all onboard devices. The domain controller can also control sensor data collection by setting trigger signals, recording sampling times, and improving synchronization consistency.

[0008] However, existing externally connected Type 1 sensors do not yet support GPTP. Usually, the moment the domain controller receives the sensor data message is used as the sensor's sampling moment, ignoring the internal calculation delay and transmission delay of the Type 1 sensor. This leads to synchronization errors, especially affecting the system's perception accuracy and control response in scenarios such as high-speed driving. Summary of the Invention

[0009] In response to the shortcomings of the existing technology, the present application provides a multi-sensor time synchronization method, electronic device, storage medium and program product, which are at least used to solve the problem of multi-sensor data synchronization errors caused by the lack of precise time synchronization of external sensors in the existing technology.

[0010] In order to achieve the above objectives and other advantages, some embodiments of the present application provide the following aspects:

[0011] In a first aspect, some embodiments of the present application provide a multi-sensor time synchronization method, which is applied to an intelligent driving system including a domain controller and a first type of sensor that does not support the GPTP protocol. The method includes:

[0012] The domain controller receives the pulse-per-second signal and the data message carrying the internal timestamp from the first type of sensor;

[0013] Capturing the reception time of at least two consecutive second pulse signals in the domain controller time system, and calculating the second scale ratio between the first type sensor time system and the domain controller time system;

[0014] Based on the second scale ratio and the timestamp in the data message, convert and obtain a mapping time point of the data generation moment of the first type of sensor in the domain controller time system;

[0015] The mapping time point is determined as the data generation moment of the first type of sensor in the domain controller time system, so as to achieve time synchronization between the first type of sensor and the domain controller.

[0016] In a second aspect, some embodiments of the present application further provide an electronic device, comprising:

[0017] One or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processors execute any one of the multi-sensor time synchronization methods described above.

[0018] In a third aspect, some embodiments of the present application further provide a computer-readable storage medium having stored thereon a computer program and / or instructions, which, when executed by a processor, implements any of the multi-sensor time synchronization methods described above.

[0019] In a fourth aspect, some embodiments of the present application further provide a computer program product, comprising a computer program and / or instructions, which, when executed by a processor, implements any of the multi-sensor time synchronization methods described above.

[0020] Compared with the related art, the solution provided in the embodiment of the present application introduces a mechanism based on the capture of continuous second pulse signals and the calculation of the second scale ratio in the intelligent driving system, and then combines it with the timestamp in the data message to accurately convert and map the corresponding time of the data generation moment of the first type of sensor in the domain controller time system. High-precision time synchronization between the first type of sensor that does not support the GPTP protocol and the domain controller is achieved. This method effectively avoids the synchronization error problem caused by GPS timing jumps and multi-device clock drift in the prior art, improves the synchronization accuracy of data fusion in the multi-sensor system, and enhances the synchronization robustness and system stability of the intelligent driving system in scenarios without an external time reference or GPS signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other implementation methods can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 This is a flow chart of a multi-sensor time synchronization method provided in an embodiment of the present application;

[0023] Figure 2 2 is a schematic diagram of the system structure of a multi-sensor time synchronization method in an intelligent driving system provided by an embodiment of the present application;

[0024] Figure 3 This is a schematic diagram of the timeline of a multi-sensor time synchronization method in an intelligent driving system provided by an embodiment of the present application;

[0025] Figure 4 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0027] First embodiment

[0028] The first embodiment of the present application relates to a multi-sensor time synchronization method, which is applied to an intelligent driving system including a domain controller and a first type of sensor that does not support the GPTP protocol. Figure 1 As shown, the method may include the following steps:

[0029] Step S1: The domain controller receives the pulse-per-second signal and the data message carrying the internal time stamp from the first type of sensor.

[0030] Specifically, in step S1, within the automotive electrical and electronic architecture, the domain controller (DC) is the core master control unit in the intelligent driving system. It is responsible for the centralized management, coordinated control, and data processing of multiple electronic control units within specific vehicle functional domains (e.g., chassis, powertrain, and intelligent driving). As the master clock node for the GPTP protocol, the DC can inherit GPTP clock information from other higher-level networks (e.g., dedicated PTP time servers), providing a stable and consistent time base for the local DC time system.

[0031] The first type of sensors that do not support the GPTP protocol include independent external sensor devices, such as positioning boxes, which usually have a GPS module and an IMU module integrated inside. The second pulse signal and GPRMC message output by the GPS module can be synchronously transmitted to the IMU module. After fusing the data, the IMU module outputs the second pulse signal, the NMEA standard GPRMC message, and the IMU data. It can also further include optional positioning results and a status message indicating whether the time is valid. This type of sensor is not limited to positioning boxes that integrate GPS modules and IMU modules. It can also be a positioning box that integrates a GPS module alone, or an inertial navigation module, an attitude reference module, or other external sensor devices that can output second pulse signals, data messages, and status messages indicating whether the time is valid.

[0032] The first-class sensors establish synchronization with the domain controller via pulse-per-second signals and data messages. The pulse-per-second signal received by the domain controller is preferably a 1PPS pulse signal, a standard signal output by the first-class sensors with one synchronization pulse per second. This signal has a physical level signal with a standard 1-second period and a waveform with a high-level to low-level duty cycle of approximately 50%. Its rising edge serves as the synchronization reference point, indicating the start of the current second. The error in the arrival time of the rising edge relative to the theoretical start of the second is less than 50 nanoseconds, and the rising edge transition time is less than 5 nanoseconds, providing a highly accurate time synchronization reference.

[0033] Since the frequency of the data message output by the first type of sensor is usually higher than the output frequency of the 1PPS pulse signal (i.e., 1Hz), the domain controller will receive data messages with a higher frequency during actual operation. For example, the GPS module usually outputs positioning and time data at a frequency of 1Hz to 10Hz, while the inertial measurement unit can achieve high-frequency data output of 100Hz to 1000Hz. The data message includes internal timestamp information generated by the internal measurement unit or positioning module of the first type of sensor. Preferably, the data message can be data output by the inertial measurement unit (IMU), the GPRMC standard message output by the global positioning system (GPS) module, or other extended format data containing sensor status and timestamp.

[0034] Step S2: Capture the receiving moments of at least two consecutive second pulse signals in the domain controller time system, and calculate the second scale ratio between the first type sensor time system and the domain controller time system.

[0035] Specifically, regarding step S2, the domain controller continuously captures the reception times of at least two 1PPS pulse signals in its own time system. Preferably, the number of consecutively captured 1PPS pulse signals is two. The domain controller calculates the ratio between the reception time difference of the two pulse signals in its own time system and the output time difference in the internal time system of the first-type sensor to obtain the second-scale ratio between the first-type sensor's time system and the domain controller's time system. This modeling of the relative relationship between the sensor's local clock frequency and the domain controller's time system frequency provides a synchronization foundation for subsequent time mapping.

[0036] Step S3: Based on the second scale ratio and the timestamp in the data message, the mapping time point of the data generation moment of the first type of sensor in the domain controller time system is converted.

[0037] Specifically, in step S3, the domain controller calculates the second-scale ratio between the first-type sensor time system and the domain controller time system. Then, based on the timestamp output by the first-type sensor contained in the received data message, it converts the timestamp from the first-type sensor time system into the domain controller time system using a preset multiplication ratio conversion method. This results in the mapping of the actual sensor data generation time point in the domain controller time system to the actual sensor data generation time point. This eliminates the need to wait for multiple consecutive frames or messages in the accumulation window. Data messages collected in a single session can be instantly converted using this relationship, achieving real-time, high-precision mapping of sensor time based on a single data message to domain controller time.

[0038] Step S4: Determine the mapping time point as the data generation moment of the first type of sensor in the domain controller time system, so as to achieve time synchronization between the first type of sensor and the domain controller.

[0039] Specifically, in step S4, the domain controller uses the mapped time point calculated based on the second-scale ratio in step S3 as the data generation time of the corresponding data message of the first-type sensor in the domain controller's time system. This data generation time is determined as the unified reference timestamp for subsequent multi-sensor data fusion processing, algorithm calculation, and data storage. This effectively eliminates synchronization errors caused by clock frequency drift errors and communication delays between the first-type sensor and the domain controller, ensuring the precise alignment of subsequent multi-source heterogeneous sensor data within the unified time system.

[0040] It is not difficult to find that compared with the related art, the solution provided by the embodiment of the present application introduces a mechanism based on the capture of continuous second pulse signals and the calculation of the second scale ratio in the intelligent driving system, and then combines the timestamp in the data message to accurately convert and map the corresponding time of the data generation moment of the first type of sensor in the domain controller time system. High-precision time synchronization between the first type of sensor that does not support the GPTP protocol and the domain controller is achieved. This method effectively avoids the synchronization error problem caused by GPS timing jumps and multi-device clock drift in the prior art, improves the synchronization accuracy of data fusion in the multi-sensor system, and enhances the synchronization robustness and system stability of the intelligent driving system in scenarios without external time reference or GPS signal.

[0041] Second embodiment

[0042] The second embodiment of the present application relates to a multi-sensor time synchronization method. The second embodiment is an improvement on the first embodiment. The specific improvement is that: in the second embodiment of the present application, the intelligent driving system also includes a second type of sensor that supports the GPTP protocol and a third type of sensor that cannot directly participate in time synchronization. The time synchronization method also includes:

[0043] The domain controller sends a synchronization signal to the second type of sensor through the GPTP protocol to achieve time synchronization between the second type of sensor and the domain controller;

[0044] The domain controller sends a trigger signal to the third type of sensor and records a sampling time when the third type of sensor samples data at a predetermined time point according to the trigger signal;

[0045] The sampling time is used as the data generation time of the third type sensor in the domain controller time system to achieve time synchronization between the third type sensor and the domain controller.

[0046] Specifically, if Figure 2As shown in the figure, the multiple sensor categories also include a second category of sensors that support the GPTP protocol and a third category of sensors that cannot directly participate in time synchronization. The second category of sensors includes: GPTP-supported LiDAR, millimeter-wave radar, vehicle chassis control units, and other sensor devices. The third category of sensors includes: cameras, depth cameras, ultrasonic sensors, infrared sensors, etc.

[0047] For the second type of sensors, the domain controller serves as the master clock node of the GPTP protocol and regularly broadcasts synchronization messages to the slave devices (i.e., the second type of sensors) in the network that support the GPTP protocol through the Ethernet communication link. The second type of sensors receive the synchronization messages and calibrate the local clock according to the timestamp information in the synchronization messages, so that the local time system of the second type of sensors remains continuously synchronized with the domain controller time system, thereby achieving the purpose of time synchronization between the second type of sensors and the domain controller through the GPTP protocol.

[0048] For third-category sensors, the domain controller sends a trigger signal to the sensor via a hardware I / O interface or communication bus, instructing it to perform data sampling at a predetermined time. While sending this trigger signal, the domain controller records the sending time in the domain controller's time system and uses this sending time as the time the third-category sensor generates data in the domain controller's time system. This method effectively eliminates time drift errors caused by the lack of an independent synchronization mechanism for third-category sensors, enabling data time synchronization between third-category sensors that lack the ability to directly participate in GPTP protocol synchronization and the domain controller's time system.

[0049] Furthermore, the domain controller includes a GPS module configured to output a 1PPS pulse-per-second signal and UTC time information. The time synchronization method further includes the domain controller periodically comparing the second scale of the domain controller's time system with the second scale represented by the 1PPS pulse signal output by the GPS module, and adjusting the operating frequency of a clock in the domain controller's time system based on the deviation if a deviation is detected exceeding a preset threshold.

[0050] Specifically, the domain controller receives two consecutive 1PPS pulse signals via its built-in GPS module. The reception times in the domain controller's time system are t_recv_2pps1 and t_recv_2pps2, respectively. Ideally, the interval between two 1PPS pulse signals should be one second. Based on this, the domain controller calculates the actual reception interval Δt = t_recv_2pps2 - t_recv_2pps1 and compares this actual interval with the theoretical value of one second.

[0051] If a deviation exceeding a preset threshold is detected, the domain controller's local time system clock is inferred to be experiencing frequency drift. Based on the measured error, the domain controller adjusts the operating frequency of its internal system clock to align with the second-scale reference output by the built-in GPS module.

[0052] To avoid synchronization instability caused by jumps due to direct system clock modification, the system can also dynamically adjust the system clock's operating frequency within a tolerable deviation range to compensate for smooth time system drift. By continuously fine-tuning the system clock's frequency, gradually aligning it with UTC time over multiple consecutive time periods, the domain controller's time system maintains consistent consistency with UTC time without experiencing sudden changes, effectively improving system stability and smoother time synchronization.

[0053] Preferably, the time alignment process can be repeated once per second during system operation to achieve continuous time maintenance with a granularity of seconds, ensuring that the data of each sensor and control node within the vehicle have a unified, accurate and traceable time reference.

[0054] In addition, when the domain controller cannot inherit the master clock information from other upper-level networks, it can obtain the UTC time reference through the built-in GPS module as the source of its own GPTP master clock, and then act as a GPTP master clock node within the intelligent driving system, broadcasting unified time to downstream devices that support the GPTP protocol.

[0055] Third embodiment

[0056] The third embodiment of the present application relates to a multi-sensor time synchronization method. The third embodiment is an improvement on the first embodiment. The specific improvement is that: in the third embodiment of the present application, a specific implementation method for calculating the second-scale ratio is provided, that is, step S2 can further include the following steps:

[0057] Step S201: continuously capturing two pulse-per-second signals sent by the first type of sensor within a preset time window through the domain controller, and recording the reception time of each pulse-per-second signal under the domain controller time system;

[0058] Step S202: Calculate the reception time difference of the two pulse per second signals in the domain controller time system;

[0059] Step S203: Calculate the ratio of the received time difference to the time difference of the two second pulse signals in the first type of sensor time system to obtain the second scale ratio between the first type of sensor time system and the domain controller time system.

[0060] Specifically, refer to Figure 3As shown, the domain controller captures two 1PPS pulse signals sent continuously by the first type of sensor, records their reception times in the domain controller time system, and records them as t_recv_lpps_2 and t_recv_1pps_1. The reception time difference of the two pulse signals in the domain controller time system is calculated as: Δtgptp = trecv_lpps_2 - trecv_lpps_1. The corresponding 1PPS pulse time difference in the local time system of the first type of sensor is Δt sensor1 , calculate the ratio of the two and obtain the second scale ratio between the first type of sensor time system and the domain controller time system, that is, second scale ratio = Δt gptp / Δt sensor1 , the second scale ratio reflects the frequency deviation relationship between the two time systems.

[0061] In practice, the first type of sensor does not bring out the absolute timestamps of two adjacent 1PPS pulse signals during interaction, but assumes that the interval between the two 1PPS pulses is 1 second, that is, Δt sensor1 = 1 second. Therefore, the second scale ratio calculation formula can be simplified to: Second scale ratio = t_recv_1pps_2 - t_recv_1pps_1. This directly yields the projection length of one second at the sensor's local location in the domain controller's time system, which is the second scale ratio between the first-category sensor time system and the domain controller's time system.

[0062] Furthermore, three to four 1PPS pulse signals can be continuously captured, and multiple scale ratio calculations can be performed on the reception time difference and sensor output time difference between adjacent pulse pairs. The final second scale ratio can be determined by weighted average or least squares fitting algorithm to effectively suppress the synchronization error caused by occasional abnormal pulse interference and improve the accuracy of the final calculation of the second scale ratio.

[0063] It is not difficult to find that in the embodiment of the present application, by continuously capturing at least two 1PPS pulse signals of the first type of sensor within a preset time window and recording their reception time in the domain controller time system, the mapping length of the local 1 second of the sensor in the domain controller time system is directly calculated without relying on additional sensor timestamp data, thereby reducing the data dependence and implementation complexity of the synchronization algorithm.

[0064] It should be noted that the third embodiment of the present application may also be an improvement based on any one or more of the first to second embodiments.

[0065] Fourth embodiment

[0066] The fourth embodiment of the present application relates to a multi-sensor time synchronization method. The fourth embodiment is an improvement on the first embodiment. The specific improvement is that: in the fourth embodiment of the present application, a specific implementation method for data time mapping in the domain controller time system is provided in combination with the second scale ratio. That is, the data message includes IMU data output by the inertial measurement unit. Step S3 can further include the following steps:

[0067] Step SA301: Obtain the first timestamp carried in the IMU data;

[0068] Step SA302: Split the first timestamp into a first integer part and a first decimal part, wherein the first integer part represents the first current second number of the IMU data in the first type of sensor time system, and the first decimal part represents the first relative time position of the IMU data relative to the first current second number;

[0069] Step SA303: Calculate a first offset corresponding to the first relative time position in the domain controller time system using the second scale ratio;

[0070] Step SA304: Add the first offset to the current second reference time to obtain the data generation time of the IMU data in the domain controller time system, where the current second reference time is the time when the domain controller receives the current second pulse signal.

[0071] Reference Figure 3 As shown, the first type of sensor has its own internal clock (t_sensor1 timeline), and the controller has a unified GPTP master clock (t_gptp timeline). These two clocks are not synchronized, and there is initial offset and frequency drift. Therefore, the IMU data's built-in timestamp cannot be directly used. The integer portion of the timestamp does not reflect the actual sampling moment, but only indicates the absolute second number of the current data within the sensor. The decimal portion can restore the precise time when the data was sampled within the current second, that is, the percentage of the time elapsed relative to the start of the current second in the length of the second. Based on the decimal portion of the timestamp, the relative time offset of the current data within the current second is calculated based on the known current second reference time (t_recv_1pps_2) and the second scale ratio. This offset is then added to the current second reference time, thereby accurately restoring the true data generation time of the IMU data in the unified time system of the domain controller.

[0072] Specifically, the domain controller receives the IMU data at the t_recv_imu_gptp moment on the time axis. The IMU data message carries a first timestamp t_sensor1_imu, which is an internal timestamp in the first type of sensor time system that marks the moment when the IMU data is generated.

[0073] The first timestamp, t_sensor1_imu, is split into its first integer and first decimal parts. The first integer part, int(t_sensor1_imu), represents the current second of the IMU data in the first sensor's time system, and the first decimal part, (t_sensor1_imu - int(t_sensor1_imu)), represents the relative time position of the IMU data within the current second. Extracting the decimal portion of the IMU data timestamp enables continuous mapping of any single-point IMU data to the domain controller time, improving the real-time and robustness of the synchronization algorithm.

[0074] Using the second scale ratio calculated from two consecutive 1PPS pulse signals in step S2, calculate the first offset corresponding to the IMU data in the domain controller time system according to the following conversion relationship: First offset = first decimal part × (t_recv_1pps_2 - t_recv_1pps_1). Add this first offset to the current second reference time t_recv_1pps_2, i.e., the time when the 1PPS synchronization signal corresponding to the IMU data was received, to obtain the data generation time t_imu_gptp of the IMU data in the domain controller time system. This is the absolute timestamp of the IMU data on the standard time axis after the timestamp is converted. The specific calculation formula is: t_imu_gptp = t_recv_1pps_2 + (t_sensor1_imu - int(t_sensor1_imu)) × (t_recv_1pps_2 - t_recv_1pps_1).

[0075] In this embodiment, the data message includes GPRMC data output by the GPS module of the first type of sensor, and the GPRMC data includes a second timestamp and a status flag indicating the validity of the data. When the GPRMC data is determined to be valid based on the status flag, step S3 may further include the following steps:

[0076] Step SB301: Obtain the second timestamp carried in the GPRMC data;

[0077] Step SB302: Split the second timestamp into a second integer part and a second decimal part, wherein the second integer part represents the second current second number of the GPRMC data in the first type of sensor time system, and the second decimal part represents the second relative time position of the GPRMC data relative to the second current second number;

[0078] Step SB303: Calculate a second offset corresponding to the second relative time position in the domain controller time system using the second scale ratio;

[0079] Step SB304: Add the second offset to the current second reference time to obtain the data generation time of the GPRMC data in the domain controller time system, where the current second reference time is the time when the domain controller receives the current second pulse signal.

[0080] In practical applications, since the GPRMC data output by the first type of sensor (such as a positioning box with a GPS module) may become invalid or abnormal due to factors such as satellite signal loss and data update lag, it is necessary to determine the status indicator of the validity of the GPRMC data after receiving the GPRMC data.

[0081] Specifically, the domain controller receives GPRMC data at the trecv_gprmc_gptp time on the timeline. GPRMC data also includes a status field, which indicates whether the current GPRMC data is valid. Only if the status field is determined to be valid does the subsequent time conversion and synchronization process proceed. If the status field is invalid, the GPRMC data is discarded to prevent abnormal data from interfering with the synchronization calculation process.

[0082] When the status flag field indicates that the data is valid, obtain the second timestamp t_sensor1_gprmc included in the GPRMC data message. Split the second timestamp t_sensor1_gprmc into a second integer part and a second decimal part, where the second integer part is int(t_sensor1_gprmc), representing the current second number of the GPRMC data in the first type sensor time system, and the second decimal part is (t_sensor1_gprmc-int(t_sensor1_gprmc)), representing the relative time position of the GPRMC data within the current second number.

[0083] Using the second scale ratio calculated by two consecutive 1PPS pulse signals in step S2, the second offset corresponding to the GPRMC data in the domain controller time system is calculated according to the following conversion relationship: second offset = second decimal part × (t_recv_lpps_2-t_recv_lpps_1). This second offset is added to the current second reference time t_recv_1pps_2, that is, the reception time of the 1PPS synchronization signal corresponding to the GPRMC data, to obtain the data generation time t_gprmc_gptp of the GPRMC data in the domain controller time system, that is, the absolute timestamp of the GPRMC data on the standard time axis of the domain controller after the timestamp is converted. The specific calculation formula is: t_gprmc_gptp = trecv_1pps_2 +

[0084] (t_sensorl_gprmc-int(t_sensorl_gprmc))×(t_recv_1pps_2-trecv_1pps_1).

[0085] It is not difficult to find that in the embodiments of this application, using the current 1PPS pulse signal as the anchor point and a second-scale ratio dynamic conversion method, it is possible to achieve real-time, high-precision mapping of the data output by the IMU module and GPS module in the unified time system of the domain controller with only a small amount of external synchronization signal assistance. This eliminates the need for forced synchronization of the internal clock of the first type of sensor, significantly reducing the complexity of system implementation and the requirements for hardware modification.

[0086] It should be noted that the fourth embodiment of the present application may also be an improvement based on any one or more of the first to third embodiments.

[0087] Fifth embodiment

[0088] The fifth embodiment of the present application relates to a multi-sensor time synchronization method. The fifth embodiment is an improvement on the first embodiment. Specifically, the improvement is as follows: In the fifth embodiment of the present application, a specific implementation method for detecting anomalies in a second pulse signal received by a domain controller is provided. The time synchronization method may further include the following steps:

[0089] Detect the reception time difference between two adjacent captured second pulse signals in the domain controller time system; determine whether the reception time difference exceeds the set normal time threshold;

[0090] When the set normal time threshold is exceeded, the captured two adjacent second pulse signals are discarded and the next set of two adjacent second pulse signals are re-acquired for abnormality detection.

[0091] Specifically, the normal time threshold can be determined based on the normal 1PPS output frequency of the first type of sensor (usually 1Hz) and the synchronization error range allowed by the system. Preferably, it can be set to 1 second ±ε, where ε is the maximum deviation allowed by the system (for example, ±100 microseconds). The domain controller detects the reception time difference of two consecutively captured adjacent 1PPS second pulse signals in the domain controller time system. When the detection result shows that the reception time difference of two adjacent 1PPS pulse signals exceeds the normal time threshold, the two captured pulse signals in this group are judged as abnormal data and eliminated, and then the next group of two adjacent 1PPS pulse signals are captured and the abnormality detection is re-executed until a valid pulse signal pair that meets the normal time threshold conditions is obtained, and then the subsequent second scale ratio calculation process is entered.

[0092] It is not difficult to see that in the embodiments of the present application, by presetting a reasonable time threshold, it is possible to monitor and eliminate abnormal pulse signal pairs that exceed the threshold range in real time. This ensures that the second-scale ratio is calculated based only on valid and reliable pulse data pairs, greatly improving the accuracy and reliability of the second-scale ratio calculation.

[0093] It should be noted that the fifth embodiment of the present application may also be an improvement based on any one or more of the first to fourth embodiments.

[0094] In a further embodiment of the present application, to enhance the versatility and scalability of the present invention's time synchronization solution, a universal sensor time synchronization adapter interface module is proposed. This module can serve as an independent universal hardware and software component to achieve unified time synchronization adaptation for various types of external sensors, facilitating integration into the domain controller architecture of an intelligent driving system. Specifically, this universal adapter module abstracts and encapsulates existing access methods for first-, second-, and third-category sensors, providing a unified sensor data reception interface, a 1PPS pulse signal capture interface, a data timestamp parsing interface, and a time mapping algorithm module based on the time scale ratio. This universal adapter module uses standardized interface protocols to achieve seamless integration with external sensors of different brands and protocols. Based on the type of connected sensor, it automatically selects and invokes the corresponding time synchronization calculation logic, such as synchronization command distribution logic for sensors supporting the GPTP protocol, 1PPS pulse signal synchronization and time scale ratio mapping logic for sensors not supporting the GPTP protocol, and trigger signal management and timestamp recording logic for sensors that cannot be directly synchronized. This significantly improves the system's modular design, reduces development and maintenance costs, and enhances the system's scalability and platform adaptability.

[0095] The step division of the above various methods is only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0096] In addition, some embodiments of the present application further provide an electronic device. The electronic device may be various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device may also be various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0097] The electronic device includes: one or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor executes a multi-sensor time synchronization method provided by any one or more of the above embodiments. Figure 4 An exemplary structural diagram of the electronic device is disclosed. The electronic device includes: one or more processors 1101, a memory 1102, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations. Among them, the components shown in this article, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0098] The electronic device may further include: an input device 1103 and an output device 1104. The processor 1101, the memory 1102, the input device 1103 and the output device 1104 may be connected via a bus or other means. Figure 4 The bus connection is taken as an example.

[0099] The input device 1103 can receive input digital or character information and generate key signal input related to user settings and function control of the electronic device, such as input devices such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, and a joystick. The output device 1104 may include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The display device may include, but is not limited to, a liquid crystal display, a light emitting diode display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0100] To provide interaction with a user, the electronic device may be a computer. The computer may include a display device (e.g., a cathode ray tube or LCD monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user. For example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback), and input from the user may be received in any form (e.g., voice input or tactile input).

[0101] In embodiments of the present application, a computer-readable medium stores a computer program / instructions. When executed by a processor, the computer program / instructions implement a multi-sensor time synchronization method provided by any one or more of the aforementioned embodiments. The computer-readable medium may be included in the electronic device described in the aforementioned embodiments, or it may exist independently and not be incorporated into the device. The computer-readable medium carries one or more computer-readable instructions.

[0102] The memory 1102 can be used as a non-transitory computer-readable storage medium to store non-transitory software programs, non-transitory computer executable programs, and modules. The processor 1101 executes the non-transitory software programs, instructions, and modules stored in the memory 1102 to execute various functional applications and data processing of the server, thereby implementing the program instructions / modules corresponding to the method provided in any one or more of the above embodiments of the present application.

[0103] The memory 1102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 1102 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 1102 may optionally include a memory remotely located relative to the processor 1101, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0104] It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above. Computer-readable media may be, for example, but not limited to: electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0105] Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and can be implemented using any method or technology for information storage. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technology, compact discs, digital versatile discs or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0106] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network or a wide area network, or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0107] In the above-described embodiment, can realize wholly or in part by software, hardware, firmware or its arbitrary combination.For example, can adopt application-specific integrated circuit, general-purpose computer or any other similar hardware device to realize.In certain embodiments, the software program of the present application can be carried out to realize above steps or function by processor.Similarly, the software program of the present application (comprising relevant data structure) can be stored in computer-readable recording medium, for example, RAM memory, magnetic or optical drive or floppy disk and similar device.In addition, some steps or functions of the present application can adopt hardware to realize, for example, as the circuit that cooperates with processor to perform each step or function.

[0108] The computer program product provided by the embodiment of the present application includes one or more computer programs / instructions, and when the computer program / instructions are executed by the processor, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instruction can be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium. For example, the computer instruction can be transmitted from a website, a computer, a server or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, a computer, a server or a data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center that includes one or more available media integrations. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state hard disk) etc.

[0109] The flowcharts or block diagrams in the accompanying drawings illustrate the possible architectures, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-specific system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] The scope of this application is defined by the appended claims rather than the foregoing description and is therefore intended to encompass within this application all changes that fall within the meaning and scope of equivalents of the claims. Any reference signs in the claims should not be construed as limiting the claims to which they relate. In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in a device claim may also be implemented by one unit or device through software or hardware. Words such as "first" and "second" are used only to distinguish descriptions and do not indicate any particular order, nor should they be understood as indicating or implying relative importance.

[0111] The above descriptions are merely specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art may easily propose variations or substitutions within the technical scope disclosed in the present application, and such variations or substitutions shall be encompassed within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of protection of the claims, and the above descriptions shall be regarded as exemplary and non-limiting.

Claims

1. A multi-sensor time synchronization method, characterized in that: Applied to an intelligent driving system including a domain controller and a first type of sensor that does not support the GPTP protocol, the method includes: The domain controller receives the pulse-per-second signal and the data message carrying the internal timestamp from the first type of sensor; Capturing the reception time of at least two consecutive second pulse signals in the domain controller time system, and calculating the second scale ratio between the first type sensor time system and the domain controller time system; Based on the second scale ratio and the timestamp in the data message, convert and obtain a mapping time point of the data generation moment of the first type of sensor in the domain controller time system; The mapping time point is determined as the data generation moment of the first type of sensor in the domain controller time system, so as to achieve time synchronization between the first type of sensor and the domain controller.

2. The multi-sensor time synchronization method according to claim 1, characterized in that: The intelligent driving system further includes a second type of sensor that supports the GPTP protocol and a third type of sensor that cannot directly participate in time synchronization. The method further includes: The domain controller sends a synchronization signal to the second type of sensor through the GPTP protocol to achieve time synchronization between the second type of sensor and the domain controller; The domain controller sends a trigger signal to the third type sensor, and records a sampling time when the third type sensor performs data sampling at a predetermined time point according to the trigger signal; The sampling time is used as the data generation time of the third type sensor in the domain controller time system, so as to achieve time synchronization between the third type sensor and the domain controller.

3. The multi-sensor time synchronization method according to claim 1, characterized in that: The step of capturing the reception time of at least two consecutive second pulse signals in the domain controller time system and calculating the second scale ratio between the first type sensor time system and the domain controller time system includes: continuously capturing, by the domain controller, two second pulse signals sent by the first type of sensor within a preset time window, and recording a reception time of each of the second pulse signals under the domain controller time system; Calculating the reception time difference of the two second pulse signals in the domain controller time system; The ratio of the received time difference to the time difference of the two second pulse signals in the first type sensor time system is calculated to obtain the second scale ratio between the first type sensor time system and the domain controller time system.

4. The multi-sensor time synchronization method according to claim 1 or 3, characterized in that: The data message includes IMU data output by the inertial measurement unit; The step of converting the data generation moment of the first type of sensor to obtain the mapping time point in the domain controller time system based on the second scale ratio and the timestamp in the data message includes: Obtaining a first timestamp carried in the IMU data; Splitting the first timestamp into a first integer part and a first decimal part, wherein the first integer part represents a first current second number of the IMU data in a first type sensor time system, and the first decimal part represents a first relative time position of the IMU data relative to the first current second number; Calculating a first offset corresponding to the first relative time position in the domain controller time system using the second scale ratio; The first offset is added to the current second reference time to obtain the data generation time of the IMU data in the domain controller time system, wherein the current second reference time is the time when the domain controller receives the current second pulse signal.

5. The multi-sensor time synchronization method according to claim 1 or 3, characterized in that: The data message includes GPRMC data output by the GPS module of the first type of sensor, and the GPRMC data includes a second timestamp and a status identifier for indicating data validity; When the GPRMC data is determined to be valid based on the status identifier, the step of converting the mapping time point of the data generation moment of the first type of sensor in the domain controller time system based on the second scale ratio and the timestamp in the data message includes: Obtaining a second timestamp carried in the GPRMC data; Splitting the second timestamp into a second integer part and a second fractional part, the second integer part representing a second current second number of the GPRMC data in the first type sensor time system, and the second fractional part representing a second relative time position of the GPRMC data relative to the second current second number; Calculating a second offset corresponding to the second relative time position in the domain controller time system using the second scale ratio; The second offset is added to the current second reference time to obtain the data generation time of the GPRMC data in the domain controller time system, wherein the current second reference time is the time when the domain controller receives the current second pulse signal.

6. The multi-sensor time synchronization method according to claim 1, characterized in that: The domain controller includes a GPS module, and the GPS module is used to output a 1PPS pulse-per-second signal and UTC time information. The method further includes: The domain controller periodically compares the second scale of the domain controller time system with the second scale represented by the 1PPS pulse signal output by the GPS module. If a deviation is detected that exceeds a preset threshold, the operating frequency of the clock in the domain controller time system is adjusted according to the deviation.

7. The multi-sensor time synchronization method according to claim 1, characterized in that: Also includes: The domain controller performs abnormality detection on the received pulse-per-second signal, specifically including: Detecting a reception time difference between two adjacent captured second pulse signals in the domain controller time system; Determining whether the reception time difference exceeds a set normal time threshold; When the set normal time threshold is exceeded, the captured two adjacent second pulse signals are discarded and the next set of two adjacent second pulse signals are re-acquired for abnormality detection.

8. An electronic device, characterized in that: The electronic device comprises: One or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor performs the multi-sensor time synchronization method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program and / or instructions stored thereon, characterized in that: When the computer program and / or the instructions are executed by a processor, the multi-sensor time synchronization method according to any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program and / or instructions, characterized in that When the computer program and / or instruction is executed by a processor, the multi-sensor time synchronization method according to any one of claims 1 to 7 is implemented.

Images