Time synchronization method and related device

Abstract

The invention discloses a time synchronization method and a related device, relates to the field of automatic driving, and synchronously aligns a clock of a camera system and a clock of a laser radar with a clock of an Ethernet switch by taking the clock of the Ethernet switch as a reference. A first synchronization signal of a camera in the camera system is aligned with a second synchronization signal of the laser radar at the whole second moment, so that the signal proportion value of each unit time length is consistent with the signal proportion values of other unit time lengths; and the starting moment of the unit time length corresponds to one first synchronizing signal and one second synchronizing signal, so that the data that the image shot by the camera and the laser radar point cloud are the same picture can be conveniently found out. Wherein the signal proportion value is the ratio of the number of the first synchronizing signals to the number of the second synchronizing signals in the unit time length. Compared with a traditional synchronization mechanism triggered through a wire harness, the complexity and cost of the system are greatly reduced, and the synchronization precision and reliability are improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a time synchronization method and related apparatus. Background Technology

[0002] In current autonomous driving systems, the synchronization mechanism between cameras and LiDAR requires a dedicated wiring harness for triggering. This not only increases the complexity and maintenance difficulty of the vehicle's wiring harness, but also causes signal attenuation during transmission within the harness, making it susceptible to interference from the external environment (such as electromagnetic interference), which leads to reduced synchronization accuracy. Summary of the Invention

[0003] In view of the above problems, this application provides a time synchronization method and related apparatus to reduce complexity and maintenance difficulty. The specific solution is as follows:

[0004] The first aspect of this application provides a time synchronization method, including:

[0005] Using the clock of the Ethernet switch as a reference, the clocks of the camera system and the LiDAR are synchronized and aligned with the clock of the Ethernet switch, respectively.

[0006] The first synchronization signal of the camera in the camera system and the second synchronization signal of the lidar are aligned at whole seconds so that the signal ratio value of each unit of time is consistent with the signal ratio value of other units of time. At the beginning of each unit of time, there is one first synchronization signal and one second synchronization signal. The signal ratio value is the ratio of the number of first synchronization signals to the number of second synchronization signals within the unit of time.

[0007] In one possible implementation, synchronizing and aligning the clocks of the camera system and the LiDAR with the clock of the Ethernet switch, respectively, based on the Ethernet switch's clock, includes:

[0008] The clock of the Ethernet switch is used as the master clock, and the clocks of the camera system and the LiDAR are used as slave clocks, respectively. Time synchronization between the slave clocks and the master clock is performed based on the Universal Precision Time Protocol.

[0009] In one possible implementation, aligning the first synchronization signal of the camera in the camera system with the second synchronization signal of the lidar at whole-second intervals includes:

[0010] Within each unit of time, the initial time of the first phase and the initial time of the second phase are aligned with the start time of the unit of time, respectively. The first phase represents the first occurrence of the first synchronization signal within the unit of time, and the second phase represents the first occurrence of the second synchronization signal within the unit of time.

[0011] In one possible implementation, aligning the first synchronization signal of the camera in the camera system with the second synchronization signal of the lidar at whole-second intervals includes:

[0012] Within each unit of time, the exposure midpoint of the first phase and the exposure time of the second phase are aligned with the start time of the unit of time, respectively. The first phase represents the first occurrence of the first synchronization signal within the unit of time, and the second phase represents the first occurrence of the second synchronization signal within the unit of time.

[0013] In one possible implementation, the time synchronization method further includes:

[0014] The error value between the first synchronization signal and the start time of the unit time is acquired at preset intervals;

[0015] The transmission time of the first synchronization signal is adjusted based on the error value.

[0016] In one possible implementation, the unit duration is a whole hundred milliseconds, and aligning the first synchronization signal of the camera in the camera system with the second synchronization signal of the lidar at whole seconds includes:

[0017] The first synchronization signal of the camera in the camera system is aligned with the second synchronization signal of the lidar at the initial moment of each integer 100 milliseconds.

[0018] A second aspect of this application provides a time synchronization device, comprising:

[0019] A clock synchronization module is used to synchronize and align the clocks of the camera system and the LiDAR with the clock of the Ethernet switch, respectively, using the Ethernet switch's clock as a reference; and,

[0020] The signal alignment module is used to align the first synchronization signal of the camera in the camera system with the second synchronization signal of the lidar at whole seconds, so that the signal ratio value of each unit of time is consistent with the signal ratio value of other units of time, and there is one first synchronization signal and one second synchronization signal at the beginning of each unit of time, and the signal ratio value is the ratio of the number of first synchronization signals to the number of second synchronization signals within the unit of time.

[0021] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the time synchronization method described in the first aspect or any implementation thereof.

[0022] A fourth aspect of this application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0023] The memory is used to store computer programs;

[0024] The processor is used to execute the computer program so that the electronic device can implement the time synchronization method of the first aspect or any implementation thereof.

[0025] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to use the time synchronization method described in the first aspect or any implementation thereof.

[0026] The sixth aspect of this application provides a vehicle, including: a vehicle body and electronic devices disposed in the vehicle body as described in the fourth aspect above.

[0027] By employing the above technical solution, the time synchronization method provided in this application uses the clock of the Ethernet switch as a reference to synchronize and align the clocks of the camera system and the LiDAR with the clock of the Ethernet switch, respectively. Then, the first synchronization signal of the camera in the camera system and the second synchronization signal of the LiDAR are aligned at whole-second intervals, so that the signal ratio value for each unit of time is consistent with the signal ratio value for other units of time. Furthermore, at the beginning of each unit of time, there is one first synchronization signal and one second synchronization signal, thus facilitating the identification of data from the same scene as the image captured by the camera and the LiDAR point cloud. The signal ratio value is the ratio of the number of first synchronization signals and the number of second synchronization signals within a unit of time. Compared to traditional synchronization mechanisms triggered by wire harnesses, this significantly reduces system complexity and cost, improves synchronization accuracy and reliability, and effectively reduces the impact of external interference. Attached Figure Description

[0028] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0029] Figure 1A diagram of a remote control system provided in this application;

[0030] Figure 2 A structural diagram of a terminal provided in this application;

[0031] Figure 3 A structural diagram of a server provided in this application;

[0032] Figure 4 A flowchart of the time synchronization method provided in this application;

[0033] Figure 5 A diagram illustrating the synchronization process with the switch provided in this application;

[0034] Figure 6 A diagram illustrating the alignment process of the synchronization signal provided in this application;

[0035] Figure 7 Alignment diagram of the two synchronization signals provided in this application;

[0036] Figure 8 The exposure midpoint image provided for this application;

[0037] Figure 9 A structural diagram of the time synchronization device provided in this application;

[0038] Figure 10 A structural diagram of the electronic device provided in this application. Detailed Implementation

[0039] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0040] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0041] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0042] See Figure 1 , Figure 1 A schematic diagram of a system architecture for remotely controlling the implementation of this time synchronization method in a vehicle is shown. The system may include a terminal 100 and a server 200. The server 200 may include one or more servers (…). Figure 1 (The example includes a server), and the server 200 can provide the method provided in the embodiments of this application to one or more terminals.

[0043] The terminal 100 may have an application for control execution installed. The application and web page can provide an interface. The terminal 100 can receive relevant parameters input by the user on the control interface and send the parameters to the server 200. The server 200 can control the vehicle based on the received parameters and return the processing result to the terminal 100.

[0044] It should be understood that in some optional implementations, the terminal 100 can also complete the action of obtaining the processing result based on the received parameters on its own, without the need for the server to cooperate. This application embodiment is not limited to this.

[0045] The following description Figure 1 The product form of the mid-terminal 100;

[0046] The terminal 100 in this application embodiment can be a mobile phone, tablet computer, wearable device, vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0047] Figure 2 A schematic diagram of an optional hardware structure for terminal 100 is shown.

[0048] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), a headphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190, and other components. Those skilled in the art will understand that... Figure 2These are merely examples of terminals or multi-functional devices and do not constitute a limitation on terminals or multi-functional devices. They may include more or fewer components than shown in the illustration, or combine certain components, or use different components.

[0049] The input unit 130 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the portable multi-functional device. Specifically, the input unit 130 may include a touchscreen 131 (optional) and / or other input devices 132. The touchscreen 131 can collect touch operations performed by the user on or near it (such as operations performed by the user using fingers, knuckles, styluses, or any suitable object on or near the touchscreen), and drive the corresponding connection devices according to a pre-set program. The touchscreen can detect the user's touch actions, convert the touch actions into touch signals and send them to the processor 170, and can receive and execute commands sent by the processor 170; the touch signal includes at least touch point coordinate information. The touchscreen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, various types of touchscreens, such as resistive, capacitive, infrared, and surface acoustic wave, can be used to implement the touchscreen. Besides the touchscreen 131, the input unit 130 may also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0050] Among them, the input device 132 can receive input data, etc.

[0051] The display unit 140 can be used to display information input by the user or information provided to the user, various menus of the terminal 100, interactive interfaces, file display, and / or playback of any multimedia file. In this embodiment, the display unit 140 can be used to display the interface for control implementation, processing results, etc.

[0052] The memory 120 can be used to store instructions and data. The memory 120 may primarily include an instruction storage area and a data storage area. The data storage area can store various types of data, such as multimedia files and text. The instruction storage area can store software units such as operating systems, applications, and instructions required for at least one function, or subsets or extended sets thereof. It may also include non-volatile random access memory. It provides the processor 170 with hardware, software, and data resources for managing the computing device, supporting control software and applications. It is also used for storing multimedia files, as well as storing running programs and applications.

[0053] The processor 170 is the control center of the terminal 100. It connects various parts of the terminal 100 via various interfaces and lines. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it performs various functions and processes data of the terminal 100, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 170. In some embodiments, the processor and memory can be implemented on a single chip; in some embodiments, they can also be implemented separately on independent chips. The processor 170 can also be used to generate corresponding operation control signals, send them to the corresponding components of the computing processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that the various functional modules therein perform corresponding functions, thereby controlling the corresponding components to act according to the instructions.

[0054] The memory 120 can be used to store software code related to control implementation, the processor 170 can execute the control implementation steps, and can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to achieve the corresponding functions.

[0055] The radio frequency unit 110 (optional) can be used for receiving and transmitting signals during information transmission or calls. For example, it can receive downlink information from the base station and process it for the processor 170; additionally, it can transmit uplink data to the base station. Typically, the RF circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the radio frequency unit 110 can also communicate wirelessly with network devices and other devices. This wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0056] In this embodiment of the application, the radio frequency unit 110 can send control objectives to the server 200 and receive processing results sent by the server 200.

[0057] It should be understood that the radio frequency unit 110 is optional and can be replaced with other communication interfaces, such as a network port.

[0058] The terminal 100 also includes a power supply 190 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0059] Terminal 100 also includes an external interface 180, which can be a standard Micro USB interface or a multi-pin connector, which can be used to connect terminal 100 to other devices for communication or to connect a charger to charge terminal 100.

[0060] Although not shown, terminal 100 may also include a flash, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with various functions, etc., which will not be described in detail here. Some or all of the methods described below can be applied to, for example... Figure 2 In the terminal 100 shown.

[0061] The following description Figure 1 The product form of the mid-range server 200;

[0062] Figure 3 A structural diagram of a server 200 is provided, as follows: Figure 3 As shown, server 200 includes bus 201, processor 202, communication interface 203, and memory 204. Processor 202, memory 204, and communication interface 203 communicate with each other via bus 201.

[0063] Bus 201 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0064] The processor 202 can be any one or more of the following processors: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

[0065] Memory 204 may include volatile memory, such as random access memory (RAM). Memory 204 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0066] The memory 204 can be used to store software code related to control implementation, and the processor 202 can execute the control implementation steps of the chip, and can also schedule other units to achieve corresponding functions.

[0067] It should be understood that the aforementioned terminal 100 and server 200 can be centralized or distributed devices. The processors (e.g., processor 170 and processor 202) in the aforementioned terminal 100 and server 200 can be hardware circuits (such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), general-purpose processors, digital signal processors (DSPs), microprocessors or microcontrollers, etc.) or combinations of these hardware circuits. For example, the processor can be a hardware system with instruction execution capabilities, such as a CPU or DSP, or a hardware system without instruction execution capabilities, such as an ASIC or FPGA, or a combination of the aforementioned hardware systems without instruction execution capabilities and hardware systems with instruction execution capabilities.

[0068] The inventors discovered in practice that while there are existing solutions using the PTP (Precision Time Protocol) for time synchronization, these solutions simply use PTP for time synchronization without optimizing or adjusting the synchronization accuracy and strategies for camera systems and LiDAR. This results in relatively large synchronization errors in current camera systems and LiDAR.

[0069] To address the aforementioned problems, this application provides a time synchronization method. The time synchronization method of this application embodiment will be described in detail below with reference to the accompanying drawings.

[0070] Reference Figure 4 , Figure 4 This is a flowchart illustrating a time synchronization method provided in an embodiment of this application, as shown below. Figure 4 As shown in the figure, the time synchronization method provided in this application embodiment may include steps 401 to 402, which are described in detail below.

[0071] 401. Using the clock of the Ethernet switch as a reference, synchronize and align the clocks of the camera system and the LiDAR with the clock of the Ethernet switch, respectively.

[0072] Specifically, Ethernet switches serve as core devices in the data network of autonomous vehicles. They enable rapid data transmission between different devices within the network. Combining PTP protocol clocks with Ethernet switches allows for high-precision clock synchronization throughout the vehicle network. See details for further information. Figure 5 As shown, when synchronizing clocks based on the PTP or gPTP (Generalized Precision Time Protocol) protocol, the system clock of the LiDAR and the clock of the Ethernet switch are synchronized using the PTP or gPTP protocol. Similarly, the clock of the SOC (System on Chip) in the camera system, i.e., the main controller, is synchronized with the clock of the Ethernet switch using the PTP or gPTP protocol. This ensures that the clocks of the LiDAR and the camera system are consistent with the clock of the Ethernet switch, guaranteeing that all three are synchronized and on the same timeline.

[0073] 402. Align the first synchronization signal of the camera and the second synchronization signal of the lidar in the camera system at whole seconds so that the signal ratio value of each unit of time is consistent with the signal ratio value of other units of time, and at the beginning of each unit of time there is one first synchronization signal and one second synchronization signal, and the signal ratio value is the ratio of the number of first synchronization signals and the number of second synchronization signals within the unit of time.

[0074] Specifically, considering that cameras and LiDAR operate at different frequencies during autonomous driving—for example, a camera operates at 30 Hz (i.e., taking 30 shots per second), while a LiDAR operates at 10 Hz (i.e., taking 10 shots per second)—although their operating frequencies are fixed per second, the start times of their corresponding synchronization signals may differ. (Refer to...) Figure 6 As shown, taking the start-up of a lidar as an example, refer to... Figure 6 As shown in the upper middle section, the synchronization signal does not start sending at the beginning of operation (i.e., the high-level phase in the diagram). Instead, the corresponding synchronization signal triggers the LiDAR to take pictures only after 5 meters. Similarly, the same situation may exist for the camera's synchronization signal. Theoretically, according to their respective operating frequencies, one out of every three frames in the camera should be identical to the LiDAR image, confirming that the image and the LiDAR point cloud are from the same source. However, because the start times of their respective synchronization signals are different, the correspondence between the two synchronization signals will deviate, leading to a discrepancy in the data indicating that the image and the LiDAR point cloud are from the same source, thus affecting the autonomous driving's judgment.

[0075] In response to the above problems, refer to Figure 6 As shown in the lower middle section, the synchronization signal of the LiDAR can be aligned at the exact second, that is, the synchronization signal is sent at time 0. Similarly, the synchronization signal of the camera is also sent at time 0. Figure 7 As shown, the upper waveform represents the camera's synchronization signal, and the lower waveform represents the LiDAR's synchronization signal. Both are represented by a high-level phase. It can be seen that within each unit of time (100ms), the ratio of the two synchronization signals is fixed at 3:1. That is, for every three camera shots, the LiDAR takes one shot, and the timing of each shot corresponds to the first shot from the camera (i.e., it takes place at the beginning of each unit of time, corresponding to the first shot from the camera). This ensures that one out of every three frames from the camera is identical to the LiDAR image, and the timestamps are fixed. This allows the autonomous driving algorithm to easily identify data where the image and the LiDAR point cloud are from the same scene and fuse them together.

[0076] Compared to traditional synchronization mechanisms triggered by wiring harnesses, this time synchronization method reduces additional hardware requirements, significantly reduces system complexity and cost, while improving synchronization accuracy and reliability and reducing the impact of external interference.

[0077] In one possible implementation, to ensure the accuracy of time synchronization, step 401 above, which uses the clock of the Ethernet switch as a reference, synchronizes and aligns the clocks of the camera system and the LiDAR with the clock of the Ethernet switch, can specifically include:

[0078] The clock of the Ethernet switch is used as the master clock, and the clocks of the camera system and the LiDAR are used as slave clocks. Time synchronization between the slave clocks and the master clock is performed based on the Universal Precision Time Protocol.

[0079] Specifically, the Ethernet switch's clock can be directly used as the master clock (eliminating the need for master clock selection). By exchanging synchronization messages between the master and slave clock devices and recording the message transmission times, the network transmission delay and the clock deviation between the master and slave devices can be calculated. Then, the slave clock is adjusted based on the deviation to achieve synchronization with the master clock. The specific process is as follows:

[0080] Step 1: The master clock sends the first synchronization message to the slave clock and records the time t1 when the first synchronization message is sent. After receiving the first synchronization message, the slave clock records the time t2 when it receives it.

[0081] Step 2: Next, the master clock sends the time t1 in the second synchronization message to the slave clock. After receiving this message, the slave clock can parse t1 and obtain the first equation: t1 + network delay + clock skew = t2.

[0082] Step 3: Send a third synchronization message from the clock to the master clock, and record the time t3 when the third synchronization message is sent. After receiving the message, the master clock records the time t4 when it is received.

[0083] Step 4: Next, the master clock puts the time t4 into the fourth synchronization message and sends it to the slave clock. After receiving this message, the slave clock can parse t4 and obtain the first equation: t3 + network delay - clock deviation = t4.

[0084] Based on the above two equations, we can obtain: network delay = [(t2-t1)+(t4-t3)] / 2, clock skew = [(t2-t1)-(t4-t3)] / 2.

[0085] It is understood that those skilled in the art can also use precise time protocols for clock synchronization, which will not be elaborated here.

[0086] In one possible implementation, step 402 above, aligning the first synchronization signal of the camera in the camera system with the second synchronization signal of the lidar at whole seconds, may specifically include:

[0087] Within each unit of time, the initial time of the first phase and the initial time of the second phase are aligned with the start time of the unit of time, respectively. The first phase represents the first synchronization signal that appears within the unit of time, and the second phase represents the first synchronization signal that appears within the unit of time.

[0088] For details, please refer to Figure 7 As shown, in the waveforms of the two synchronization signals, the upper high-level phase represents the camera's shutter activation signal, and the lower high-level phase represents the LiDAR's activation signal. The camera's first synchronization signal and the LiDAR's second synchronization signal are aligned at the initial moment of every 100 milliseconds. The LiDAR can use its own phase-locking function to adjust the phase of its 10Hz operating frequency to align with the 100-millisecond intervals. The camera system's shutter signal is set to 30Hz, and according to the system time, it also starts operating at 100-millisecond intervals. At this time, the LiDAR's initial exposure phase coincides with the camera's shutter phase.

[0089] In one possible implementation, to achieve more precise synchronization between the camera's synchronization signal and the lidar's synchronization signal, step 402 aligns the first synchronization signal of the camera and the second synchronization signal of the lidar in the camera system at whole-second intervals, including:

[0090] Within each unit of time, the exposure midpoint of the first phase and the exposure time of the second phase are aligned with the start time of the unit of time, respectively. The first phase represents the first synchronization signal that appears within the unit of time, and the second phase represents the first synchronization signal that appears within the unit of time.

[0091] Specifically, refer to Figure 8 As shown, because the exposure times of the camera and the LiDAR are not necessarily the same, the camera's shutter signal can be fine-tuned so that the midpoint of the camera's exposure coincides with the LiDAR's exposure time. In this case, it can be assumed that one out of every three frames from the camera is identical to the LiDAR's image, and the timestamps are fixed. Figure 8 The upper part shows the exposure time of the camera, and the lower part shows the exposure time of the LiDAR. The midpoint of the division by the dashed line can be taken as the trigger time for the synchronization signal between the two.

[0092] In one possible implementation, to further ensure the accuracy of time synchronization between the camera and the LiDAR, the error value between the first synchronization signal and the start time of the unit time period can be obtained at preset intervals. Then, the transmission time of the first synchronization signal is adjusted according to the error value.

[0093] Specifically, because the crystal oscillators of the camera system and the Ethernet switch are different, there will be a slight error in the clocks of the two systems. This error will accumulate over time and eventually lead to phase deviation. Therefore, by periodically (e.g., every minute) judging the difference between the camera shutter phase and the trigger time in whole hundreds of milliseconds, phase adjustment can be made in a timely manner to ensure the accuracy of time synchronization.

[0094] The above describes a time synchronization method provided by an embodiment of this application. The following describes the apparatus for performing the above time synchronization method.

[0095] Please see Figure 9 , Figure 9 This is a schematic diagram of a time synchronization device provided in an embodiment of this application. Figure 9 As shown, the time synchronization device includes:

[0096] The clock synchronization module 901 is used to synchronize and align the clocks of the camera system and the LiDAR with the clock of the Ethernet switch, respectively, using the clock of the Ethernet switch as a reference; and,

[0097] The signal alignment module 902 is used to align the first synchronization signal of the camera in the camera system with the second synchronization signal of the lidar at whole seconds, so that the signal ratio value of each unit of time is consistent with the signal ratio value of other units of time, and there is one first synchronization signal and one second synchronization signal at the beginning of the unit of time, and the signal ratio value is the ratio of the number of first synchronization signals to the number of second synchronization signals in the unit of time.

[0098] In one possible implementation, the clock synchronization module 901 uses the clock of the Ethernet switch as a reference to synchronize and align the clocks of the camera system and the LiDAR with the clock of the Ethernet switch, respectively. This process includes:

[0099] The clock of the Ethernet switch is used as the master clock, and the clocks of the camera system and the LiDAR are used as slave clocks, respectively. Time synchronization between the slave clocks and the master clock is performed based on the Universal Precision Time Protocol.

[0100] In one possible implementation, the signal alignment module 902 aligns the first synchronization signal of the camera in the camera system with the second synchronization signal of the lidar at whole-second intervals, including:

[0101] Within each unit of time, the initial time of the first phase and the initial time of the second phase are aligned with the start time of the unit of time, respectively. The first phase represents the first occurrence of the first synchronization signal within the unit of time, and the second phase represents the first occurrence of the second synchronization signal within the unit of time.

[0102] In one possible implementation, the signal alignment module 902 aligns the first synchronization signal of the camera in the camera system with the second synchronization signal of the lidar at whole-second intervals, including:

[0103] Within each unit of time, the exposure midpoint of the first phase and the exposure time of the second phase are aligned with the start time of the unit of time, respectively. The first phase represents the first occurrence of the first synchronization signal within the unit of time, and the second phase represents the first occurrence of the second synchronization signal within the unit of time.

[0104] In one possible implementation, it also includes: a signal fine-tuning module, used for

[0105] The error value between the first synchronization signal and the start time of the unit time is acquired at preset intervals;

[0106] The transmission time of the first synchronization signal is adjusted based on the error value.

[0107] In one possible implementation, the unit duration is a whole hundred milliseconds. The process by which the signal alignment module 902 aligns the first synchronization signal of the camera in the camera system with the second synchronization signal of the lidar at whole seconds includes:

[0108] The first synchronization signal of the camera in the camera system is aligned with the second synchronization signal of the lidar at the initial moment of each integer 100 milliseconds.

[0109] This application also provides an electronic device in its embodiments. (See reference...) Figure 10 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 10 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0110] like Figure 10 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003. When the electronic device is powered on, the RAM 1003 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0111] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1008 including, for example, memory card, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 10 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0112] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the time synchronization methods provided in this application.

[0113] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the time synchronization methods provided in this application.

[0114] This application also provides a vehicle, including: a vehicle body and an electronic device as described above disposed in the vehicle body.

[0115] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0116] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0117] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0118] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A time synchronization method, characterized in that, include: Using the clock of the Ethernet switch as a reference, the clocks of the camera system and the LiDAR are synchronized and aligned with the clock of the Ethernet switch, respectively. The first synchronization signal of the camera in the camera system and the second synchronization signal of the lidar are aligned at whole seconds so that the signal ratio value of each unit of time is consistent with the signal ratio value of other units of time. At the beginning of each unit of time, there is one first synchronization signal and one second synchronization signal. The signal ratio value is the ratio of the number of first synchronization signals to the number of second synchronization signals within the unit of time.

2. The time synchronization method according to claim 1, characterized in that, The step of synchronizing and aligning the clocks of the camera system and the LiDAR with the clock of the Ethernet switch, respectively, based on the Ethernet switch's clock, includes: The clock of the Ethernet switch is used as the master clock, and the clocks of the camera system and the LiDAR are used as slave clocks, respectively. Time synchronization between the slave clocks and the master clock is performed based on the Universal Precision Time Protocol.

3. The time synchronization method according to claim 1, characterized in that, Aligning the first synchronization signal of the camera in the camera system with the second synchronization signal of the lidar at whole seconds includes: Within each unit of time, the initial time of the first phase and the initial time of the second phase are aligned with the start time of the unit of time, respectively. The first phase represents the first occurrence of the first synchronization signal within the unit of time, and the second phase represents the first occurrence of the second synchronization signal within the unit of time.

4. The time synchronization method according to claim 1, characterized in that, Aligning the first synchronization signal of the camera in the camera system with the second synchronization signal of the lidar at whole seconds includes: Within each unit of time, the exposure midpoint of the first phase and the exposure time of the second phase are aligned with the start time of the unit of time, respectively. The first phase represents the first occurrence of the first synchronization signal within the unit of time, and the second phase represents the first occurrence of the second synchronization signal within the unit of time.

5. The time synchronization method according to claim 1, characterized in that, Also includes: The error value between the first synchronization signal and the start time of the unit time is acquired at preset intervals; The transmission time of the first synchronization signal is adjusted based on the error value.

6. The time synchronization method according to any one of claims 1 to 5, characterized in that, The unit duration is in whole 100 milliseconds, and aligning the first synchronization signal of the camera in the camera system with the second synchronization signal of the lidar at whole seconds includes: The first synchronization signal of the camera in the camera system is aligned with the second synchronization signal of the lidar at the initial moment of each integer 100 milliseconds.

7. A time synchronization device, characterized in that, include: The clock synchronization module is used to synchronize and align the clocks of the camera system and the lidar with the clock of the Ethernet switch, respectively, using the clock of the Ethernet switch as a reference. as well as, The signal alignment module is used to align the first synchronization signal of the camera in the camera system with the second synchronization signal of the lidar at whole seconds, so that the signal ratio value of each unit of time is consistent with the signal ratio value of other units of time, and there is one first synchronization signal and one second synchronization signal at the beginning of each unit of time, and the signal ratio value is the ratio of the number of first synchronization signals to the number of second synchronization signals within the unit of time.

8. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the time synchronization method as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the time synchronization method as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the time synchronization method as described in any one of claims 1 to 6.

11. A vehicle, characterized in that, include: The vehicle body and the electronic device as described in claim 9 disposed in the vehicle body.

Images