Vehicle-mounted multi-sensor synchronization method, device, equipment and storage medium

Abstract

The application relates to the communication technical field, and discloses a vehicle-mounted multi-sensor synchronization method, device, equipment and storage medium. The method comprises the following steps: based on a set collection period and a set collection phase corresponding to each sensor, performing frequency division or frequency multiplication processing on a local clock signal respectively to generate a collection trigger pulse suitable for the corresponding sensor; wherein the set collection periods have a multiple relationship, and the set collection phases of the sensors with the same layout position are the same; and the collection trigger pulse of each sensor is sent to the corresponding sensor to trigger the sensors to synchronously collect data. Through the technical scheme, the synchronization of data is controlled from the data collection source of each sensor, and the accuracy of the data synchronization of multiple sensors of different types and controlled by different controllers is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a method, apparatus, device and storage medium for synchronizing multiple sensors in a vehicle. Background Technology

[0002] To ensure safe vehicle operation, current vehicles are typically equipped with multiple sensors, such as multiple cameras, at least one lidar sensor, and at least one positioning sensor. In the implementation of technologies such as autonomous driving, automatic driving, and assisted driving, synchronous data from these sensors is required for coordinated perception and positioning.

[0003] Because each sensor in a vehicle independently controls and collects data, and some sensors even have different controllers, the data from these sensors is not synchronized. Currently, the main method for synchronizing sensor data is software fusion. However, this method modifies the original data, reducing the accuracy and consistency of the sensor data. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this application provides a vehicle-mounted multi-sensor synchronization method, apparatus, device and storage medium.

[0005] In a first aspect, embodiments of this application provide a method for synchronizing multiple sensors in a vehicle, the method comprising:

[0006] Based on the set acquisition period and set acquisition phase corresponding to each sensor, the local clock signal is divided or multiplied to generate an acquisition trigger pulse adapted to the corresponding sensor; wherein, the set acquisition periods are related by multiples, and the set acquisition phases of the sensors with the same deployment position are the same.

[0007] The acquisition trigger pulse of each of the sensors is sent to the corresponding sensor to trigger the synchronous acquisition of data by each sensor.

[0008] Secondly, embodiments of this application also provide an in-vehicle multi-sensor synchronization device, the device comprising:

[0009] The acquisition trigger pulse generation module is used to divide or multiply the local clock signal according to the set acquisition period and set acquisition phase corresponding to each sensor, and generate an acquisition trigger pulse adapted to the corresponding sensor; wherein, the set acquisition periods are related by multiples, and the set acquisition phases of the sensors with the same deployment position are the same.

[0010] The sensor synchronization module is used to send the acquisition trigger pulse of each of the sensors to the corresponding sensors to trigger the synchronous acquisition of data by each sensor.

[0011] Thirdly, embodiments of this application also provide an electronic device, which includes:

[0012] Processor and memory;

[0013] The processor executes the steps of the vehicle multi-sensor synchronization method described in any embodiment of this application by calling the program or instructions stored in the memory.

[0014] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a program or instructions that cause a computer to perform the steps of the vehicle-mounted multi-sensor synchronization method described in any embodiment of this application.

[0015] Compared with the prior art, the technical solution provided in this application has the following advantages: a set acquisition period and a set acquisition phase with synchronization characteristics are determined for each sensor, and the transmission of acquisition trigger pulses of each sensor is uniformly controlled by these set acquisition periods and set acquisition phases, ensuring that each sensor can be triggered synchronously to acquire data. This solves the problem of poor data synchronization caused by each sensor independently controlling data acquisition, realizes the synchronization of data from the data acquisition source of each sensor, improves the accuracy of data synchronization of multiple sensors of different types and controlled by different controllers, and thus greatly improves the accuracy and consistency of vehicle perception and positioning technologies. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the control flow for data collection by multiple sensors inside a vehicle, provided in an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of the structure of an in-vehicle multi-sensor synchronization device provided in an embodiment of this application;

[0020] Figure 3This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0021] Figure 4 This is a flowchart of a vehicle-mounted multi-sensor synchronization method provided in an embodiment of this application. Detailed Implementation

[0022] To better understand the above-mentioned objectives, features, and advantages of this application, the solution of this application will be described in further detail below. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of this application, but this application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of this application, and not all embodiments.

[0024] Current vehicle-mounted multi-sensor synchronization technologies primarily employ software fusion to process data collected from different sensors, achieving data synchronization. However, this approach modifies the original data, reducing sensor accuracy and failing to ensure good consistency of the fused multi-sensor data. Based on these considerations, this application provides a vehicle-mounted multi-sensor synchronization solution. This solution determines a set acquisition period and phase with synchronization characteristics for each sensor of different types and controlled by different controllers within the vehicle. These set acquisition periods and phases are then used to uniformly control the emission of acquisition trigger pulses for each sensor, ensuring that all sensors are triggered synchronously to collect data. This controls data synchronization from the data acquisition source of each sensor, improving the accuracy of data synchronization and significantly enhancing the accuracy and consistency of vehicle perception, positioning, and control technologies.

[0025] The vehicle-mounted multi-sensor synchronization scheme provided in this application is mainly used for data synchronization of various sensors in a vehicle equipped with multiple sensors. Figure 1 This is a schematic diagram illustrating the control flow for data collection from multiple sensors in an intelligent driving vehicle, as provided in an embodiment of this application. Figure 1As shown, the vehicle includes at least a main controller (Micro Control Unit, MCU) 110, at least one sub-controller (System on Chip, SOC) 120, at least one splitter 130, and multiple sensors. These sensors include at least one positioning sensor (such as a GPS or BeiDou receiver) 140, multiple lidar sensors 150, multiple cameras 160, and an inertial measurement unit (IMU) 170. The main controller 110 generates corresponding acquisition trigger pulses based on a local clock signal and the set acquisition period and set acquisition phase of each sensor. Then, the main controller 110 sends the acquisition trigger pulses corresponding to the lidar sensors to the splitter 130, so that the splitter 130 divides the acquisition trigger pulses into multiple pulses and sends them simultaneously to each lidar sensor 150; the main controller 110 simultaneously sends the acquisition trigger pulses corresponding to the cameras to each camera 160; the main controller 110 simultaneously sends the acquisition trigger pulses corresponding to the inertial sensor 170 to the inertial sensor 170. After being triggered by the aforementioned acquisition trigger pulses, each sensor acquires data and feeds the acquired data back to the main controller 110. The main controller 110 also receives the position information sent by the positioning sensor 140 through the sub-controller 120. Thus, the main controller 110 can generate synchronous acquisition trigger pulses, thereby uniformly controlling the data acquisition of each sensor and realizing the synchronization of data from each sensor from the source.

[0026] Figure 2 This is a functional block diagram of the vehicle-mounted multi-sensor synchronization device 200 according to an embodiment of this application. In some embodiments, the vehicle-mounted multi-sensor synchronization device 200 can be implemented as follows: Figure 1 The main controller 110, or a part thereof, is used to control the synchronous data acquisition of various sensors in the vehicle. For example... Figure 2 As shown, the vehicle-mounted multi-sensor synchronization device 200 includes at least a trigger pulse generation module 210 and a sensor synchronization module 220.

[0027] The acquisition trigger pulse generation module 210 is used to divide or multiply the local clock signal according to the set acquisition period and set acquisition phase corresponding to each sensor, and generate an acquisition trigger pulse adapted to the corresponding sensor.

[0028] The set acquisition period refers to the acquisition period preset for the sensor. In some embodiments, to ensure complete synchronization of data from sensors of the same type, the same set acquisition period is set for sensors of the same type. For example, the set acquisition periods for each camera are the same, and the set acquisition periods for each LiDAR are the same. In some embodiments, to ensure synchronization of data from different types of sensors, the set acquisition periods with different values ​​are set as multiples of each other. For example, the set acquisition period for LiDAR is an integer multiple of the set acquisition period for the camera. In some embodiments, to further ensure the data synchronization of the sensors, the same set acquisition phase is set for each sensor with the same deployment location.

[0029] In some embodiments, the sensors are a front-view camera, a right-view camera, a rear-view camera, a left-view camera, a front-view radar, a right-view radar, a rear-view radar, a left-view radar, and a positioning sensor. In this embodiment, the front-view camera, right-view camera, rear-view camera, and left-view camera have the same set acquisition period, and the front-view radar, right-view radar, rear-view radar, and left-view radar also have the same set acquisition period. Furthermore, the set acquisition period of the positioning sensor is an integer multiple of the set acquisition period of the radar, and the set acquisition period of the radar is an integer multiple of the set acquisition period of the camera. Additionally, cameras and radars facing the same direction have the same set acquisition phase. For example, the set acquisition phase of the front-view camera and front-view radar is 0°; the set acquisition phase of the right-view camera and right-view radar is 90°; the set acquisition phase of the rear-view camera and rear-view radar is 180°; and the set acquisition phase of the left-view camera and left-view radar is 270°. Based on the above description, the relationship between each sensor and its set acquisition period and set acquisition phase can be obtained as shown in the following table:

[0030]

[0031] Specifically, a set acquisition period and a set acquisition phase are pre-set for each sensor. For each sensor, the acquisition trigger pulse generation module 210 divides or multiplies the local clock signal according to the set acquisition period and set acquisition phase corresponding to that sensor to generate an acquisition trigger pulse adapted to that sensor. The acquisition trigger pulse can be generated by configuring a crystal oscillator in hardware or by using software (such as a counter).

[0032] In some embodiments, when the local clock signal is synchronized with an external clock source, there is a periodic and phase deviation between the local clock signal before and after synchronization. Since the acquisition trigger pulse is generated based on the local clock signal, there are also time and phase differences in the acquisition trigger pulses generated before and after clock synchronization. Therefore, the generation process of the acquisition trigger pulse needs to be adjusted. In this case, the acquisition trigger pulse generation module 210 generates acquisition trigger pulses adapted to each sensor as follows: The acquisition trigger pulse generation module 210 first calculates the number of acquisition trigger pulses to be transmitted (i.e., the number of transmissions) within the clock cycle of the synchronized local clock signal (i.e., the third clock cycle) according to the set acquisition cycle of the sensor. Then, it calculates the time difference between the third clock cycle and the clock cycle of the local clock signal before synchronization (i.e., the first clock cycle), and distributes this time difference evenly to each acquisition trigger pulse transmitted within the third clock cycle to adjust the time interval between each acquisition trigger pulse and determine the transmission time of each acquisition trigger pulse. Furthermore, the transmission phase of each acquisition trigger pulse is adjusted to the set acquisition phase corresponding to the sensor. Subsequently, based on the transmission time and phase of each acquisition trigger pulse determined above, the synchronized local clock signal is divided or multiplied to generate each acquisition trigger pulse of the sensor. In the above process, the clock period refers to the duration of transmitting one clock pulse. For example, if one clock pulse is transmitted every 1 second (s), then the clock period is 1 second.

[0033] The sensor synchronization module 220 is used to send the acquisition trigger pulse of each sensor to the corresponding sensor to trigger synchronous data acquisition by each sensor. Specifically, after the acquisition trigger pulse is generated, the sensor synchronization module 220 sends the acquisition trigger pulse to the corresponding sensor in its corresponding signal transmission mode. For example, for Figure 1 For each lidar 150, the sensor synchronization module 220 sends its corresponding acquisition trigger pulse to the splitter 130. The splitter 130 then distributes the acquisition trigger pulse in multiples to each lidar 150 according to the number of lidars 150. For example, for... Figure 1 In each camera 160, the sensor synchronization module 220 directly sends its corresponding acquisition trigger pulse to the corresponding camera 160. Upon receiving the acquisition trigger pulse, the sensor acquires data. Because the acquisition trigger pulse has its corresponding set acquisition period and set acquisition phase, the data acquired by each sensor based on its acquisition trigger pulse is synchronized. For example, for the left-view camera and left-view lidar in the table above, the third frame image acquired by the left-view camera and the radar data acquired by the left-view lidar are synchronized.

[0034] In some embodiments, the vehicle-mounted multi-sensor synchronization device 200 further includes a clock synchronization module 230, used to synchronize the local clock signal based on the input clock signal of an external clock source before generating a trigger pulse adapted to the corresponding sensor by dividing or multiplying the local clock signal according to the set acquisition period and set acquisition phase corresponding to each sensor. The external clock source has the characteristic of global clock uniformity; for example, it can be a satellite clock source such as BeiDou or GPS, or a Network Time Protocol (NTP) clock source that synchronizes computer time. Specifically, in order to ensure that the timestamps of the data collected by each sensor can more accurately correspond to the time of other vehicles or the cloud, it is necessary to synchronize the vehicle's local clock signal to the clock signal of the external clock source (i.e., the input clock signal).

[0035] In some embodiments, the clock synchronization module 230 includes an input clock signal acquisition submodule, a target clock source determination submodule, and a clock synchronization submodule.

[0036] An input clock signal acquisition submodule is used to acquire input clock signals from at least two external clock sources. Specifically, after the vehicle starts, the input clock signal from each external clock source is collected. In some embodiments, a satellite clock signal from a satellite is received as the input clock signal from the satellite clock source, and a clock signal from an NTP server is received as the input clock signal from the NTP clock source.

[0037] The target clock source determination submodule is used to determine the target clock source based on the priority of each input clock signal and each external clock source. Specifically, a priority is pre-set for each external clock source based on the accuracy of its clock signal. For example, a satellite clock source has a higher priority than an NTP clock source. After acquiring the input clock signal of each external clock source, the target clock source determination submodule can determine the external clock source with the valid input clock signal and the highest priority as the target clock source based on the content of each input clock signal and its corresponding priority. In some embodiments, at least one valid target input clock signal can be determined from the input clock signals first. For example, the validity of the signal can be determined based on the data validity identification information carried in the input clock signal. Then, the target clock source is determined according to the priority corresponding to each target input clock signal. For example, under the above priority settings, if all input clock signals are valid, then the satellite clock source is determined as the target clock source. This ensures the validity of the input clock signals used for subsequent clock synchronization, avoids invalid synchronization processes, and further improves clock synchronization efficiency and real-time performance.

[0038] The clock synchronization submodule is used to synchronize the local clock signal based on the input clock signal of the target clock source. Specifically, it adjusts the local clock signal to synchronize it with the target clock source's clock, using the target clock source's input clock signal as a reference. For vehicles, which contain numerous SOC controllers and sensors, the MCU controller, after clock synchronization, must also send its synchronized local clock signal to each SOC controller and sensor to ensure clock synchronization across multiple SOC controllers and consistency of timestamps recorded by the sensors when acquiring data.

[0039] In some embodiments, the clock synchronization submodule is specifically used to: divide or multiply the input clock signal based on the first clock cycle of the local clock signal and the second clock cycle of the input clock signal to generate a processed input clock signal; and adjust the period and phase of the local clock signal based on the time difference between the third clock cycle of the processed input clock signal and the first clock cycle, and the phase difference between the processed input clock signal and the local clock signal within one clock cycle, to complete the synchronization of the local clock signal. Specifically, the clock synchronization process first ensures that the local clock signal and the input clock signal have the same clock cycle, and then adjusts the phase deviation within one clock cycle. Therefore, in this embodiment, the synchronized clock cycle (i.e., the third clock cycle) is first determined based on the first clock cycle and the second clock cycle. For example, if the second clock cycle is 1s, and the first clock cycle is 0.9s, then the third clock cycle is determined to be 1s; if the first clock cycle is 1.8s or 2.1s, then the third clock cycle is determined to be 2s; if the first clock cycle is 0.48s or 0.51s, then the third clock cycle is determined to be 0.5s. Then, the input clock pulse is divided or multiplied according to the third clock cycle to obtain the processed input clock pulse. The clock cycle of this processed input clock pulse is the third clock cycle. For example, in the example where the first clock cycle is 0.9s and the second clock cycle is 1s, the input clock pulse can be used as the processed input clock pulse. In the example where the first clock cycle is 1.8s and the second clock cycle is 1s, the input clock pulse needs to be divided. In the example where the first clock cycle is 0.48s and the second clock cycle is 1s, the input clock pulse needs to be multiplied. After that, since the local clock signal is synchronized to the input clock signal, it is necessary to calculate and allocate the deviation of the processed input clock signal relative to the local clock signal. After the above processing, the time difference between the third clock cycle and the first clock cycle is very small, and the first clock cycle can be directly adjusted to the third clock cycle. In addition, the phase difference between the clock pulse of the processed input clock signal and the clock pulse of the local clock signal within one clock cycle is calculated, and the phase of the clock pulse of the local clock signal within one clock cycle is adjusted according to this phase difference to complete the clock synchronization process. For example, in the case of a first clock period of 0.9s and a second clock period of 1s, the time difference between the two is 0.1s. The local clock signal can be directly adjusted to transmit a clock pulse once per second. Then, the phase of the local clock signal is adjusted according to the phase of the input clock signal.

[0040] It should be noted that the clock synchronization process also requires time format conversion according to the clock standard. For example, given that the LiDAR, MCU, SOC, and other controllers in the vehicle support pulse-of-seconds (PPS) signals and GPRMC format time information from serial-port-based GPS, these can be used as the clock source standard. In this case, in addition to processing the time difference and phase difference mentioned above, both the local clock signal and the input clock signal need to be converted to GPRMC format.

[0041] Figure 3 This is a schematic diagram of the structure suitable for implementing the electronic device according to the embodiments of this application.

[0042] like Figure 3 As shown, the electronic device includes a central processing unit (CPU) 301, which can execute various processes described in the foregoing embodiments according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage section 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output interface (I / O interface) 305 is also connected to the bus 304.

[0043] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. A removable medium 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 310 as needed so that computer programs read from it can be installed into storage section 308 as needed.

[0044] Specifically, according to embodiments of this application, the in-vehicle multi-sensor synchronization method described herein can be implemented as a computer software program. For example, embodiments of this application include a computer program product comprising a computer program tangibly contained on a readable medium thereof, the computer program containing program code for performing the in-vehicle multi-sensor synchronization method. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311.

[0045] In another aspect, this application also provides a non-transitory computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device described above; or it may be a standalone computer-readable storage medium not assembled into the electronic device. The computer-readable storage medium stores one or more programs that are used by one or more processors to execute the vehicle multi-sensor synchronization method described in this application.

[0046] The vehicle-mounted multi-sensor synchronization method provided in this application is mainly applicable to scenarios involving data synchronization of various sensors in a vehicle equipped with multiple sensors. The vehicle-mounted multi-sensor synchronization method provided in this application can be executed by a vehicle-mounted multi-sensor synchronization device, which can be implemented in software and / or hardware. Figure 4 This is a flowchart of a vehicle-mounted multi-sensor synchronization method provided in an embodiment of this application. See also... Figure 4 The vehicle-mounted multi-sensor synchronization method specifically includes:

[0047] S410: Based on the set acquisition period and set acquisition phase corresponding to each sensor, the local clock signal is divided or multiplied to generate an acquisition trigger pulse adapted to the corresponding sensor.

[0048] The set acquisition period refers to the acquisition period preset for the sensor. In some embodiments, to ensure complete synchronization of data from sensors of the same type, the same set acquisition period is set for sensors of the same type. For example, the set acquisition periods for each camera are the same, and the set acquisition periods for each LiDAR are the same. In some embodiments, to ensure synchronization of data from different types of sensors, the set acquisition periods with different values ​​are set as multiples of each other. For example, the set acquisition period for LiDAR is an integer multiple of the set acquisition period for the camera. In some embodiments, to further ensure the data synchronization of the sensors, the same set acquisition phase is set for each sensor with the same deployment location.

[0049] In some embodiments, the sensors are a front-view camera, a right-view camera, a rear-view camera, a left-view camera, a front-view radar, a right-view radar, a rear-view radar, a left-view radar, and a positioning sensor. In this embodiment, the front-view camera, right-view camera, rear-view camera, and left-view camera have the same set acquisition period, and the front-view radar, right-view radar, rear-view radar, and left-view radar also have the same set acquisition period. Furthermore, the set acquisition period of the positioning sensor is an integer multiple of the set acquisition period of the radar, and the set acquisition period of the radar is an integer multiple of the set acquisition period of the camera. Additionally, cameras and radars facing the same direction have the same set acquisition phase. For example, the set acquisition phase of the front-view camera and front-view radar is 0°; the set acquisition phase of the right-view camera and right-view radar is 90°; the set acquisition phase of the rear-view camera and rear-view radar is 180°; and the set acquisition phase of the left-view camera and left-view radar is 270°. Based on the above description, the relationship between each sensor and its set acquisition period and set acquisition phase can be obtained as shown in the following table:

[0050]

[0051] Specifically, a set acquisition period and a set acquisition phase are pre-set for each sensor. For each sensor, the local clock signal is divided or multiplied according to the set acquisition period and set acquisition phase to generate an acquisition trigger pulse adapted to that sensor. The acquisition trigger pulse can be generated by configuring a crystal oscillator in hardware or by using software (such as a counter).

[0052] In some embodiments, S410 includes: for any sensor: determining the number of times a acquisition trigger pulse is transmitted within a third clock cycle based on the sensor's set acquisition period; wherein the third clock cycle is the clock cycle of the synchronized local clock signal; based on the number of transmissions, allocating the time difference between the third clock cycle and the first clock cycle of the local clock signal to the transmission process of each acquisition trigger pulse within the third clock cycle to adjust the transmission time of each acquisition trigger pulse; adjusting the transmission phase of each acquisition trigger pulse based on the sensor's set acquisition phase, and performing frequency division or frequency multiplication on the synchronized local clock signal based on each transmission time and transmission phase to generate each acquisition trigger pulse of the sensor.

[0053] The clock period refers to the duration of transmitting one clock pulse. For example, if a clock pulse is transmitted once every 1 second (s), then the clock period is 1 second.

[0054] Specifically, when the local clock signal is synchronized with an external clock source, there is a periodic and phase deviation between the local clock signal before and after synchronization. Since the acquisition trigger pulse is generated based on the local clock signal, there are also time and phase differences in the acquisition trigger pulses generated before and after synchronization. Therefore, the generation process of the acquisition trigger pulse needs to be adjusted. In this case, the process of generating acquisition trigger pulses adapted to each sensor can be implemented as follows: First, calculate the number of acquisition trigger pulses to be transmitted (i.e., the number of transmissions) within the clock cycle of the synchronized local clock signal (i.e., the third clock cycle) based on the sensor's set acquisition cycle. Then, calculate the time difference between the third clock cycle and the clock cycle of the local clock signal before synchronization (i.e., the first clock cycle), and distribute this time difference evenly among each acquisition trigger pulse transmitted within the third clock cycle to adjust the time interval between each acquisition trigger pulse and determine the transmission time of each acquisition trigger pulse. Furthermore, adjust the transmission phase of each acquisition trigger pulse to the set acquisition phase corresponding to the sensor. Then, based on the transmission time and phase of each acquisition trigger pulse determined above, the synchronized local clock signal is divided or multiplied to generate each acquisition trigger pulse of the sensor.

[0055] S420: Send the acquisition trigger pulse of each sensor to the corresponding sensor to trigger the synchronous acquisition of data by each sensor.

[0056] Specifically, after the acquisition trigger pulse is generated, it is sent to the corresponding sensor using its corresponding signal transmission method. For example, for Figure 1 For each lidar 150, the sensor synchronization module 220 sends its corresponding acquisition trigger pulse to the splitter 130. The splitter 130 then distributes the acquisition trigger pulse in multiples to each lidar 150 according to the number of lidars 150. For example, for... Figure 1 In each camera 160, the sensor synchronization module 220 directly sends its corresponding acquisition trigger pulse to the corresponding camera 160. Upon receiving the acquisition trigger pulse, the sensor acquires data. Because the acquisition trigger pulse has its corresponding set acquisition period and set acquisition phase, the data acquired by each sensor based on its acquisition trigger pulse is synchronized. For example, for the left-view camera and left-view lidar in the table above, the third frame image acquired by the left-view camera and the radar data acquired by the left-view lidar are synchronized.

[0057] In some embodiments, before dividing or multiplying the local clock signal based on the set acquisition period and set acquisition phase corresponding to each sensor to generate an acquisition trigger pulse adapted to the corresponding sensor, the vehicle-mounted multi-sensor synchronization method further includes: synchronizing the local clock signal based on the input clock signal of an external clock source. The external clock source has the characteristic of global clock uniformity; for example, it can be a satellite clock source such as BeiDou or GPS, or a Network Time Protocol (NTP) clock source that synchronizes computer time. Specifically, to ensure that the timestamps of the data collected by each sensor can more accurately correspond to the time of other vehicles or the cloud, it is necessary to synchronize the vehicle's local clock signal to the clock signal of the external clock source (i.e., the input clock signal).

[0058] In some embodiments, synchronizing a local clock signal based on an input clock signal from an external clock source includes: acquiring input clock signals from at least two external clock sources; determining a target clock source based on the priority of each input clock signal and each external clock source; and synchronizing the local clock signal based on the input clock signal of the target clock source.

[0059] Specifically, priorities are pre-assigned based on the accuracy of the clock signals from each external clock source. For example, satellite clock sources have higher priority than NTP clock sources. Upon vehicle startup, input clock signals from each external clock source are collected. In some embodiments, satellite clock signals are received as input clock signals for satellite clock sources, and clock signals from NTP servers are received as input clock signals for NTP clock sources. After acquiring the input clock signals from each external clock source, the external clock source with the highest priority and valid input clock signal is determined as the target clock source based on the content of each input clock signal and its corresponding priority. In some embodiments, at least one valid target input clock signal can be identified from the input clock signals. For example, the validity of the signal can be determined based on the data validity identifier information carried in the input clock signal. Then, the target clock source is determined according to the priority corresponding to each target input clock signal. For example, with the above priority settings, if all input clock signals are valid, then the satellite clock source is determined as the target clock source. This ensures the validity of subsequent input clock signals used for clock synchronization, avoids invalid synchronization processes, and further improves clock synchronization efficiency and real-time performance. Then, using the input clock signal of the target clock source as a reference, the local clock signal is adjusted to keep it synchronized with the clock of the target clock source. For vehicles, there are many SOC controllers and sensors inside. Therefore, after clock synchronization, the MCU controller also needs to send its synchronized local clock signal to each SOC controller and each sensor to ensure clock synchronization of multiple SOC controllers and consistency of timestamps recorded when sensors collect data.

[0060] In some embodiments, synchronizing a local clock signal based on an input clock signal from a target clock source includes: dividing or multiplying the input clock signal based on a first clock cycle of the local clock signal and a second clock cycle of the input clock signal to generate a processed input clock signal; and adjusting the period and phase of the local clock signal based on the time difference between the third clock cycle and the first clock cycle of the processed input clock signal, and the phase difference between the processed input clock signal and the local clock signal within one clock cycle, to complete the synchronization of the local clock signal.

[0061] Specifically, the clock synchronization process first ensures that the local clock signal and the input clock signal have the same clock period, and then adjusts the phase deviation within one clock period. Therefore, in this embodiment, the synchronized clock period (i.e., the third clock period) is first determined based on the first and second clock periods. For example, if the second clock period is 1s, and the first clock period is 0.9s, then the third clock period is determined to be 1s; if the first clock period is 1.8s or 2.1s, then the third clock period is determined to be 2s; if the first clock period is 0.48s or 0.51s, then the third clock period is determined to be 0.5s. Then, the input clock pulse is divided or multiplied according to the third clock period to obtain the processed input clock pulse. The clock period of this processed input clock pulse is the third clock period. For example, in the example above where the first clock period is 0.9s and the second clock period is 1s, the input clock pulse can be used as the processed input clock pulse. In the example above where the first clock period is 1.8s and the second clock period is 1s, the input clock pulse needs to be divided. In the example above, where the first clock period is 0.48s and the second clock period is 1s, the input clock pulses need to be frequency-multiplied. Then, since the local clock signal is being synchronized to the input clock signal, the deviation of the processed input clock signal relative to the local clock signal needs to be calculated and allocated. After this processing, the time difference between the third clock period and the first clock period is very small, so the first clock period can be directly adjusted to the third clock period. Additionally, the phase difference between the processed input clock signal's clock pulse and the local clock signal's clock pulse within one clock period is calculated, and the phase of the local clock signal's clock pulse within one clock period is adjusted based on this phase difference to complete the clock synchronization process. For example, in the example above where the first clock period is 0.9s and the second clock period is 1s, the time difference is 0.1s, so the local clock signal can be directly adjusted to transmit a clock pulse once per second. Then, the phase of the local clock signal is adjusted according to the phase of the input clock signal.

[0062] It should be noted that the clock synchronization process also requires time format conversion according to the clock standard. For example, given that the LiDAR, MCU, SOC, and other controllers in the vehicle support pulse-of-seconds (PPS) signals and GPRMC format time information from serial-port-based GPS, these can be used as the clock source standard. In this case, in addition to processing the time difference and phase difference mentioned above, both the local clock signal and the input clock signal need to be converted to GPRMC format.

[0063] The above-described vehicle-mounted multi-sensor synchronization method in this application determines a set acquisition period and a set acquisition phase with synchronization characteristics for each sensor, and uses these set acquisition periods and set acquisition phases to uniformly control the emission of acquisition trigger pulses of each sensor, ensuring that each sensor can be triggered synchronously to acquire data. This solves the problem of poor data synchronization caused by each sensor independently controlling data acquisition, and realizes the synchronization of data from the data acquisition source of each sensor. It improves the accuracy of data synchronization of multiple sensors of different types and controlled by different controllers, thereby greatly improving the accuracy and consistency of vehicle perception and positioning technologies.

[0064] It should be noted that the terminology used in this application is for the purpose of describing specific embodiments only and is not intended to limit the scope of this application. As shown in the specification and claims of this application, unless the context clearly indicates otherwise, words such as "a," "an," "an," and / or "the" are not specifically singular and may include plural. The term "and / or" includes any one and all combinations of one or more of the associated listed items. Relational terms such as "first," "second," and "third" are used only to distinguish one entity or operation from another entity or operation and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element.

[0065] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A vehicle-mounted multi-sensor synchronization method, characterized by, The method comprises the following steps: Based on the set acquisition period and the set acquisition phase corresponding to each sensor, the local clock signal is respectively divided or multiplied to generate a collection trigger pulse suitable for the corresponding sensor; wherein the set acquisition periods have a multiple relationship, and the set acquisition phases of the sensors arranged at the same position are the same; The collection trigger pulse of each sensor is sent to the corresponding sensor to trigger the synchronous data collection of each sensor; Before the step of generating a collection trigger pulse suitable for the corresponding sensor by dividing or multiplying the local clock signal based on the set acquisition period and the set acquisition phase corresponding to each sensor, the method further comprises the following steps: Synchronizing the local clock signal based on the input clock signal of the external clock source; After the clock synchronization, the method of generating a collection trigger pulse suitable for the corresponding sensor by dividing or multiplying the local clock signal based on the set acquisition period and the set acquisition phase corresponding to each sensor comprises the following steps: For any sensor: Based on the set acquisition period of the sensor, determine the number of emission times of the collection trigger pulse in the third clock period; wherein the third clock period is the clock period of the synchronized local clock signal; Based on the number of emission times, allocate the time difference between the third clock period and the first clock period of the local clock signal to the emission process of each collection trigger pulse in the third clock period to adjust the emission time of each collection trigger pulse; Based on the set acquisition phase of the sensor, adjust the emission phase of each collection trigger pulse, and based on the emission time and the emission phase, divide or multiply the synchronized local clock signal to generate each collection trigger pulse of the sensor.

2. The method of claim 1, wherein, The step of synchronizing the local clock signal based on the input clock signal of the external clock source comprises the following steps: Obtain the input clock signal of at least two external clock sources; Based on the priority of each external clock source and each input clock signal, determine a target clock source; Synchronize the local clock signal based on the input clock signal of the target clock source.

3. The method of claim 2, wherein, The step of synchronizing the local clock signal based on the input clock signal of the target clock source comprises the following steps: Based on the first clock period of the local clock signal and the second clock period of the input clock signal, divide or multiply the input clock signal to generate a processed input clock signal; Based on the time difference between the third clock period of the processed input clock signal and the first clock period, and the phase difference between the processed input clock signal and the local clock signal in one clock period, respectively adjust the period and phase of the local clock signal to complete the synchronization of the local clock signal.

4. The method of claim 1, wherein, Each sensor is a front-view camera, a right-view camera, a rear-view camera, a left-view camera, a front-view radar, a right-view radar, a rear-view radar, a left-view radar, and a positioning sensor. The set collection periods of the cameras are the same, the set collection periods of the radars are the same, and the set collection period of the positioning sensor is an integer multiple of the set collection period of the radars, and the set collection period of the radars is an integer multiple of the set collection period of the cameras; The set collection phases of the cameras and the radars in the same direction are the same.

5. The method of claim 4, wherein, The set collection phases of the cameras and the radars in the same direction are the same, including: The set collection phases of the front-view cameras and the front-view radars are both 0°; The set collection phases of the right-view cameras and the right-view radars are both 90°; The set collection phases of the rear-view cameras and the rear-view radars are both 180°; The set collection phases of the left-view cameras and the left-view radars are both 270°.

6. An in-vehicle multi-sensor synchronization apparatus characterized by comprising: Including: A collection trigger pulse generation module is configured to perform frequency division or frequency multiplication processing on a local clock signal based on a set collection period and a set collection phase corresponding to each sensor, to generate a collection trigger pulse adapted to the corresponding sensor; wherein the set collection periods have a multiple relationship, and the set collection phases of the sensors arranged at the same position are the same; A sensor synchronization module is configured to send the collection trigger pulse of each sensor to the corresponding sensor to trigger the sensors to synchronously collect data; The device further includes a clock synchronization module configured to synchronize the local clock signal based on an input clock signal of an external clock source before the frequency division or frequency multiplication processing on the local clock signal based on a set collection period and a set collection phase corresponding to each sensor, to generate a collection trigger pulse adapted to the corresponding sensor; The collection trigger pulse generation module is specifically configured to, before and after clock synchronization, for any sensor: determine the number of times of emission of the collection trigger pulse in a third clock period based on the set collection period of the sensor; wherein the third clock period is the clock period of the synchronized local clock signal; allocate the time difference between the third clock period and the first clock period of the local clock signal to the emission process of each collection trigger pulse in the third clock period based on the number of times of emission, to adjust the emission time of each collection trigger pulse; adjust the emission phase of each collection trigger pulse based on the set collection phase of the sensor, and perform frequency division or frequency multiplication processing on the synchronized local clock signal based on the emission time and the emission phase, to generate each collection trigger pulse of the sensor.

7. An electronic device, comprising: The electronic device includes: a processor and a memory; the processor is configured to execute the steps of the method according to any one of claims 1 to 5 by invoking programs or instructions stored in the memory.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores programs or instructions, which cause the computer to execute the steps of the method according to any one of claims 1 to 5.

Images