Time synchronization method for multiple sensors

Abstract

The embodiment of the invention discloses a multi-sensor time synchronization method. The method comprises the following steps: a master device sends a first message and a second message to a slave device to complete first communication between the master device and the slave device; obtaining a first equation based on the first communication; the master device sends a third message and a fourth message to the slave device to complete second communication between the master device and the slave device; obtaining a second equation based on the second communication; calculating network delay and clock skew according to the first equation and the second equation; performing clock correction on the master device and the slave device according to the network delay and the clock skew so as to complete time synchronization of the multiple sensors; the master device and the slave device are sensors, and time synchronization is carried out between the master device and the slave device based on a PTP protocol. The time synchronization method does not depend on GPS signals any more. Moreover, the time synchronization method can realize accurate time synchronization among the plurality of sensors, and provides a basis for subsequent fusion of the plurality of sensors and calibration of a coordinate system.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-sensor fusion, and in particular to a multi-sensor time synchronization method. Background Art

[0002] In the field of autonomous robots, being able to freely manipulate robots in complex and unknown environments is an extremely important challenge. Accurately estimating one's own posture and position and perceiving the environment are necessary prerequisites for completing this challenge. On this basis, the robot can perform a number of automated tasks including navigation, planning, control, and autonomous decision-making. With the development of satellite positioning technology, the Global Navigation Satellite System (GNSS) can solve the problem of autonomous positioning in most unknown scenarios. However, the positioning accuracy of GNSS is heavily dependent on the quality of satellite signals. When the environment in which the robot is located, such as an urban canyon, blocks or interferes with the satellite signal, the positioning accuracy of the system will be severely reduced or even fail. In addition, the positioning data obtained through GNSS has a low frequency and is difficult to meet the positioning needs of the robot. For this reason, the Inertial Navigation System (INS) that uses an Inertial Measurement Unit (IMU) to assist GNSS positioning has gradually become the main solution. INS uses the high-frequency inertial measurement output of IMU to maintain the positioning accuracy during GNSS failure and increase the output frequency of positioning. Complete reliance on IMU measurements for dead reckoning (DR) will lead to divergence in positioning results, so the introduction of inertial measurement units still does not solve the defects of the global positioning system.

[0003] In recent years, with the development of sensor technology, the simultaneous localization and mapping (SLAM) technology based on monocular cameras or lidar has become the mainstream solution to the problem of autonomous positioning in unmanned systems. The SLAM algorithm is an advanced technology that analyzes the measurement data of the sensors equipped by the unmanned system to achieve the robot's self-positioning and accurate construction of the surrounding environment map in a complex and unplanned environment. This process greatly enhances the robot's ability to recognize its own state and the external environment, and provides the necessary information basis for the robot's navigation and task execution. The principle is to use the data obtained by the sensor at the current moment to associate with the previous data in the global map, so as to solve the position and posture changes during this period of movement, and then fuse the latest sensor data into the global map through the posture.

[0004] Thanks to the fusion of multiple sensors, SLAM algorithms can pay more attention to the geometric features in the scene while accurately estimating the system state. The integration of high-dimensional geometric features is crucial to improving the performance of SLAM systems. First, compared with conventional point features, higher-dimensional features carry richer structural information, such as lines, planes, and surfaces, which not only speeds up the map construction process, but also provides deeper support for the robot's self-navigation in complex environments. Secondly, in different application scenarios, such as corridors, auditoriums, or underground parking lots, the fusion of high-dimensional features can significantly improve the system's adaptability to the environment. The number of point features in these environments may be insufficient, while line and plane features are more prominent, thereby enhancing the accuracy and scope of application of the system. Finally, different sensors have different sensitivities to different types of geometric features. For example, cameras are better at capturing point features, but it is more difficult to identify planes; on the contrary, lidar can easily identify plane features in three-dimensional space, but has limitations in distinguishing point features. Therefore, by integrating different geometric features, we can make full use of the advantages of various sensors to provide more comprehensive data support for SLAM and enhance its performance and robustness in diverse environments.

[0005] Multi-sensor fusion also brings many disadvantages. The asynchrony between time and space between sensors is the most serious problem in the fusion process. Any lack of synchronization will have an irreparable impact on the system. The loose coupling solution used in fusion cannot fully utilize the characteristics of all sensors, while the tightly coupled solution is too dependent on the quality of sensor data. The increase in the number of sensors will increase the load on the computing platform, and the asynchronous communication between sensors will make the system unable to receive data synchronously.

[0006] In a multi-sensor fusion SLAM system, since sensors usually have independent clock sources and coordinate systems, but uncalibrated clock sources will produce clock drift over time, the data of sensors that are not synchronized in time and space cannot truly reflect the motion state of the system, which makes sensor fusion impossible. In order to ensure that the data of each sensor is true and valid, it is indispensable to synchronize the time and space of each sensor.

[0007] Each sensor usually has a built-in clock, but each clock will drift over time, resulting in unstable time differences between the clocks, so it is necessary to unify the clock source used by each sensor. The common way to unify the clock source is usually: GPS-based "PPS+GPRMC", specifically:

[0008] A successfully calibrated GNSS receiver will generate a synchronization pulse signal once per second (1Hz), namely PPS (Pulse Per Second), with a pulse width usually ranging from 5 milliseconds to 100 milliseconds; at the same time, the receiver will also send time synchronization information in the GPRMC format through the serial port. When the sensor device receives the PPS pulse signal, it will reset the system time (milliseconds and smaller units) to zero based on the built-in atomic clock, and restart the timing with this moment as the new starting point. Subsequently, when the device receives the GPRMC data message, it will extract the UTC time information from it and synchronize it to the sensor clock, thus completing a time synchronization. Obviously, this timing method is heavily dependent on GPS signals. Summary of the invention

[0009] In view of the defects of the prior art, an object of the embodiments of the present invention is to provide a time synchronization method for multiple sensors.

[0010] To achieve the above object, an embodiment of the present invention provides a multi-sensor time synchronization method, including:

[0011] The master device sends a first message and a second message to the slave device to complete the first communication between the master device and the slave device;

[0012] Based on the first communication, a first formula is obtained;

[0013] The master device sends a third message and a fourth message to the slave device to complete the second communication between the master device and the slave device;

[0014] Based on the second communication, a second formula is obtained;

[0015] Calculate network delay and clock deviation according to the first formula and the second formula;

[0016] Performing clock correction on the master device and the slave device according to the network delay and the clock deviation to complete time synchronization of multiple sensors;

[0017] The master device and the slave device are sensors, and time synchronization is performed between the master device and the slave device based on the PTP protocol.

[0018] The first message is a Sync message, the second message is a Follow_Up message, the third message is a Delay_Req message, and the fourth message is a Delay_Resp message.

[0019] As a specific implementation of the present application, the first communication between the master device and the slave device is specifically as follows:

[0020] The master device sends a synchronization signal, which is a Sync message, and marks the time point of sending the synchronization signal as t1;

[0021] The slave device receives the synchronization information and records the receiving time as t2;

[0022] The master device sends a Follow_Up message, where the Follow_Up message includes time information of t1;

[0023] The slave device receives the Follow_Up message and extracts t1 therefrom.

[0024] As a specific implementation of the present application, the second communication between the master device and the slave device is specifically as follows:

[0025] The slave device sends a delay request to the master device, the delay request is a Delay_Req message, and synchronously records the time point of sending the delay request as t3;

[0026] The master device receives the delay request and records the receiving time as t4;

[0027] The master device transmits the recorded time t4 back to the slave device via a delayed response; the delayed response is a Delay_Res message;

[0028] The slave device receives the Delay_Res message, parses it and obtains t4.

[0029] Among them, the first calculation formula is: t1+network delay+clock deviation=t2; the second calculation formula is: t3+network delay-clock deviation=t4.

[0030] By implementing the multi-sensor time synchronization method provided in the embodiment of the present invention, the master and slave devices perform time synchronization based on the PTP protocol, and the time synchronization method no longer relies on GPS signals. Moreover, the time synchronization method can achieve accurate time synchronization between multiple sensors, providing a basis for subsequent multi-sensor fusion and coordinate system calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the specific implementation of the present invention or the technical solution in the prior art, the drawings required for use in the specific implementation or the description of the prior art will be briefly introduced below.

[0032] Figure 1 It is a PTP time synchronization principle diagram provided by an embodiment of the present invention;

[0033] Figure 2 It is a flow chart of a multi-sensor time synchronization method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0036] Laser radar, also known as Light Detection and Ranging (LiDAR) system, is mainly composed of laser transmitter, laser receiver and signal processing unit. According to its working principle, it is mainly divided into two categories: triangulation method and time of flight (ToF) method.

[0037] In triangulation, light beams from a laser transmitter are reflected back from an object and then captured by laser receivers at different locations. By analyzing the geometric relationship of these light beams, the system is able to calculate the actual distance to the object. LiDAR based on the triangulation principle is relatively low in cost, but lacks accuracy in long-distance measurement and is usually used in single-line LiDAR for indoor positioning and distance measurement.

[0038] The time-of-flight lidar technology is further divided into two categories: pulsed and phased. Pulsed lidar includes a timer, a laser transmitter, and a laser receiver. Its working principle is intuitive and simple, can provide high-precision measurement results, and has excellent resistance to environmental interference.

[0039] Phase-type LiDAR can provide high angular resolution and more accurate distance measurement, but it is sensitive to the environment and has a high cost. The principle of phase-type LiDAR is: by modulating the light intensity of the emitted light wave, LiDAR can indirectly estimate the time difference by analyzing the phase difference between the emitted and received light wave waveforms. Compared with pulse-type LiDAR, this method eliminates the need for direct timing.

[0040] The ranging model of LiDAR is very simple. It only measures the distance from a certain point in space to the LiDAR itself. The ranging model of a single laser probe is also called the RAE (Range Azimuth Elevation) model.

[0041] The inventive concept of the present invention is: a high-precision time synchronization protocol PTP based on the network is used for time synchronization between master and slave devices. The master and slave devices are multiple sensors of different types, such as pulsed laser radar, phase laser radar, etc. PTP exchanges synchronization messages between the master and slave devices and records the message sending time, thereby calculating the network transmission delay and the clock deviation between the master and slave devices. PTP defines four synchronization messages: Sync, Follow_Up, Delay_Req, and Delay_Resp.

[0042] Please refer to Figure 1 and Figure 2 , is a multi-sensor time synchronization method provided by an embodiment of the present invention, comprising the following steps:

[0043] S1, the master device sends a first message and a second message to the slave device to complete the first communication between the master device and the slave device.

[0044] Specifically, the master device (master port) sends a synchronization signal (Sync message) and marks the time point of sending the signal as t1. When this signal is received by the slave device (slave port), the slave device (slave port) records the reception time as t2. Subsequently, the master device (master port) sends a Follow_Up message to the slave device (slave port), which contains the t1 time information. The slave device (slave port) receives the Follow_Up message, completing the first communication between the master and slave devices, and extracts the t1 time information from the Follow_Up message.

[0045] S2: Based on the first communication, obtain a first formula.

[0046] Specifically, the first calculation formula is: t1+network delay+clock deviation=t2.

[0047] S3, the master device sends a third message and a fourth message to the slave device to complete the second communication between the master device and the slave device.

[0048] Specifically, the slave device (slave port) sends a delay request (Delay_Req message) to the master device (master port), and synchronously records the time point when the request is sent as t3. When the master device (master port) receives the request, the recorded receiving time is t4. Then, the master device (master port) returns the recorded t4 time to the slave device (slave port) through a delay response (Delay_Res message). When the slave device (slave port) receives the message, it will parse and obtain the t4 time information.

[0049] S4, obtaining a second formula based on the second communication.

[0050] Specifically, the second calculation formula: t3+network delay-clock deviation=t4.

[0051] S5, calculating the network delay and clock deviation according to the first formula and the second formula.

[0052] It can be calculated that network delay = [(t2-t1)+(t4-t1)] / 2 and clock deviation = [(t2-t1)-(t4-t3)] / 2.

[0053] S6, performing clock correction on the master device and the slave device according to the network delay and the clock deviation to complete time synchronization of multiple sensors.

[0054] From the above description, it can be known that the multi-sensor time synchronization method provided in the embodiment of the present invention is implemented, and the master and slave devices are synchronized based on the PTP protocol, and the time synchronization method no longer relies on GPS signals. Moreover, the time synchronization method can achieve accurate time synchronization between multiple sensors, providing a basis for the subsequent fusion of multiple sensors and calibration of the coordinate system.

[0055] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0056] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0057] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0058] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0059] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A multi-sensor time synchronization method, characterized in that: include: The master device sends a first message and a second message to the slave device to complete the first communication between the master device and the slave device; Based on the first communication, a first formula is obtained; The master device sends a third message and a fourth message to the slave device to complete the second communication between the master device and the slave device; Based on the second communication, a second formula is obtained; Calculate network delay and clock deviation according to the first formula and the second formula; Performing clock correction on the master device and the slave device according to the network delay and the clock deviation to complete time synchronization of multiple sensors; The master device and the slave device are sensors, and time synchronization is performed between the master device and the slave device based on the PTP protocol.

2. The multi-sensor time synchronization method according to claim 1, characterized in that: The first message is a Sync message, and the second message is a Follow_Up message.

3. The multi-sensor time synchronization method according to claim 2, characterized in that: The first communication between the master device and the slave device is as follows: The master device sends a synchronization signal, which is a Sync message, and marks the time point of sending the synchronization signal as t1; The slave device receives the synchronization information and records the receiving time as t2; The master device sends a Follow_Up message, where the Follow_Up message includes time information of t1; The slave device receives the Follow_Up message and extracts t1 therefrom.

4. The multi-sensor time synchronization method according to claim 3, characterized in that: The first calculation formula is: t1+network delay+clock deviation=t2.

5. The multi-sensor time synchronization method according to claim 4, characterized in that: The third message is a Delay_Req message, and the fourth message is a Delay_Resp message.

6. The multi-sensor time synchronization method according to claim 5, characterized in that: The second communication between the master device and the slave device is as follows: The slave device sends a delay request to the master device, the delay request is a Delay_Req message, and synchronously records the time point of sending the delay request as t3; The master device receives the delay request and records the receiving time as t4; The master device transmits the recorded time t4 back to the slave device via a delayed response; the delayed response is a Delay_Res message; The slave device receives the Delay_Res message, parses it and obtains t4.

7. The multi-sensor time synchronization method according to claim 6, characterized in that: The second calculation formula is: t3+network delay-clock deviation=t4.

8. The multi-sensor time synchronization method according to claim 7, characterized in that: Network delay = [(t2-t1)+(t4-t1)] / 2, clock deviation = [(t2-t1)-(t4-t3)] / 2.

Images