Method for detecting time delay of lidar point cloud, detection system and lidar

Abstract

The application provides a laser radar point cloud delay detection method, a detection system and a laser radar. The detection method comprises: performing a point cloud delay detection operation to obtain a data packet point cloud delay, wherein the data packet point cloud delay at least comprises a communication link delay between the laser radar and an external terminal and a data packet total delay. The point cloud delay detection operation can obtain the data packet point cloud delay at least comprising the communication link delay and the data packet total delay, and the obtained data packet point cloud delay has higher precision, which is beneficial to improvement of laser radar performance.

Description

Technical Field

[0001] This invention relates to the field of lidar, and in particular to a method, system and lidar for detecting point cloud delay in lidar. Background Technology

[0002] LiDAR is a commonly used ranging sensor, characterized by its long detection range, high resolution, and low susceptibility to environmental interference. It is widely used in fields such as autonomous driving, intelligent robots, and drones. In recent years, with the rapid development of autonomous driving technology, LiDAR, as a core sensor for distance perception, has become indispensable.

[0003] Autonomous driving reaction time falls under the category of system response time. To achieve a braking distance of no more than 30cm from 100km / h, the overall system response time must not exceed 10 milliseconds. To build a safer traffic environment, autonomous driving must enter an era where "milliseconds matter," and low latency is a prerequisite for autonomous driving.

[0004] The time required for autonomous driving operations under current technology includes multiple aspects such as data acquisition, analysis, algorithm decision-making, and machine transmission, each of which takes far more than 1 millisecond. The latency caused by communication accounts for a very small proportion of the entire time chain.

[0005] Therefore, in autonomous driving operations, low end-to-end latency depends more on improvements in sensors, processors, algorithms, and machine transmission. LiDAR point cloud latency, which refers to the internal data transmission time of the sensor, is a crucial indicator of whether LiDAR performance meets standards. If the LiDAR point cloud latency does not meet autonomous driving standards, it will increase the autonomous driving response time, thereby increasing the risks associated with autonomous driving.

[0006] However, there is currently no effective method or system for detecting the latency of LiDAR point clouds. Therefore, it is necessary to accurately detect the latency of LiDAR point clouds to improve the performance of LiDAR systems and meet the low latency standards required for autonomous driving. Summary of the Invention

[0007] The problem solved by this invention is to accurately obtain the point cloud delay of lidar.

[0008] To address the above problems, this invention provides a method for detecting the delay of lidar point clouds, comprising:

[0009] A point cloud delay detection operation is performed to obtain the data packet point cloud delay, which includes at least the communication link delay between the lidar and the external terminal and the total data packet delay.

[0010] Optionally, the point cloud delay detection operation includes: obtaining the communication link delay between the lidar and the external terminal; performing data packet transmission and reception to determine the total delay of the data packets transmitted between the lidar and the external terminal; and obtaining the data packet point cloud delay based on the communication link delay and the total delay of the data packets.

[0011] Optionally, before obtaining the communication link delay between the lidar and the external terminal, the point cloud delay detection operation further includes: calibrating the lidar clock and the external terminal clock.

[0012] Optionally, the step of calibrating the lidar clock and the external terminal clock includes: synchronizing the lidar clock and the external terminal clock; or, obtaining time synchronization accuracy based on the calibrated lidar clock and the external terminal clock.

[0013] Optionally, the step of obtaining the communication link delay between the lidar and the external terminal includes: obtaining the communication link delay based on the mean and standard deviation of the communication link delay between the lidar and the external terminal obtained multiple times.

[0014] Optionally, multiple point cloud latency detection operations are performed to obtain multiple data packet point cloud latency; the detection method further includes: obtaining the point cloud latency of the lidar based on the multiple data packet point cloud latency; and determining whether the lidar meets the autonomous driving standard based on the point cloud latency of the lidar.

[0015] Optionally, in the step of obtaining the point cloud delay of the lidar, at least one of the maximum value, average value, and median of the point cloud delay of the plurality of data packets is used as the point cloud delay of the lidar.

[0016] Optionally, the steps of sending and receiving data packets to determine the total delay of the data packets include: obtaining the packet reception time based on the time when the external terminal obtains the data packets; parsing the data packets to obtain the point cloud time; and determining the total delay of the data packets based on the packet reception time and the point cloud time.

[0017] Optionally, the data packet includes: the transmission time of the trigger signal; the step of parsing the data packet to obtain the point cloud time includes: parsing the data packet to obtain the transmission time of the trigger signal as the point cloud time.

[0018] Optionally, the step of parsing the data packet to obtain the point cloud time includes: obtaining the data packet filling time based on the interval of the control module sending the trigger signal, the buffer waiting time, and the data filling time; and obtaining the point cloud time based on the data packet filling time and a preset lead time.

[0019] Optionally, the lead amount can be set according to the number of laser pulses corresponding to the data of each point obtained from the lidar point cloud.

[0020] Optionally, the control module and the transceiver module communicate wirelessly; in the step of obtaining the data packet filling time, the data packet filling time is obtained based on the interval time of the control module sending the trigger signal, the buffer waiting time, the data filling time, and the communication time between the control module and the transceiver module.

[0021] Accordingly, a lidar point cloud delay detection system includes: the lidar point cloud delay detection system is suitable for implementing the detection method of the present invention.

[0022] In addition, the present invention also provides a lidar, comprising:

[0023] The system includes a transmitting module and a receiving module; the echo formed by the beam emitted by the transmitting module after being reflected by the target is received by the receiving module to obtain a data packet for generating a point cloud; the lidar also includes a point cloud delay detection module, which is suitable for implementing the detection method of the present invention.

[0024] Furthermore, the present invention also provides a lidar, comprising:

[0025] The system includes a transmitting module and a receiving module. The beam emitted by the transmitting module is reflected by the target, and the resulting echo is received by the receiving module to obtain a data packet for generating a point cloud. The lidar also includes a point cloud delay detection module, which is adapted to perform point cloud delay detection operations to obtain the data packet point cloud delay. The data packet point cloud delay includes at least the communication link delay between the lidar and the external terminal and the total delay of the data packet.

[0026] Optionally, the point cloud latency detection module includes: a communication detection unit, which is adapted to obtain the communication link latency between the lidar and the external terminal; a transmit / receive detection unit, which is adapted to perform data packet transmission and reception to determine the total data packet latency; and a processing unit, which is adapted to obtain the data packet point cloud latency based on the communication link latency and the total data packet latency.

[0027] Optionally, the point cloud delay detection module further includes a clock calibration unit, which is suitable for calibrating the lidar clock and the external terminal clock.

[0028] Optionally, the clock calibration unit is adapted to synchronize the lidar clock and the external terminal clock; or, the clock calibration unit is adapted to obtain time synchronization accuracy based on the calibrated lidar clock and the external terminal clock.

[0029] Optionally, the point cloud latency detection module performs multiple point cloud latency detection operations to obtain multiple data packet point cloud latencies; the lidar further includes: a statistics module, which is adapted to obtain the point cloud latency of the lidar based on the multiple data packet point cloud latencies; and an evaluation module, which is adapted to determine whether the lidar meets the autonomous driving standard based on the point cloud latency of the lidar.

[0030] Optionally, the statistics module is adapted to use at least one of the maximum, average, and median of the point cloud delay of the plurality of data packets as the point cloud delay of the lidar.

[0031] Optionally, the communication detection unit is adapted to obtain the communication link delay based on the mean and standard deviation of the communication link delay between the lidar and the external terminal obtained multiple times.

[0032] Optionally, the transceiver detection unit includes: an extractor adapted to obtain the packet reception time based on the time when the external terminal obtains the data packet; a parser adapted to parse the data packet to obtain the point cloud time; and a delay processor adapted to determine the total delay of the data packet based on the packet reception time and the point cloud time.

[0033] Optionally, the data packet includes: the transmission time of the trigger signal; the parser parses the data packet to obtain the transmission time of the trigger signal as the point cloud time.

[0034] Optionally, the parser has a preset lead time; the parser obtains the data packet filling time based on the interval of the trigger signal sent by the control module, the buffer waiting time, and the data filling time; the parser also obtains the point cloud time based on the data packet filling time and the preset lead time.

[0035] Optionally, the lead amount can be set according to the number of laser pulses corresponding to the data of each point obtained from the lidar point cloud.

[0036] Optionally, the control module and the transceiver module communicate wirelessly; the parser obtains the data packet filling time based on the interval between the control module sending trigger signals, the buffer waiting time, the data filling time, and the communication time between the control module and the transceiver module.

[0037] Compared with the prior art, the technical solution of the present invention has the following advantages:

[0038] The point cloud delay detection operation can obtain the data packet point cloud delay, which includes at least the communication link delay and the total data packet delay. The obtained data packet point cloud delay has higher accuracy, which is beneficial to the improvement of LiDAR performance. It enables the LiDAR point cloud delay to meet the autonomous driving standards, reduces the autonomous driving reaction time, and reduces the risks of autonomous driving. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating an embodiment of the lidar point cloud delay detection method of the present invention;

[0040] Figure 2 yes Figure 1 The flowchart shown is a schematic diagram of the point cloud delay detection operation steps in the embodiment of the lidar point cloud delay detection method.

[0041] Figure 3 yes Figure 2 The flowchart shown in this embodiment of the lidar point cloud delay detection method illustrates the steps for determining the total delay of data packets through data packet transmission and reception.

[0042] Figure 4 yes Figure 1 The diagram shown is a schematic representation of point cloud data generation and data packet transmission in a lidar point cloud delay detection method embodiment.

[0043] Figure 5 yes Figure 2 The diagram shows a timeline of data packet transmission in an embodiment of the method for detecting point cloud delay using lidar.

[0044] Figure 6 This is a functional block diagram of an embodiment of the lidar of the present invention;

[0045] Figure 7 yes Figure 6 The diagram shows the functional block diagram of the transceiver detection unit in the lidar embodiment shown. Detailed Implementation

[0046] As can be seen from the background technology, existing technologies lack effective detection methods and systems for detecting the latency of LiDAR point clouds, which makes it difficult to characterize the performance of LiDAR systems and fails to meet the LiDAR latency standards for autonomous driving.

[0047] To address the aforementioned technical problem, the present invention provides a method for detecting point cloud latency of a lidar system, comprising: performing a point cloud latency detection operation to obtain data packet point cloud latency, wherein the data packet point cloud latency includes at least: the communication link latency between the lidar system and an external terminal and the total latency of the data packets.

[0048] In the technical solution of this invention, the point cloud delay detection operation can obtain the data packet point cloud delay, including the communication link delay and the total data packet delay. The obtained data packet point cloud delay has higher accuracy, which is beneficial to the improvement of LiDAR performance. This makes the LiDAR point cloud delay meet the autonomous driving standards, reduces the autonomous driving reaction time, and reduces the risk of autonomous driving.

[0049] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] refer to Figure 1 The diagram shows a flowchart of an embodiment of the lidar point cloud delay detection method of the present invention.

[0051] like Figure 1 As shown, the method for detecting the point cloud delay of the lidar includes: firstly, performing step S100 to perform a point cloud delay detection operation to obtain the data packet point cloud delay, wherein the data packet point cloud delay includes at least the communication link delay between the lidar and the external terminal and the total data packet delay.

[0052] The point cloud delay detection operation can obtain the data packet point cloud delay, including the communication link delay and the total data packet delay. The obtained data packet point cloud delay has higher accuracy, which is beneficial to the improvement of lidar performance.

[0053] It should be noted that the external terminal can be an external detection terminal. For vehicle-mounted LiDAR, the external terminal can also be the vehicle's control system, such as an autonomous driving control system. For vehicle-mounted LiDAR, the data packet is a UDP data packet.

[0054] Reference Figure 2 , showed Figure 1 The diagram shows a flowchart illustrating the point cloud delay detection steps in an embodiment of the lidar point cloud delay detection method.

[0055] like Figure 2 As shown, the point cloud delay detection operation includes: first, executing step S110 to obtain the communication link delay between the lidar and the external terminal; then executing step S120 to perform data packet transmission and reception to determine the total data packet delay between the lidar and the external terminal; finally executing step S130 to obtain the data packet point cloud delay based on the communication link delay and the total data packet delay.

[0056] Data packet point cloud delay refers to the time difference between the moment a data packet containing point cloud data is sent and the moment the trigger signal that generates the point cloud data is sent. Obtaining the data packet point cloud delay effectively characterizes the point cloud delay performance of the lidar, which is beneficial for improving lidar performance. The trigger signal is suitable for controlling the lidar to emit a beam for detection.

[0057] It should be noted that the point cloud delay detection operation further includes: before executing step S110 to obtain the communication link delay between the lidar and the external terminal, executing step S101 to calibrate the lidar clock and the external terminal clock.

[0058] In some embodiments of the present invention, the step of calibrating the lidar clock and the external terminal clock includes: obtaining a time synchronization accuracy based on the calibrated lidar clock and the external terminal clock to calibrate the lidar clock and the external clock. Wherein, the time synchronization accuracy is an accuracy parameter of the lidar clock, suitable for characterizing the time difference between the lidar clock and the external terminal clock.

[0059] In some embodiments of the present invention, the step of obtaining time synchronization accuracy includes: first, based on the master clock, obtaining the time synchronization parameters of the lidar clock and the external terminal clock within a preset time period according to the lidar clock and the external terminal clock; and obtaining the time synchronization accuracy according to the time synchronization parameters of the lidar clock and the external terminal clock.

[0060] Specifically, when the lidar is an automotive lidar, the master clock is a PTP master clock (i.e., PTPMaster). The mean and standard deviation of the delay between the lidar clock and the PTP master clock within a preset time period, such as 30 consecutive minutes, are obtained as the lidar clock's time synchronization parameter 1. The mean and standard deviation of the delay between the external terminal clock and the PTP master clock within the preset time period are obtained as the external terminal clock's time synchronization parameter 2. The square root of the sum of the means and the square of the standard deviations in time synchronization parameters 1 and 2 is taken as the time synchronization accuracy T_ptp.

[0061] It should be noted that in other embodiments of the present invention, the lidar clock and the external terminal clock can also be directly synchronized. When the lidar clock and the external terminal clock are fully synchronized, the time synchronization accuracy T_ptp = 0.

[0062] Continue to refer to Figure 2 After calibrating the lidar clock and the external terminal clock, the point cloud delay detection operation includes: executing step S110 to obtain the communication link delay between the lidar and the external terminal.

[0063] In some embodiments of the present invention, the step of performing step S110 to obtain the communication link delay between the lidar and the external terminal includes: obtaining the communication link delay based on the mean and standard deviation of the communication link delay between the lidar and the external terminal obtained multiple times.

[0064] Specifically, after the lidar is powered on and connected to the external terminal, the communication link delay T_trans (also known as network transmission delay) between the lidar and the external terminal can be obtained by checking the lidar's IP address via network ping. The communication link delay T_trans can be directly obtained by checking the lidar's network port. In some embodiments, the mean and standard deviation of N consecutive network checks can be used as the communication link delay T_trans.

[0065] The point cloud delay detection operation further includes: performing step S120 to send and receive data packets to determine the total delay of data packets between the lidar and the external terminal.

[0066] Reference Figure 3 , showed Figure 2 The flowchart shown in this embodiment of the lidar point cloud delay detection method illustrates the steps of sending and receiving data packets to determine the total delay of the data packets.

[0067] In some embodiments of the present invention, the step of performing step S120, which involves sending and receiving data packets to determine the total delay of data packets between the lidar and the external terminal, includes: step S121, obtaining the packet reception time TC based on the time when the external terminal obtains the data packet; step S122, parsing the data packet to obtain the point cloud time TL; and step S123, determining the total delay of the data packets T_revc based on the packet reception time TC and the point cloud time TL.

[0068] Specifically, in step S121, the time when the external terminal receives the data packet is used as the packet reception time TC. For vehicle-mounted LiDAR, the packet reception time is the UPD packet reception time.

[0069] In step S122, the point cloud time TL is a timestamp of the point cloud data in the data packet. For example, the timestamp of the point cloud data can be the time when the trigger signal that generates the point cloud data is sent, where the timestamp is specifically a timestamp.

[0070] In some embodiments of the present invention, the step of obtaining the point cloud time includes: obtaining the data packet filling time TU based on the interval T0 of the control module sending the trigger signal, the buffer waiting time T4, and the data filling time T6; and obtaining the point cloud time TL based on the data packet filling time TU and the preset advance amount δT.

[0071] Specifically, the data packet has multiple data blocks, and the data packet filling time TU is the time when all data blocks of the data packet are filled. For vehicle-mounted LiDAR, the data packet is a UDP data packet. In some embodiments, the UDP data packet includes at least two data blocks, where the length of each data block is preset, and the amount of data contained in each UDP data packet is also preset. For example, in a LiDAR with 128 channels, the ranging data of each channel is stored sequentially in a UDP data packet, and the data packet is sent after the ranging data is collected.

[0072] Reference Figure 4 , showed Figure 1 The diagram shown is a schematic representation of point cloud data generation and data packet transmission in a lidar point cloud delay detection method embodiment.

[0073] like Figure 4 As shown, the lidar 910 includes a control module 911 and a transmitting module 912, wherein the control module 911 generates a trigger signal to control the transmitting module 912 to generate a transmitted beam 913 for detection; the control module 911 also receives point cloud data generated by the receiving module 917 receiving echo signal 914 to obtain the data packet.

[0074] It should be noted that in some embodiments, the lidar includes an upper compartment and a lower compartment, with the control module 911 located in the lower compartment and the transmitting module 912 located in the upper compartment.

[0075] Specifically, such as Figure 4As shown, after the control module 911 generates a trigger signal and the transmitting module 912 receives the trigger signal, the transmitting module 912 generates and emits a light beam according to a preset emission timing sequence. The processing time of the transmitting module 912 is T2, which is the emission delay of the emitted light beam. Therefore, after the transmitting module 912 receives the trigger signal, after the emission delay T2, the light source in the transmitting module 912 generates and emits an emitted light beam 913. The emitted light beam 913 is reflected by the target object 915 to form an echo signal 914. The time between the emission of the emitted light beam 913 and the reception of the echo signal 914 is the measurement time T3. That is, after the emitted light beam 913 is emitted, after the measurement time T3, the receiving module 917 receives the echo signal 914, thereby generating point cloud data, and then the transmitting module 912 waits to receive the next trigger signal.

[0076] It should be noted that one trigger signal triggers one set of light emission timing. The light emission timing triggered by one trigger signal completes the above-mentioned beam emission and reception process before the next trigger signal is generated, thereby generating one set of point cloud data. That is, the time length occupied by the light emission timing of one trigger signal is less than or equal to the interval time T0 of the trigger signal issued by the control module 911, i.e., T2+T3≤T0. In other words, the interval time T0 of the trigger signal issued by the control module 911 includes: the beam emission delay T2 and the measurement time T3.

[0077] Furthermore, the transmitting module 912 includes a buffer unit 916, and the point cloud data generated by the receiving module 917 receiving the echo signal is first stored in the buffer unit 916; the point cloud data stays in the buffer unit 916 for a period of time T4, i.e., the buffer waiting time T4. In some embodiments of the present invention, the point cloud data stays in the buffer unit 916 until the next set of point cloud data is generated and stored.

[0078] It should be noted that the buffer waiting time T4 is determined according to the settings of the lidar. In some embodiments of the present invention, during the operation of the lidar, a certain amount of point cloud data is always stored in the buffer unit 916. For example, in some embodiments, the buffer unit 916 stores two sets of point cloud data in a 2-in, 2-out manner. That is, in a single-echo lidar, the buffer waiting time T4 is twice the interval T0 between the control module 911 sending the trigger signal, while in a dual-echo lidar, the buffer waiting time T4 is once the interval T0 between the control module 911 sending the trigger signal.

[0079] As mentioned earlier, the data packet includes multiple data blocks, and the data filling time T6 is the time taken to fill the data packet. For vehicle-mounted LiDAR, the UDP data packet has multiple data blocks, and the packet is sent after the multiple data blocks are filled, with a delay of data filling time T6 during this period.

[0080] Specifically, the data filling time T6 is set according to the echo mode of the lidar. For example, for a single-echo lidar, the data filling time T6 is the interval T0 between the control module 911 sending the trigger signal. That is, the point cloud data in the first data block will be sent after a delay of one interval T0 between the control module 911 sending the trigger signal.

[0081] In some embodiments of the present invention, the control module 911 communicates with the transmitting module 912 / receiving module 917 in a wireless mode; in the step of obtaining the data packet filling time TU, the data packet filling time TU is obtained based on the interval T0 of the control module 911 sending the trigger signal, the buffer waiting time T4, the data filling time T6, and the communication time between the control module 911 and the transmitting module 912 / receiving module 917.

[0082] Specifically, the communication time between the control module 911 and the transmitting module 912 / receiving module 917 is limited by the bandwidth of the wireless communication between the control module 911 and the transmitting module 912 / receiving module 917. Therefore, the communication time between the control module 911 and the transmitting module 912 / receiving module 917 includes: the time T1 required for the control module 911 to send a trigger signal to the transmitting module 912, i.e., after the control module 911 in the lower compartment sends the trigger signal, and after time T1, the transmitting module 912 in the upper compartment receives the trigger signal. The communication time between the control module 911 and the transmitting module 912 / receiving module 917 also includes: the time T5 required for the receiving module 917 to send point cloud data to the control module 911, i.e., after the receiving module 917 in the upper compartment sends the point cloud data, and after time T5, the control module 911 in the lower compartment receives the point cloud data.

[0083] It can be seen that from the moment the trigger signal for generating the point cloud data is sent, specifically the point cloud time TL, until the time when the data packet is filled, i.e., the data packet filling time TU, the time elapsed is T0+T4+T6+(T1+T5). Therefore, in the step of obtaining the data packet filling time TU, starting from the point cloud time TL, the data packet filling time TU is obtained according to the interval time T0 of the trigger signal sent by the control module 911, the buffer waiting time T4, and the data filling time T6, i.e., TU=TL+[T0+T4+T6+(T1+T5)].

[0084] It should be noted that in other embodiments of the present invention, the control module and the transmitting module 912 / receiving module 917 communicate in other forms, and the communication time between the control module and the transmitting module 912 / receiving module 917 can also be set in other forms, or even omitted.

[0085] Continue to refer to Figure 3 The lead time δT is an estimated value. The point cloud time TL is obtained by setting the lead time estimate. If the lead time δT is accurately estimated, the obtained point cloud time TL can be as close as possible to the time when the trigger signal that generates the point cloud data is sent.

[0086] The lead time δT can be set empirically. In some embodiments of the present invention, the lead time δT is set according to the echo mode of the lidar, that is, according to the number of laser pulses corresponding to the data of each point obtained in the lidar point cloud. Specifically, it can be set according to the echo mode of the lidar and the interval time of the trigger signal transmission. For example, for a dual-echo lidar, the lead time δT is set to 3 times the interval time of the trigger signal transmission; for a single-echo lidar, the lead time δT is set to 0.

[0087] Therefore, in the step of obtaining the point cloud time TL, the point cloud time TL is obtained based on the difference between the data packet filling time TU and the preset advance amount δT, that is: TL = TU - δT.

[0088] It should be noted that in some embodiments of the present invention, the point cloud time is obtained by the data packet filling time TU and the advance amount δT. However, this approach is only an example. For lidar with a retransmission module, the sending time of the trigger signal that generates the point cloud data is stored in the data packet as a timestamp of the point cloud data, so the point cloud time TL can be obtained directly from the data packet.

[0089] In some embodiments of the present invention, the data packet includes: the transmission time of the trigger signal; the step of parsing the data packet to obtain the point cloud time includes: parsing the data packet to obtain the transmission time of the trigger signal as the point cloud time TL.

[0090] Specifically, the data packet includes the detection angle and the transmission time of the trigger signal corresponding to the detection angle; therefore, the transmission time of the trigger signal corresponding to the detection angle can be read from the data packet based on the detection angle, thereby obtaining the point cloud time TL.

[0091] For vehicle-mounted LiDAR, the transmission time of the trigger signal corresponding to the detection angle is recorded in the UDP data packet as a timestamp of the point cloud data in the data packet; therefore, the timestamp of the point cloud data in the corresponding data packet can be read based on the detection angle as the point cloud time TL.

[0092] Continue to refer to Figure 3 After obtaining the packet reception time TC and the point cloud time TL, step S123 is executed to determine the total delay T_revc of the data packet based on the packet reception time TC and the point cloud time TL.

[0093] Specifically, in the step of determining the total delay T_revc of the data packet, the total delay T_revc of the data packet is determined based on the difference between the packet reception time TC and the point cloud time TL: T_recv=TC-TL=TC-(TU-δT)=TC-[T0+T4+T6+(T1+T5)-δT].

[0094] Continue to refer to Figure 2 After obtaining the communication link delay T_trans and the total data packet delay T_revc, step S130 is executed to obtain the data packet point cloud delay T based on the communication link delay T_trans and the total data packet delay T_revc.

[0095] Reference Figure 5 , showed Figure 2 The diagram shows a timeline of data packet transmission in an embodiment of the lidar point cloud delay detection method.

[0096] As mentioned above, the data packet point cloud delay T refers to the time difference between the time t2 when the data packet of the point cloud data is sent and the time t1 when the trigger signal that generates the point cloud data is sent.

[0097] Therefore, the data packet point cloud delay T is:

[0098] T = t2 - t1

[0099] =t1+[T0+T4+T6+(T1+T5)]+T_send-t1

[0100] =[T0+T4+T6+(T1+T5)]+T_recv-δT-T_ptp-T_trans

[0101] =T_recv-T_trans-T_ptp-δT+[T0+T4+T6+(T1+T5)] (1)

[0102] The packet sending time T_send is the time required to send the packet after it is filled, that is, the point cloud data in the packet is time-stamped and then sent after the packet sending time T_send.

[0103] It should be noted that in some embodiments of the present invention, the data packet includes the transmission time of the trigger signal, that is, the point cloud time TL can be used as the transmission time of the trigger signal.

[0104] Therefore, the data packet point cloud delay T is:

[0105] T = t2 - t1

[0106] =(TC-T_ptp-T_trans)-TL

[0107] =TC-TL-T_ptp-T_trans

[0108] =T_recv-T_ptp-T_trans (2)

[0109] In addition, in some embodiments of the present invention, the lidar clock and the external terminal clock are completely synchronized, so the time synchronization accuracy T_ptp = 0.

[0110] Therefore, in these cases, the data packet point cloud delay T is: T = T_recv - T_trans.

[0111] It should be noted that in some embodiments of the present invention, in the step of performing point cloud delay detection operation to obtain data packet point cloud delay, multiple point cloud delay detection operations are performed to obtain multiple data packet point cloud delays.

[0112] In some embodiments of the present invention, in the step of performing multiple point cloud delay detection operations to obtain multiple data packet point cloud delays, a preset number of point cloud delay detection operations are performed to obtain a preset number of data packet point cloud delays. Specifically, in some embodiments, no less than 10,000 point cloud delay detection operations are performed to obtain more than 10,000 data packet point cloud delays.

[0113] Reference Figure 1 The diagram shows a flowchart of another embodiment of the lidar point cloud delay detection method of the present invention.

[0114] After performing multiple point cloud delay detection operations to obtain multiple data packet point cloud delays, the detection method further includes: executing step S201 to obtain the point cloud delay of the lidar based on the multiple data packet point cloud delays.

[0115] In some embodiments of the present invention, in the step of obtaining the point cloud delay of the lidar, at least one of the maximum value, average value and median of the point cloud delay of the plurality of data packets is used as the point cloud delay of the lidar.

[0116] Specifically, in some embodiments, in the step of obtaining the point cloud delay of the lidar, the maximum value among the multiple data packet point cloud delays is obtained based on the multiple data packet point cloud delays, and is used as the point cloud delay of the lidar, that is, the point cloud delay of the lidar is the longest point cloud delay among the data packet point cloud delays of the lidar.

[0117] In some embodiments of the present invention, the lidar is an in-vehicle lidar for autonomous driving. Therefore, after obtaining the point cloud delay of the lidar, step S202 is executed to determine whether the lidar meets the autonomous driving standard based on the point cloud delay. Specifically, when the point cloud delay of the lidar is less than or equal to the autonomous driving standard, it is determined that the lidar meets the autonomous driving standard.

[0118] For example, the autonomous driving standard is 5 milliseconds, and the point cloud latency of the LiDAR is the longest point cloud latency in the point cloud latency of the LiDAR data packets. That is, if the longest point cloud latency in the point cloud latency of the LiDAR data packets does not exceed 5 milliseconds, then the LiDAR is judged to meet the autonomous driving standard.

[0119] Accordingly, the present invention also provides a lidar point cloud delay detection system, which is suitable for implementing the detection method of the present invention.

[0120] The lidar point cloud delay detection system is suitable for implementing the detection method of the present invention. Therefore, the specific technical solution of the lidar point cloud delay detection system is referred to the embodiments of the aforementioned detection method, and will not be repeated here.

[0121] Furthermore, this invention also provides a lidar, comprising: a transmitting module and a receiving module; the echo formed by the beam emitted by the transmitting module after reflection by the target is received by the receiving module to obtain a data packet for generating a point cloud; the lidar further comprises: a lidar point cloud delay detection system, wherein the lidar point cloud delay detection system is the lidar point cloud delay detection system of this invention. Therefore, the specific technical solution of the lidar point cloud delay detection system refers to the aforementioned embodiment of the lidar point cloud delay detection system, and will not be repeated here.

[0122] refer to Figure 6 The diagram shows a functional block diagram of an embodiment of the lidar of the present invention.

[0123] The lidar includes a transmitting module 912 and a receiving module 917. The echo signal formed by the reflection of the beam emitted by the transmitting module 912 is received to obtain data packets used to generate point clouds. The lidar also includes a point cloud delay detection module 100, which is adapted to perform point cloud delay detection operations to obtain the data packet point cloud delay. The data packet point cloud delay includes at least the communication link delay between the lidar and the external terminal and the total delay of the data packets.

[0124] The point cloud delay detection module 100 can perform the point cloud delay detection operation to obtain the data packet point cloud delay, including the communication link delay and the total data packet delay. The obtained data packet point cloud delay has higher accuracy, which is beneficial to the improvement of lidar performance.

[0125] It should be noted that the external terminal can be an external detection terminal. For vehicle-mounted LiDAR, the external terminal can also be the vehicle's control system, such as an autonomous driving control system. For vehicle-mounted LiDAR, the data packet is a UDP data packet.

[0126] like Figure 6 As shown, the point cloud delay detection module 100 includes: a communication detection unit 110, which is adapted to obtain the communication link delay between the lidar and the external terminal; a transceiver detection unit 120, which is adapted to perform data packet transmission and reception to determine the total data packet delay; and a processing unit 130, which is adapted to obtain the data packet point cloud delay based on the communication link delay and the total data packet delay.

[0127] It should be noted that the point cloud delay detection module 100 further includes a clock calibration unit 101, which is suitable for calibrating the lidar clock and the external terminal clock.

[0128] In some embodiments of the present invention, the clock calibration unit 101 obtains the time synchronization accuracy based on the calibration lidar clock and the external terminal clock to calibrate the lidar clock and the external clock. The time synchronization accuracy is an accuracy parameter of the lidar clock, suitable for characterizing the time difference between the lidar clock and the external terminal clock.

[0129] In some embodiments of the present invention, the clock calibration unit 101 obtains the synchronization parameters of the lidar clock and the external terminal clock within a preset time period based on the master clock, according to the lidar clock and the external terminal clock; the clock calibration unit 101 also obtains the synchronization accuracy based on the synchronization parameters of the lidar clock and the external terminal clock.

[0130] Continue to refer to Figure 6 The communication detection unit 110 obtains the communication link delay between the lidar and the external terminal.

[0131] In some embodiments of the present invention, the communication detection unit 110 obtains the communication link delay based on the mean and standard deviation of the communication link delay between the lidar and the external terminal obtained multiple times.

[0132] The transceiver detection unit 120 controls the transmission module 912 to transmit and receive data packets to determine the total delay of the data packets.

[0133] Reference Figure 7 , showed Figure 6 The diagram shows the functional block diagram of the transceiver detection unit in the lidar embodiment shown.

[0134] In some embodiments of the present invention, the transceiver detection unit 120 includes: an extractor 121, which is adapted to obtain a packet reception time TC based on the time when the external terminal obtains the data packet; a parser 122, which is adapted to parse the data packet to obtain a point cloud time TL; and a delay processor 123, which is adapted to determine the total delay T_revc of the data packet based on the packet reception time TC and the point cloud time TL.

[0135] Specifically, the extractor 121 obtains the time when the external terminal receives the data packet from the external terminal as the packet reception time TC. For vehicle-mounted LiDAR, the packet reception time is the UPD packet reception time.

[0136] In some embodiments of the present invention, the parser 122 obtains the data packet filling time TU based on the interval T0 of the control module sending the trigger signal, the buffer waiting time T4, and the data filling time T6; the parser 122 also obtains the point cloud time TL based on the data packet filling time TU and the preset advance amount δT.

[0137] In some embodiments of the present invention, the control module 911 communicates with the transmitting module 912 / receiving module 917 in a wireless mode; the parser 122 obtains the data packet filling time based on the interval T0 of the control module 911 sending the trigger signal, the buffer waiting time T4, the data filling time T6, and the communication time between the control module 911 and the transmitting module 912 / receiving module 917.

[0138] Continue to refer to Figure 7 After obtaining the packet reception time TC and the point cloud time TL, the delay processor 123 determines the total delay T_revc of the data packet based on the packet reception time TC and the point cloud time TL.

[0139] It should be noted that in some embodiments of the present invention, the point cloud delay detection module performs multiple point cloud delay detection operations to obtain multiple data packet point cloud delays.

[0140] In some embodiments of the present invention, the point cloud latency detection module performs a preset number of point cloud latency detection operations to obtain a preset number of data packet point cloud latencies. Specifically, in some embodiments, the point cloud latency detection module performs no less than 10,000 point cloud latency detection operations to obtain more than 10,000 data packet point cloud latencies.

[0141] Reference Figure 6 The diagram shows a functional block diagram of another embodiment of the lidar of the present invention.

[0142] The lidar further includes: a statistics module 201, which obtains the point cloud latency of the lidar based on the point cloud latency of the multiple data packets; and an evaluation module 202, which determines whether the lidar meets the autonomous driving standard based on the point cloud latency of the lidar.

[0143] In some embodiments of the present invention, the statistics module 201 uses at least one of the maximum value, average value and median of the point cloud delay of the plurality of data packets as the point cloud delay of the lidar.

[0144] Specifically, in some embodiments, the statistics module 201 obtains the maximum value among the multiple data packet point cloud delays based on the multiple data packet point cloud delays, and uses it as the point cloud delay of the lidar, that is, the point cloud delay of the lidar is the longest point cloud delay among the point cloud delays of the lidar's transmit and receive data packets.

[0145] In some embodiments of the present invention, the lidar is an in-vehicle lidar for autonomous driving. Therefore, after obtaining the point cloud latency of the lidar, the evaluation module 202 determines whether the lidar meets the autonomous driving standard based on the point cloud latency. Specifically, when the point cloud latency of the lidar is less than or equal to the autonomous driving standard, it is determined that the lidar meets the autonomous driving standard.

[0146] For example, the autonomous driving standard is 5 milliseconds, and the point cloud latency of the LiDAR is the longest point cloud latency among the point cloud latency of the LiDAR's data transmission and reception. That is, when the longest point cloud latency of the LiDAR's data transmission and reception does not exceed 5 milliseconds, the evaluation module 202 determines that the LiDAR meets the autonomous driving standard.

[0147] In summary, the point cloud delay detection operation can obtain the data packet point cloud delay, including the communication link delay and the total data packet delay. The obtained data packet point cloud delay has higher accuracy, which is beneficial to the improvement of lidar performance.

[0148] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A method for detecting delay in point clouds using lidar, characterized in that, include: A point cloud delay detection operation is performed to obtain the data packet point cloud delay, which includes at least the communication link delay between the lidar and the external terminal and the total data packet delay. The point cloud latency detection operation includes: performing data packet transmission and reception to determine the total latency of data packets transmitted between the lidar and the external terminal; The steps for determining the total delay of data packets transmitted between the lidar and the external terminal by performing data packet transmission and reception include: obtaining the packet reception time based on the time when the external terminal receives the data packet; parsing the data packet to obtain the point cloud time; and determining the total delay of the data packet based on the difference between the packet reception time and the point cloud time. The steps of parsing the data packet to obtain the point cloud time include: obtaining the data packet filling time based on the interval of the trigger signal sent by the control module, the buffer waiting time, and the data filling time; and obtaining the point cloud time based on the data packet filling time and a preset lead time. Perform multiple point cloud latency detection operations to obtain the point cloud latency of multiple data packets; The detection method further includes: obtaining the point cloud delay of the lidar based on the point cloud delay of the multiple data packets; Based on the point cloud latency of the lidar, it is determined whether the lidar meets the autonomous driving standard. When the point cloud latency of the lidar is less than or equal to the autonomous driving standard, it is determined that the lidar meets the autonomous driving standard.

2. The detection method as described in claim 1, characterized in that, The point cloud delay detection operation includes: The communication link delay between the lidar and the external terminal is obtained; The data packet point cloud delay is obtained based on the communication link delay and the total data packet delay.

3. The detection method as described in claim 2, characterized in that, Before obtaining the communication link delay between the lidar and the external terminal, the point cloud delay detection operation further includes: calibrating the lidar clock and the external terminal clock.

4. The detection method as described in claim 3, characterized in that, The steps for calibrating the lidar clock and external terminal clock include: Synchronize the lidar clock and the external terminal clock; Alternatively, time synchronization accuracy can be obtained based on the calibrated lidar clock and the external terminal clock.

5. The detection method as described in claim 2, characterized in that, The steps for obtaining the communication link delay between the lidar and the external terminal include: The communication link delay is obtained by taking the mean and standard deviation of the communication link delay between the lidar and the external terminal multiple times.

6. The detection method as described in claim 1, characterized in that, In the step of obtaining the point cloud delay of the lidar, at least one of the maximum value, average value and median of the point cloud delay of the plurality of data packets is used as the point cloud delay of the lidar.

7. The detection method as described in claim 1, characterized in that, The data packet includes: the time of transmission of the trigger signal; The step of parsing the data packet to obtain the point cloud time includes: parsing the data packet to obtain the sending time of the trigger signal as the point cloud time.

8. The detection method as described in claim 1, characterized in that, The lead amount is set based on the number of laser pulses corresponding to the data of each point obtained from the lidar point cloud.

9. The detection method as described in claim 1, characterized in that, The control module and the transceiver module communicate wirelessly. In the step of obtaining the data packet filling time, the data packet filling time is obtained based on the interval time of the control module sending the trigger signal, the buffer waiting time, the data filling time, and the communication time between the control module and the transceiver module.

10. A lidar point cloud delay detection system, characterized in that, include: The lidar point cloud delay detection system is suitable for implementing the detection method according to any one of claims 1 to 9.

11. A lidar, characterized in that, include: Transmitter module, receiver module; The beam emitted by the transmitting module is reflected by the target, and the resulting echo is received by the receiving module to obtain a data packet for generating a point cloud. The lidar also includes: a lidar point cloud delay detection system as described in claim 10.

12. A lidar, characterized in that, include: Transmitter module, receiver module; The beam emitted by the transmitting module is reflected by the target, and the resulting echo is received by the receiving module to obtain a data packet for generating a point cloud. The lidar further includes a point cloud delay detection module, which is adapted to perform point cloud delay detection operations to obtain data packet point cloud delay. The data packet point cloud delay includes at least the communication link delay between the lidar and the external terminal and the total delay of the data packet. The point cloud latency detection module includes: a transmit / receive detection unit, which is adapted to transmit and receive data packets to determine the total latency of the data packets; the transmit / receive detection unit includes: an extractor, which is adapted to obtain the packet reception time based on the time when the external terminal obtains the data packets; a parser, which is adapted to parse the data packets to obtain the point cloud time; and a latency processor, which is adapted to determine the total latency of the data packets based on the difference between the packet reception time and the point cloud time. The parser has a preset lead time. The parser obtains the data packet filling time based on the interval of the trigger signal sent by the control module, the buffer waiting time, and the data filling time. The parser also obtains the point cloud time based on the data packet filling time and the preset lead time. The point cloud latency detection module performs multiple point cloud latency detection operations to obtain the point cloud latency of multiple data packets; The lidar further includes: a statistics module, which obtains the point cloud latency of the lidar based on the point cloud latency of the multiple data packets; and an evaluation module, which determines whether the lidar meets the autonomous driving standard based on the point cloud latency of the lidar. When the point cloud latency of the lidar is less than or equal to the autonomous driving standard, the evaluation module determines that the lidar meets the autonomous driving standard.

13. The lidar as described in claim 12, characterized in that, The point cloud delay detection module includes: A communication detection unit, which is adapted to obtain the communication link delay between the lidar and an external terminal; The processing unit is adapted to obtain the data packet point cloud delay based on the communication link delay and the total data packet delay.

14. The lidar as described in claim 13, characterized in that, The point cloud delay detection module also includes a clock calibration unit, which is suitable for calibrating the lidar clock and the external terminal clock.

15. The lidar as described in claim 14, characterized in that, The clock calibration unit is suitable for synchronizing the lidar clock and the external terminal clock; Alternatively, the clock calibration unit is adapted to obtain time synchronization accuracy based on the calibration lidar clock and the external terminal clock.

16. The lidar as described in claim 12, characterized in that, The point cloud latency detection module performs multiple point cloud latency detection operations to obtain the point cloud latency of multiple data packets; The lidar also includes: A statistics module, which is adapted to obtain the point cloud delay of the lidar based on the point cloud delay of the multiple data packets; An evaluation module is provided, which is suitable for determining whether the lidar meets the autonomous driving standards based on the point cloud latency of the lidar.

17. The lidar as described in claim 16, characterized in that, The statistical module is adapted to use at least one of the maximum, average, and median point cloud delays of the plurality of data packets as the point cloud delay of the lidar.

18. The lidar as described in claim 13, characterized in that, The communication detection unit is adapted to obtain the communication link delay based on the mean and standard deviation of the communication link delay between the lidar and the external terminal obtained multiple times.

19. The lidar as described in claim 12, characterized in that, The data packet includes: the time of transmission of the trigger signal; The parser parses the data packet to obtain the sending time of the trigger signal as the point cloud time.

20. The lidar as described in claim 12, characterized in that, The lead amount is set based on the number of laser pulses corresponding to the data of each point obtained from the lidar point cloud.

21. The lidar as described in claim 12, characterized in that, The control module and the transceiver module communicate wirelessly. The parser obtains the data packet filling time based on the interval between the trigger signals sent by the control module, the buffer waiting time, the data filling time, and the communication time between the control module and the transceiver module.

Images