Light detection and distance measurement device and event detection method thereof

By introducing an event detection method in LiDAR, only data whose distance and reflectivity difference exceeds the threshold are output, the problem of redundant data in the fast response task of frame-based LiDAR is solved, and more efficient data transmission and response speed are achieved.

CN119064898APending Publication Date: 2024-12-03COMPERTUM MICROSYSTEMS INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310888058.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-31
Filing Date
2023-07-19
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing frame-based LiDARs generate a large amount of redundant data in fast response tasks, which is difficult to meet the needs of fast response such as tracking, positioning and action estimation.

Method used

Using the event detection method, data is output only when the difference between the pixel distance or reflectivity of the current frame and the previous frame exceeds the threshold, meaningful point cloud data is obtained through the event detector, reducing redundant data transmission.

Benefits of technology

It realizes rapid response in tracking, positioning and action estimation tasks, reduces redundant data transmission, and improves the efficiency and response speed of LiDAR.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119064898A_ABST
    Figure CN119064898A_ABST
Patent Text Reader

Abstract

The invention discloses a light detection and distance measurement device and an event detection method thereof. An event detection method for a light detection and ranging device includes obtaining a first current distance of a first pixel of a current frame or a first current reflectance of the first pixel of the current frame; and only upon determining that a first distance difference between a first current distance of a first pixel of a current frame and a first previous distance of a first pixel of a previous frame exceeds a distance threshold or only upon determining that a first current reflectance of a first pixel of a current frame and a first previous reflectance of a first pixel of a previous frame After the first reflectivity difference exceeds a reflectivity threshold value, the first current distance or the first current reflectivity is output. A plurality of pixels including a first pixel are regularly mapped to a two-dimensional field of view of a light detection and ranging device to improve response.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to a light detection and ranging (LiDAR) for surveying a surrounding environment, and more specifically, to a LiDAR for tracking, positioning, and motion estimation in a surrounding environment. Background Art

[0002] LiDAR emits invisible laser signals to objects within the field of view (FOV) and measures the time delay between the outgoing and returning laser signals to calculate the distance. Existing LiDAR (also called frame-based LiDAR) sends a frame of point cloud data at each time interval.

[0003] For machine vision tasks that require a large amount of data (such as object recognition), existing frame-based LiDAR is suitable. However, for tasks that require fast response (such as tracking, positioning, and motion estimation), frame-based LiDAR that generates a large amount of redundant data is not ideal. Therefore, it is necessary to improve existing LiDAR to ensure fast response. Summary of the invention

[0004] Therefore, the main object of the present application is to provide a light detection and ranging device and an event detection method thereof to improve the response.

[0005] The present application discloses an event detection method for a light detection and ranging device, comprising obtaining a first current distance of a first pixel of a current frame or a first current reflectivity of the first pixel of the current frame; and outputting the first current distance or the first current reflectivity only after determining that a first distance difference between the first current distance of the first pixel of the current frame and a first previous distance of the first pixel of a previous frame exceeds a distance threshold or only after determining that a first reflectivity difference between the first current reflectivity of the first pixel of the current frame and a first previous reflectivity of the first pixel of the previous frame exceeds a reflectivity threshold, wherein a plurality of pixels including the first pixel are regularly mapped to a two-dimensional field of view of the light detection and ranging device.

[0006] The present application discloses a light detection and distance measurement device, comprising a light transmitter, for transmitting pulse light, wherein the pulse light is invisible light; a light receiver, optically coupled to the light transmitter, and for capturing reflected pulse light, wherein the reflected pulse light represents the pulse light reflected by at least one object; a judgment circuit, coupled to the light transmitter and the light receiver, and for determining a plurality of distances relative to the at least one object and a plurality of reflectivities of the at least one object; and an event detector, coupled to the judgment circuit, and for obtaining a first current distance of a first pixel of a current frame and a first current reflectivity of the first pixel of the current frame, wherein the event detector The device is used to output the first current distance or the first current reflectivity only after determining that a first distance difference between the first current distance of the first pixel of the current frame and a first previous distance of the first pixel of a previous frame exceeds a distance threshold or only after determining that a first reflectivity difference between the first current reflectivity of the first pixel of the current frame and a first previous reflectivity of the first pixel of the previous frame exceeds a reflectivity threshold, wherein the multiple distances include the first current distance, the multiple reflectivities include the first current reflectivity, and the multiple pixels including the first pixel are regularly mapped to a one-dimensional field of view of the light detection and ranging device. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 and Figure 2 They are respectively schematic diagrams of the LiDAR devices of the embodiments of the present application.

[0008] Figure 3 It is a schematic diagram of the mapping of point cloud data of a two-dimensional FOV of a LiDAR device and the corresponding data structure according to an embodiment of the present application.

[0009] Figure 4 It is a flow chart of an event detection method according to an embodiment of the present application.

[0010] Figure 5 and Figure 6 They are respectively schematic diagrams of the LiDAR devices of the embodiments of the present application.

[0011] Figure 7 It is a schematic diagram of the mapping of point cloud data of a two-dimensional FOV of a LiDAR device and the corresponding data structure according to an embodiment of the present application.

[0012] Figure 8 It is a flow chart of an event detection method according to an embodiment of the present application.

[0013] Fig. 9 is a schematic diagram of a LiDAR device according to an embodiment of the present application.

[0014] Fig.10 It is a schematic diagram of the mapping of point cloud data of a two-dimensional FOV of a LiDAR device and the corresponding data structure according to an embodiment of the present application.

[0015] The reference numerals are described as follows:

[0016] 10 LiDAR Device

[0017] 100 Optical Transceiver

[0018] 110 Optical Transmitter

[0019] 130 Optical Receiver

[0020] 150 Judgment Circuit

[0021] 170 Storage Circuit

[0022] 190, 290, 690, 990 event detectors DETAILED DESCRIPTION

[0023] Figure 1 1 is a schematic diagram of a LiDAR device 10 according to an embodiment of the present application. The LiDAR device 10 includes an optical transceiver 100 , a determination circuit 150 , a storage circuit 170 and an event detector 190 .

[0024] The optical transceiver 100 may include an optical transmitter 110 and an optical receiver 130. The optical transmitter 110 may include one or more light sources, each of which is used to emit pulsed light (e.g., invisible pulsed light). The optical receiver 130 is used to capture reflected pulsed light. In one embodiment, the optical receiver 130 is a Geiger mode avalanche photodiode receiver, which may include one or more photodetectors for measuring the light intensity of the reflected pulsed light and converting it into a photocurrent (i.e., a current signal), and the photocurrent may be related to (e.g., proportional to) the light intensity. In one embodiment, the optical transceiver 100 may also include a beam steering unit.

[0025] The judgment circuit 150 is used to find out the distance between the object and the LiDAR device 10 and the reflectivity of the object. The judgment circuit 150 can detect the transmission timing of the pulse light emitted by the optical transmitter 110 and the reception timing of the reflected pulse light received by the optical receiver 130 to determine the distance between the object and the LiDAR device. Assuming that the intensity of the pulse light can be preset or pre-measured by the judgment circuit 150, the judgment circuit 150 can detect the intensity of the reflected pulse light to determine the reflectivity of the object. The reflectivity can be related to the ratio between the intensity of the reflected pulse light and the intensity of the pulse light.

[0026] The storage circuit 170 (eg, a memory device) may include at least one memory unit (eg, Figure 3Memory units M00 to Mmn) shown in (b), wherein at least one memory unit is used to store / record all pixels (e.g. Figure 3 The distance from the pixel [0,0] to [m,n]) shown in (a) (e.g. Figure 3 (b) shows the distance d00 to dmn) or the reflectivity (e.g. Figure 3 In terms of pixel [m, n], the distance dmn of frame t1 can be expressed as d t1 mn, and the reflectivity Rmn of frame t1 can be expressed as R t1 mn.

[0027] The event detector 190 (e.g., a digital transmitter) is used to detect the distance between a certain pixel (e.g., [i, j]) in a certain frame (which may be called the current frame) (e.g., t1) and the distance (e.g., d t0 ij) difference (e.g. |d t1 ij-d t0 ij|) exceeds a distance threshold dth, or when the reflectivity of the pixel in the (current) frame (e.g., t1) is greater than the reflectivity of the pixel in the (most recent / previous) frame (e.g., t0) (e.g., R t0 ij) (e.g. |R t1 ij-R t0 ij|) exceeds the reflectivity threshold Rth, asynchronously outputs the distance (e.g., d t1 ij) or reflectivity (such as R t1 ij) to the processing circuit 10CPU.

[0028] In one embodiment, the processing circuit 10CPU (eg, a central processing unit (CPU)) may be disposed outside the LiDAR device 10. Alternatively, the LiDAR device 10 may include the processing circuit 10CPU.

[0029] In short, as an event-based LiDAR device, the LiDAR device 10 can efficiently extract meaningful point cloud data of one or some pixels of one or some frames from all point cloud data of all pixels of all frames detected by the LiDAR device 10. By analyzing the meaningful point cloud data detected by the LiDAR device 10, the processing circuit 10CPU can quickly detect / identify abnormal objects or events that may occur, and provide valuable insights for further analysis.

[0030] In one embodiment, after the optical receiver 130 receives the reflected pulse light of one or some pixels [i, j] of frame t1, the event detector 190 compares the difference |d ij between the current frame t1 and the distance dij of the previous frame t0 for each pixel. t1 ij-d t0 For each pixel, if the difference is greater than the distance threshold dth, a flag is set indicating an abnormal distance event, and the event detector 190 immediately sets the distance dth to t1 ij and its corresponding reflectivity R t1 ij is output to the processing circuit 10CPU. If no event indicating an abnormal distance is flagged, the event detector 190 compares the difference between the reflectivity Rij of the current frame t1 and its previous frame t0 for each pixel. t1 ij-R t0 For each pixel, if the difference is greater than the reflectivity threshold Rth, an event indicating abnormal reflectivity is flagged and the event detector 190 immediately reports the reflectivity R t1 ij and its corresponding distance d t1 ij is output externally to the processing circuit 10CPU. If no event indicating abnormal reflectivity is flagged, the event detector 190 may begin checking the next pixel or the next frame.

[0031] In another embodiment, the event detector 190 may first compare the difference between the reflectivity Rij of the current frame t1 and the reflectivity Rij of the previous frame for each pixel. t1 ij-R t0 ij|, and then compare the difference between the distance dij between the current frame t1 and its previous frame |d t1 ij-d t0 ij|.

[0032] Figure 2 2 is a schematic diagram of a LiDAR device 20 according to an embodiment of the present application. The LiDAR device 20 includes a light transmitter 210, a light beam steering unit 220, a light receiver 230, a judgment circuit 250, a storage circuit 170, and an event detector 290. The judgment circuit 250 may include an extraction circuit 250E, a comparator 250sCMP, converters 250TDC and 250rC. The extraction circuit 250E may include an amplifier 250TIA and a comparator 250dCMP.

[0033] The light emitter 210 may include one or more light sources; the light receiver 230 may include one or more light detectors 230d.

[0034] The beam control unit 220 (optically coupled between the light emitter 210 and the light receiver 230) is used to guide / steer the pulse light emitted by the light emitter 210 and its reflected pulse light, so that the reflected pulse light can be received by the light receiver 230. Accordingly, the emitter 210 can scan in a bidirectional / two-dimensional FOV. In one embodiment, the pulse light incident to the beam control unit 220 is parallel or coaxial with the reflected pulse light deflected by the beam control unit 220. Alternatively, the pulse light deflected by the beam control unit 220 is parallel or coaxial with the reflected pulse light incident to the beam control unit 220.

[0035] The amplifier 250TIA (e.g., a trans-impedance amplifier (TIA)) can be used to measure the amplitude of the photocurrent (generated by the photodetector 230d coupled to the amplifier 250TIA) and convert the photocurrent into a photovoltage VP250 (related to the detected light intensity). For example, when the photodetector 230d receives a light pulse, the photodetector 230d can generate a current pulse that matches the intensity of the light pulse. The amplifier 250TIA then generates a voltage pulse VP250 (corresponding to the received current pulse). The amplifier 250TIA can also be used as an electronic filter (e.g., a low-pass filter) to remove / reduce high-frequency electrical noise. The amplifier 250TIA may include one or more voltage amplification stages to increase the amplitude of the voltage pulse VP250 to exceed the amplitude of the voltage pulse directly converted from the received current pulse.

[0036] The comparator 250dCMP (coupled between the amplifier 250TIA and the converter 250TDC) is used to compare the photovoltage VP250 with the reference voltage Vref. For example, when the photovoltage VP250 rises to be higher than (or falls to be lower than) the reference voltage Vref, the comparator 250dCMP can generate a signal (such as an edge detection signal or a digital high-level signal) according to the comparison result to indicate the rising edge (or falling edge) of the photovoltage VP250.

[0037] The comparator 250sCMP (coupled between the optical transmitter 210 and the converter 250TDC) is used to detect the change in the intensity of the pulsed light. When the comparator 250sCMP detects the rising edge (or falling edge) of the signal PL210 (corresponding to the pulsed light from the optical transmitter 210) (i.e., when the rising edge of the pulsed light is detected), the comparator 250sCMP can output a signal (e.g., an edge detection signal or a digital high-level signal).

[0038] The converter 250TDC (e.g., a time-to-digital converter) is used to measure the time difference TDd between the time TDC2 of the rising edge of the photovoltage VP250 and the time TDC1 of the rising edge of the signal PL210. The time TDC1 may correspond to the sending timing of the pulse light sent by the optical transmitter 210, and the time TDC2 may correspond to the receiving timing of the reflected pulse light received by the optical receiver 230. For example, Figure 2 (b) is a timing diagram showing the time sequence of the times TDC1 and TDC2. The converter 250TDC can convert the time difference TDd into a value corresponding to the distance dij (between the object and the LiDAR device 20).

[0039] Converter 250rC (e.g., a current-to-reflectivity converter) is used to convert the amplitude of the photocurrent (generated by the photodetector 230d), the amplitude of the signal PL210 (corresponding to the intensity of the pulsed light), or the received signal strength indication (RSSI) into a reflectivity Rij (related to the light intensity detected by the photodetector 230d).

[0040] The storage circuit 170 is coupled between the converters 250TDC and 250rC and the event detector 290, and is used to store / record the distance or reflectivity provided by the converters 250TDC and 250rC.

[0041] The event detector 290 is used to calculate the difference (eg |d|) between the distance / reflectivity of all pixels of a certain frame (eg, t1) and the distance / reflectivity of all pixels of the previous frame (eg, t0) almost one by one. t1 ij-d t0 ij|or|R t1 ij-R t0 ij|): For one frame, event detector 290 may determine whether the distance / reflectivity difference of one pixel exceeds a distance threshold / reflectivity threshold, and then determine whether the distance / reflectivity difference of another pixel exceeds a distance threshold / reflectivity threshold.

[0042] Figure 3(a) shows a bidirectional / two-dimensional scanning pattern 30SP (spanning the horizontal field of view 3HFOV and the vertical field of view 3VFOV) of pulsed light sent by the light emitter 210 of the LiDAR device (e.g., 20 or 50) through the beam steering unit 220. The LiDAR device can be programmed to scan the pulsed light along the bidirectional / two-dimensional scanning pattern 30SP, which may include a sequence of pixels [0,0] to [m,n]. Each of the pixels [0,0] to [m,n] is associated with a specific location / point within the entire FOV being measured by the LiDAR device (i.e., a one-to-one mapping), and reflections may occur from objects located in the three-dimensional space (corresponding to pixels [0,0] to [m,n]). A single cycle / frame of the bidirectional / two-dimensional scanning pattern 30SP may include (m+1)×(n+1) pixels. The bidirectional / two-dimensional scanning pattern 30SP may be modified to accommodate different system configurations.

[0043] Figure 3 The data structure 30DS shown in (b) may include a set of data elements DS00 to DSmn. Each data element DSij corresponds to a pixel (e.g., [i, j]) and may include point cloud data (e.g., distance dij or reflectivity Rij associated with pixel [i, j]). In one embodiment, the data elements DS00 to DSmn may occupy a continuous area (e.g., memory cells M00 to Mmn) within the storage circuit 170. Alternatively, the physical addresses of the data elements DS00 to DSmn may be sparsely discrete.

[0044] For example, when the LiDAR device scans, the beam steering unit can guide the pulsed light to pixels [0,0]-[m,n] along the bidirectional / two-dimensional scanning pattern 30SP. At each pixel (e.g., [i,j]), the pulsed light is incident and bounces back from the object and is then captured by the light receiver 230. The distance dij relative to the object and the reflectivity Rij of the object are measured and stored in the corresponding memory unit Mij.

[0045] For example, Figure 4 4 is a flow chart of an event detection method 40 (or event generation method) according to an embodiment of the present application. The event detection method 40 may be compiled into code and executed by an event detector of a LiDAR device (e.g., 10, 20, or 50), and may include the following steps:

[0046] Step S402: The LiDAR device may scan pixel [i, j] in frame t1.

[0047] Step S404: The distance d between pixel [i, j] in frame t1 t1 ij and reflectivity R t1ij may be stored in the memory circuit 170 of the LiDAR device or in the memory circuit of the event detector (eg, a register or cache of the event detector). The event detector may read / access the distance d t1 ij and reflectivity R t1 ij.

[0048] Step S406: The event detector may compare the distance threshold dth with (the distance d of the current frame t1 t1 The distance d between ij and the previous frame t0 t0 The difference between ij t1 ij-d t0 ij|. If the difference |d t1 ij-d t0 If ij| exceeds the distance threshold dth, execute step S408; otherwise, execute step S412.

[0049] Step S408: The event detector may mark the event event d .

[0050] Step S410: The event detector can immediately output the distance d of pixel [i, j] t1 ij and reflectivity R t1 ij (to processing circuit 10CPU).

[0051] Step S412: The event detector may compare the reflectivity threshold Rth with (the reflectivity R of the current frame t1) t1 ij and the reflectivity R of the previous frame t0 t0 ij) difference between |R t1 ij-R t0 ij|. If the difference |R t1 ij-R t0 If ij| exceeds the reflectivity threshold Rth, execute step S414; otherwise, execute step S416.

[0052] Step S414: The event detector may mark the event event R .

[0053] Step S416: The event detector can calculate the distance d of the previous frame t0 t0 ij is replaced by the distance d of the current frame t1 t1 ij, and the reflectivity R of the previous frame t0 t0 ij is replaced by the reflectivity R of the current frame t1 t1 ij.

[0054] Step S418: The event detector may determine whether the pixel [i, j] is the last pixel [m, n] of the frame t1. If not, execute step S420; otherwise, execute step S422.

[0055] Step S420: The event detector may move to the next pixel (eg, [i, j+1] or [i+1, j]) of frame t1. In other words, the event detector may retrieve / capture another distance (eg, d t1 i(j+1) or d t1 (i+1)j) and reflectivity (e.g. R t1 i(j+1) or R t1 (i+1)j), so that the event detector can process the point cloud data of subsequent pixels.

[0056] Step S422: The event detector may transfer to the next frame t2. In other words, the event detector may retrieve / capture another distance d of the pixel [0,0] of the next frame t2 (relative to step S404). t2 00 and reflectivity R t2 00, so that the event detector can process the point cloud data of subsequent frames.

[0057] In one embodiment, steps S408 and S410 can be performed simultaneously; steps S414 and S410 can be performed simultaneously. In one embodiment, steps S406 (or S412) and S416 can be performed simultaneously.

[0058] In one embodiment, the distance threshold dth or the reflectivity threshold Rth may be a preset constant value applicable to all pixels or frames. Alternatively, the distance threshold dth or the reflectivity threshold Rth may vary with pixels or frames.

[0059] In one embodiment, the Li DAR device has the ability to switch between normal mode and event mode. In normal mode, the Li DAR device can be used as a frame-based Li DAR device, and the Li DAR device can capture point cloud data within the entire FOV at a frequency f (e.g., 30 Hz) in one frame. The point cloud data of all pixels of the previous frame (e.g., t0) can be sent to the processing circuit 10CPU through the event detector at the previous time point tt0, and the point cloud data of all pixels of the current frame (e.g., t1) can be sent out at the current time point tt1, where tt1=tt0+1 / f. The Li DAR device can only generate and output point cloud data at intervals of 1 / f. The event detector can essentially transmit point cloud data in parallel. This may result in redundant data transmission when there is no significant change in distance or reflectivity in a frame. For machine vision tasks that require a large amount of data, a Li DAR device in normal mode is suitable.

[0060] In the event mode, the LiDAR device can be used as an event-based LiDAR device, and the LiDAR device can perform the event detection method 40. The LiDAR device in the event mode only needs to identify an event (e.g.,d or event R ), the Li DAR device can asynchronously send the corresponding point cloud data to the processing circuit 10CPU immediately and without delay: in other words, not all point cloud data of each pixel received by the event detector 290 is sent out; not all point cloud data of each frame received by the event detector 290 is sent out. Any point cloud data that is not marked as an event will not be sent out to minimize redundant data transmission. For example, the event detector can output the point cloud data of the first pixel (e.g., [0,1]) of the first frame (e.g., t1) at a first time point, the point cloud data of the first pixel (e.g., [0,1]) of the second frame (e.g., t2) at a second time point, the point cloud data of the second pixel (e.g., [m,1]) of the fourth frame (e.g., t4) at a fourth time point, and the point cloud data of the third pixel (e.g., [8,5]) of the fifth frame (e.g., t5) at a fifth time point. There may be no significant change between each pixel of the third frame and each pixel of the second frame, so the event detector may not output any point cloud data of the third frame (e.g., t3) to avoid unnecessary data transmission. The time difference between the second time point and the first time point may be different from the time difference between the fifth time point and the fourth time point or the time interval of 1 / f. The present application enables the processing circuit to perceive local / sudden changes in distance or reflectivity and detect whether there is a (fast) oncoming vehicle or whether the vehicle suddenly stops. The LiDAR device in event mode can provide point cloud data at a frequency faster than the frequency f, without waiting for a fixed 1 / f time interval to send out (possibly redundant) data. The LiDAR device can operate in event mode to perform tasks that require fast response, such as tracking, positioning, and motion estimation.

[0061] Figure 5 is a schematic diagram of a LiDAR device 50 according to an embodiment of the present application. The LiDAR device 50 may be an event-based coaxial LiDAR device like the LiDAR device 20. Unlike the LiDAR device 20, the determination circuit 550 of the LiDAR device 50 may include an extraction circuit 550E, a comparator 250sCMP, and a converter 550TDC. The extraction circuit 550E may include an amplifier 550TIA and a comparator 250dCMP. The converter 250rC is removed / emptied from the LiDAR device 50.

[0062] Unlike amplifier 250TIA, amplifier 550TIA may be a different type of transimpedance amplifier having a duration time (related to the magnitude of the photocurrent generated by photodetector 230d). Therefore, the magnitude of the photocurrent may be measured by calculating the time difference TDr between the time TDC3 of the falling edge of photovoltage VP550 and the time TDC2 of the rising edge of photovoltage VP550.

[0063] Unlike the converter 250TDC, the converter 550TDC can measure the time difference TDd between the time TDC2 and the time TDC1, and convert the time difference TDd into a value corresponding to the distance between the object and the LiDAR device 50. The converter 550TDC can measure the time difference TDr between the time TDC3 and the time TDC2, and convert the time difference TDC into a value corresponding to a reflectivity, which is related to the light intensity detected by the light detector 230d. For example, Figure 3 (b) is a timing diagram showing the time sequence of the times TDC1 , TDC2 , and TDC3 .

[0064] An event-based coaxial LiDAR device (e.g., 20 or 50) can achieve early event detection. However, once an event is detected at a certain pixel (e.g., [0,0]) in the (current) frame (e.g., t1), the LiDAR device will not trigger / initiate detection of another event for the same pixel until the next frame (e.g., t2), which is a time interval 1 / f later than the current frame, so the temporal resolution is not improved. In one embodiment, multiple events may occur in the vicinity of a frame, and this requires a higher level of attention. This increased level of attention is critical to distinguishing real events from false events (which may be caused by surrounding noise). Events identified by the LiDAR device (e.g., adjacent events) can indicate an area of ​​interest, and identifying events only in one or some areas of interest can improve temporal resolution. In addition, the main disadvantage of a coaxial LiDAR device may be the use of a small number of light detectors to cover a small FOV, which requires a beam steering unit to fully extend its FOV. However, since most beam steering units are mechanically based, it takes a lot of time (eg, 1 / 30 Hz) to scan the entire FOV.

[0065] Figure 6 6 is a schematic diagram of a LiDAR device 60 according to an embodiment of the present application. The LiDAR device 60 (which can improve the temporal resolution) can be an event-based non-coaxial LiDAR device, and can include a light transmitter 610, a light receiver 630, a determination circuit 650, a storage circuit 170, and an event detector 690.

[0066] The light transmitter 610 may include a plurality of light sources, each light source being used to emit pulsed light that flashes the entire FOV (at once or simultaneously). The light receiver 630 may include a light detector 230d to receive reflected pulsed light in a bidirectional / two-dimensional FOV (at once or simultaneously).

[0067] The determination circuit 650 may include an extraction circuit 650E, a comparator 250sCMP, and a converter 550TDC. Each extraction circuit 650E may include an amplifier 550TIA and a comparator 250dCMP. The beam steering unit 220 and the converter 250rC are removed / emptied from the LiDAR device 60.

[0068] The event detector 690 is used to calculate the difference (eg, |d|) between the distance / reflectivity of all pixels of a certain frame (eg, t1) and the distance / reflectivity of all pixels of the previous frame (eg, t0) almost simultaneously or at once. t1 ij-d t0 ij|or|R t1 ij-R t0 ij|).

[0069] Figure 7 (a) shows a bidirectional / two-dimensional scanning pattern 70SP of pulsed light (spanning the horizontal field of view 7HFOV and the vertical field of view 7VFOV) sent by the light emitter of the LiDAR device (e.g., 60). The LiDAR device may not be provided with a beam steering unit (e.g., 220). In one flash, all light sources of the light emitter emit pulsed light covering the entire FOV, and once the pulsed light hits an object, it will be reflected from the object and then captured by the light detector 230d array covering the entire FOV. A single cycle / frame of the bidirectional / two-dimensional scanning pattern 70SP may include (m+1)×(n+1) pixels, and the number of extraction circuits 650E may be equal to (m+1)×(n+1).

[0070] For example, Figure 8 FIG. 8 is a flow chart of an event detection method 80 of a LiDAR device (such as 10 or 60) according to an embodiment of the present application. The event detection method 80 may include the following steps:

[0071] Step S802 : The LiDAR device may flash all pixels [0,0] to [m,n] in frame t1 .

[0072] Step S804: The distance d between pixels [0,0] and [m,n] in frame t1 t1 00 to d t1 mn and reflectivity R t1 00 to R t1 mn may be stored in the memory circuit 170 of the LiDAR device or in a register or cache of the event detector. The event detector may read / access the distance d t1 00 to d t1 mn and reflectivity R t1 00 to R t1 mn.

[0073] Step S806: For each pixel (eg, [0,0], ..., or [m,n]), the event detector may compare the distance threshold dth with (the distance d of the current frame t1) t1 00, ..., or d t1 The distance d between mn and the previous frame t0 t0 00, ..., or d t0 mn) difference (e.g. |d t1 00-d t0 00|, ..., or |d t1 mn-d t0 mn|). If the difference |d t1 00-d t0 00| to|d t1 mn-d t0 If any one of mn| exceeds the distance threshold dth, execute step S808; otherwise, execute step S812.

[0074] Step S808: The event detector may detect each difference (eg, |d t1 01-d t0 01|、|d t1 mn-d t0 mn|) (e.g. d t1 01, d t1 mn)Mark event event d .

[0075] Step S810: The event detector can immediately output the event event d or event R The distance (e.g. d) of the corresponding pixels (e.g. [0,1], [m,n]) t1 01, d t1 mn) and the corresponding reflectivity (e.g. R t1 01. R t1 mn) (to the processing circuit 10CPU).

[0076] Step S812: For each pixel, the event detector may compare the reflectivity threshold Rth with (the reflectivity R t1 00, ..., or R t1 mn and the reflectivity R of the previous frame t0 t0 00, ..., or R t0 mn) difference (e.g. |R t1 00-R t0 00|, ..., or |R t1 mn-R t0 mn|). If the difference |R t1 00-R t0 00| to | Rt1 mn-R t0 If any one of mn| exceeds the reflectivity threshold Rth, execute step S814; otherwise, execute step S816.

[0077] Step S814: The event detector may detect each difference (eg, |R t1 mn-R t0 mn|) reflectivity (e.g. R t1 mn)Mark event event R .

[0078] Step S816: For each pixel, the event detector can calculate the distance (e.g., d t0 00, ..., or d t0 mn) is replaced by the distance of the current frame t1 (e.g. d t1 00, ..., or d t1 mn), and the reflectivity of the previous frame t0 (e.g., R t0 00, ..., or R t0 mn) is replaced by the reflectivity of the current frame t1 (e.g. R t1 00, ..., or R t1 mn).

[0079] Step S818: The event detector may transfer to the next frame t2. In other words, the event detector may retrieve / capture the other distance d from the pixel [0,0] to [m,n] of the next frame t2 (relative to step S804). t2 00 to d t2 mn and reflectivity R t2 00 to R t2 mn, so that the event detector can process the point cloud data of subsequent frames.

[0080] In one embodiment, steps S808 and S810 may be performed simultaneously; steps S814 and S810 may be performed simultaneously. In one embodiment, steps S806 (or S812) and S816 may be performed simultaneously.

[0081] For non-coaxial LiDAR devices (e.g., 60), the frame rate is typically in the nanosecond to microsecond range and can be defined by the refresh rate of the light emitter or the dead time of the light detector 230d. The frame rate of non-coaxial LiDAR devices is much higher than the frame rate of coaxial LiDAR devices (e.g., 20). The frame rate of coaxial LiDAR devices is typically on the order of milliseconds or tenths of a second and can be determined by the time it takes the beam steering unit 220 to scan the entire FOV. Once the event detector of the coaxial LiDAR device recognizes an event (e.g., event d or eventR ), the event detector can immediately send the point cloud data corresponding to the event to the processing circuit 10CPU asynchronously, without waiting to send the point cloud data before processing the next frame: for example, the event detector can output the point cloud data of the pixels (e.g., [0,1], [3,5], [m,1]) of the first frame (e.g., t1) at a first time point, and output the point cloud data of the pixels (e.g., [0,1], [10,2]) of the third frame (e.g., t3) at a third time point. Non-coaxial LiDAR devices can provide high temporal resolution, which is conducive to continuous tracking of visual features.

[0082] Fig. 9 is a schematic diagram of a LiDAR device 90 according to an embodiment of the present application. The LiDAR device 90 may be an event-based coaxial or non-coaxial LiDAR device.

[0083] The determination circuit 950 may include extraction circuits 950E1 to 950Ey and a converter 550TDC. Each of the extraction circuits 950E1 to 950Ex may include a comparator 250sCMP (coupled to a light source 910s of the light emitter 610), an amplifier 250TIA (coupled to a light detector 230d), and a comparator 250dCMP.

[0084] The light beam steering unit 920 (optically coupled between the light emitter 610 and the light receiver 630) is used to guide / steer the pulse light and its reflected pulse light. Alternatively, the light beam steering unit 920 (optically coupled to one of the light emitter 610 and the light receiver 630) is used to guide / steer one of the pulse light and its reflected pulse light. Accordingly, the emitter 610 can scan at least a portion of the two-dimensional / two-dimensional FOV.

[0085] The event detector 990 is used to calculate the difference (eg, |d|) between the distance / reflectivity of all pixels of a certain frame (eg, t1) and the distance / reflectivity of all pixels of the previous frame (eg, t0) almost group by group or row by row. t1 ij-d t0 ij|or|R t1 ij-R t0 ij|): For one frame, the event detector 990 may determine whether the distance / reflectivity difference of one group / row of pixels exceeds a distance threshold / reflectivity threshold, and then determine whether the distance / reflectivity difference of another group / row of pixels exceeds a distance threshold / reflectivity threshold.

[0086] Fig.10(a) shows a bidirectional / two-dimensional scanning pattern 10SP of pulsed light (spanning the horizontal field of view 10HFOV and the vertical field of view 10VFOV) sent by the light emitter of the LiDAR device (e.g., 90). The LiDAR device can be programmed to scan the pulsed light along the bidirectional / two-dimensional scanning pattern 10SP, which may include a sequence of pixels (e.g., [0,0] to [0,n]). In one flash, all light sources of the light emitter emit pulsed light (covering a portion of the entire FOV), and once the pulsed light hits an object, it will be reflected from the object and then captured by the light detector 230d (covering a portion of the entire FOV). A single cycle / frame of the bidirectional / two-dimensional scanning pattern 10SP may include (m+1)×(n+1) pixels, where y=(n+1).

[0087] The LiDAR device 10 can be implemented with the LiDAR device 20, 50, 60 or 90. For details or variations of the beam steering unit, light emitter, light source, light receiver or light detector, reference can be made to U.S. Patent Application Nos. 18 / 084,562 and 17 / 900,864, the disclosures of which are incorporated herein by reference in their entirety and become part of this specification. The judgment circuit (e.g., 250), the storage circuit 170, the event detector (e.g., 290), the extraction circuit (e.g., 250E) or the processing circuit 10CPU can be implemented with a combination of software, firmware and / or hardware. Variables i, j, m, n, x or y can be any positive integer.

[0088] The descriptions of "first", "second", etc. mentioned throughout the specification are only used to distinguish different components, and do not limit the order, priority, time sequence of method execution, or the existence of all components. The embodiments can be combined in various ways without conflict.

[0089] In summary, the event-based LiDAR device can select meaningful point cloud data (one or some pixels of one or some frames) from all the point cloud data acquired by the event-based LiDAR device. Once the event-based LiDAR device identifies any meaningful point cloud data, the event-based LiDAR device can immediately send the meaningful point cloud data for timely detection and processing. Therefore, the processing circuit can quickly identify potential anomalies by analyzing the meaningful point cloud data received from the event-based LiDAR device, and provide appropriate instructions or valuable data for further analysis. In addition, the present application reduces redundant data transmission.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A light detection and ranging device, characterized in that: include: A light emitter, used to emit pulsed light, wherein the pulsed light is invisible light; a light receiver optically coupled to the light transmitter and configured to capture reflected pulsed light, wherein the reflected pulsed light represents the pulsed light reflected by at least one object; a determination circuit coupled to the light transmitter and the light receiver and used to determine a plurality of distances relative to the at least one object and a plurality of reflectivities of the at least one object; and an event detector, coupled to the determination circuit, and used to obtain a first current distance of a first pixel of a current frame and a first current reflectivity of the first pixel of the current frame, wherein the event detector is used to output the first current distance only after determining that a first distance difference between the first current distance of the first pixel of the current frame and a first previous distance of a first pixel of a previous frame exceeds a distance threshold, or to output the first current reflectivity only after determining that a first reflectivity difference between the first current reflectivity of the first pixel of the current frame and a first previous reflectivity of a first pixel of a previous frame exceeds a reflectivity threshold, The multiple distances include the first current distance, the multiple reflectivities include the first current reflectivity, and a plurality of pixels including the first pixel are regularly mapped to the two-dimensional field of view of the light detection and ranging device.

2. The light detection and ranging device according to claim 1, characterized in that: Also includes: a beam steering unit, optically coupled to the light emitter and the light receiver, and used to steer the pulse light and the reflected pulse light so that the light emitter scans the two-dimensional field of view, The pulse light incident on the beam guiding unit is parallel or coaxial with the reflected pulse light redirected by the beam guiding unit, or the pulse light redirected by the beam guiding unit is parallel or coaxial with the reflected pulse light incident on the beam guiding unit. Wherein, the light emitter comprises at least one light source for emitting the pulsed light, The optical receiver includes at least one optical detector for capturing the reflected pulse light. The event detector determines, substantially pixel by pixel, whether a plurality of distance differences, including the first distance difference, exceeds the distance threshold or whether a plurality of reflectivity differences, including the first reflectivity difference, exceeds the reflectivity threshold.

3. The light detection and ranging device according to claim 1, characterized in that: The light emitter includes a plurality of light sources for emitting the pulsed light to flash the entire two-dimensional field of view at one time, The optical receiver includes a plurality of optical detectors for receiving the reflected pulse light from the two-dimensional field of view at one time. The event detector determines whether a plurality of distance differences, including the first distance difference, or a plurality of reflectivity differences, including the first reflectivity difference, exceeds the distance threshold or exceeds the reflectivity threshold for substantially all of the plurality of pixels at once.

4. The light detection and ranging device according to claim 1, characterized in that: Also includes: a beam steering unit, optically coupled to the light emitter or the light receiver, and used to steer the pulse light and the reflected pulse light so that the light emitter scans a portion of the two-dimensional field of view, The light emitter includes a plurality of light sources for emitting the pulse light to flash a portion of the two-dimensional field of view at one time, The optical receiver includes a plurality of optical detectors for capturing the reflected pulse light from a portion of the two-dimensional field of view at one time, The event detector determines whether a plurality of distance differences, including the first distance difference, exceeds the distance threshold or whether a plurality of reflectivity differences, including the first reflectivity difference, exceeds the reflectivity threshold substantially group by group.

5. The light detection and ranging device according to claim 1, characterized in that: The event detector is used for not outputting the second current distance or the second current reflectivity after determining that a second distance difference between a second current distance of a second pixel of the current frame and a second previous distance of the second pixel of the previous frame does not exceed the distance threshold and determining that a second reflectivity difference between a second current reflectivity of the second pixel of the current frame and a second previous reflectivity of the second pixel of the previous frame does not exceed the reflectivity threshold, The plurality of pixels include the second pixel.

6. The light detection and ranging device according to claim 1, characterized in that: Also includes: A storage circuit is coupled between the event detector and the judgment circuit, wherein the storage circuit includes a plurality of memory units, and the plurality of memory units are respectively used to store the plurality of distances and the plurality of reflectivities in the entire two-dimensional field of view.

7. The light detection and ranging device according to claim 1, characterized in that: The judgment circuit comprises: a converter for measuring a distance-related time difference between a reception timing of the reflected pulse light corresponding to the first pixel and a transmission timing of the pulse light corresponding to the first pixel or a reflectivity-related time difference between a rising edge of a photovoltage and a falling edge of the photovoltage, The converter is used to convert the distance-related time difference into the first current distance or convert the reflectivity-related time difference into the first current reflectivity, and the rising edge of the photovoltage is associated with the receiving timing.

8. The light detection and ranging device according to claim 1, characterized in that: The determination circuit includes at least one extraction circuit, and each of the at least one extraction circuit includes: an amplifier coupled to one of the at least one photodetector of the light receiver and configured to convert a photocurrent into a photovoltage, wherein the photodetector is configured to convert a light intensity of the reflected pulse light corresponding to the first pixel into the photocurrent; and A comparator is coupled to the amplifier and is used to determine a falling edge of the photovoltage or a rising edge of the photovoltage.

9. The light detection and ranging device according to claim 1, characterized in that: The judgment circuit comprises: A converter is coupled to one of the at least one photodetector of the light receiver and is used to convert a light intensity of the reflected pulse light corresponding to the first pixel into the first current reflectivity.

10. The light detection and ranging device according to claim 1, characterized in that: The judgment circuit comprises: A comparator is coupled to the light emitter and is used to determine a sending timing of the pulse light corresponding to the first pixel.

11. An event detection method for a light detection and ranging device, characterized in that: The event detection method comprises: Acquire a first current distance of a first pixel of a current frame or a first current reflectivity of the first pixel of the current frame; and the first current distance is outputted only after it is determined that a first distance difference between the first current distance of the first pixel of the current frame and a first previous distance of a first pixel of a previous frame exceeds a distance threshold, or the first current reflectivity is outputted only after it is determined that a first reflectivity difference between the first current reflectivity of the first pixel of the current frame and a first previous reflectivity of a first pixel of a previous frame exceeds a reflectivity threshold, Wherein, a plurality of pixels including the first pixel are regularly mapped to the two-dimensional field of view of the light detection and ranging device.

12. The event detection method according to claim 11, characterized in that: Also includes: determining substantially pixel by pixel whether a plurality of distance differences exceed the distance threshold or a plurality of reflectivity differences exceed the reflectivity threshold, wherein the plurality of distance differences include the first distance difference and the plurality of reflectivity differences include the first reflectivity difference, Wherein, a light emitter of the light detection and ranging device includes at least one light source for emitting pulsed light, and the pulsed light is invisible light. Wherein, a light receiver of the light detection and ranging device includes at least one light detector for capturing reflected pulse light, wherein the reflected pulse light represents the pulse light reflected by at least one object, A beam guiding unit of the optical detection and ranging device is used to steer the pulse light and the reflected pulse light so that the light emitter scans the two-dimensional field of view. The pulse light incident on the beam guiding unit is parallel or coaxial with the reflected pulse light redirected by the beam guiding unit, or the pulse light redirected by the beam guiding unit is parallel or coaxial with the reflected pulse light incident on the beam guiding unit.

13. The event detection method according to claim 11, characterized in that: Also includes: determining, for all of the plurality of pixels substantially at once, whether a plurality of distance differences exceed the distance threshold or whether a plurality of reflectivity differences exceed the reflectivity threshold, wherein the plurality of distance differences include the first distance difference, and the plurality of reflectivity differences include the first reflectivity difference, Wherein, a light emitter of the light detection and ranging device includes a plurality of light sources for emitting pulse light to flash the entire two-dimensional field of view at one time, wherein the pulse light is invisible light. An optical receiver of the optical detection and ranging device includes a plurality of optical detectors for receiving reflected pulse light from the two-dimensional field of view at one time, wherein the reflected pulse light represents the pulse light reflected by at least one object.

14. The event detection method according to claim 11, characterized in that: Also includes: determining whether a plurality of distance differences exceeds the distance threshold or whether a plurality of reflectivity differences exceeds the reflectivity threshold substantially group by group, wherein the plurality of distance differences include the first distance difference and the plurality of reflectivity differences include the first reflectivity difference, Wherein, a light emitter of the light detection and ranging device includes a plurality of light sources for emitting pulse light to flash a portion of the two-dimensional field of view at one time, wherein the pulse light is invisible light. A light receiver of the light detection and ranging device includes a plurality of light detectors for capturing reflected pulse light from a portion of the two-dimensional field of view at one time, wherein the reflected pulse light represents the pulse light reflected by at least one object. A beam guiding unit of the optical detection and ranging device is used to steer the pulse light and the reflected pulse light so that the light emitter scans a portion of the two-dimensional field of view.

15. The event detection method according to claim 11, characterized in that: Also includes: Acquire a second current distance of a second pixel of the current frame and a second current reflectivity of the second pixel of the current frame; as well as After determining that a second distance difference between the second current distance of the second pixel of the current frame and a second previous distance of the second pixel of the previous frame does not exceed the distance threshold and determining that a second reflectivity difference between the second current reflectivity of the second pixel of the current frame and a second previous reflectivity of the second pixel of the previous frame does not exceed the reflectivity threshold, not outputting the second current distance or the second current reflectivity, The plurality of pixels include the second pixel.

16. The event detection method according to claim 11, characterized in that: A storage circuit of the light detection and ranging device includes multiple memory units, which are used to store multiple distances and multiple reflectivities in the entire two-dimensional field of view, respectively. The multiple distances include the first current distance, and the multiple reflectivities include the first current reflectivity.

17. The event detection method according to claim 11, characterized in that: A converter of the light detection and ranging device is used to measure a distance-related time difference between a reception timing of a reflected pulse light corresponding to the first pixel and a transmission timing of a pulse light corresponding to the first pixel, or a reflectivity-related time difference between a rising edge of a photovoltage and a falling edge of the photovoltage, The converter is used to convert the distance-related time difference into the first current distance or convert the reflectivity-related time difference into the first current reflectivity, and the rising edge of the photovoltage is associated with the receiving timing.

18. The event detection method according to claim 11, characterized in that: One of at least one photodetector of a photoreceiver of the light detection and ranging device is used to convert a light intensity of the reflected pulse light corresponding to the first pixel into a photocurrent, One of the at least one amplifier of the light detection and ranging device is coupled to the light detector and is used to convert the photocurrent into a photovoltage, One of the at least one comparator of the light detection and ranging device is used to determine a falling edge of the photovoltage or a rising edge of the photovoltage.

19. The event detection method according to claim 11, characterized in that: A converter of the light detection and ranging device is used to convert a light intensity of the reflected pulse light corresponding to the first pixel into the first current reflectivity.

20. The event detection method according to claim 11, characterized in that: A comparator of the light detection and distance measurement device is used to determine a sending timing of the pulse light corresponding to the first pixel.