Lidar apparatus and method for operating a lidar apparatus

Abstract

The present invention relates to a method for operating a lidar apparatus (10) of a vehicle (12), wherein in the method, a transmission unit (14) of the lidar apparatus (10) emits laser beams into a total capture region (13) of the lidar apparatus, and scans the capture region (13) within a scanning cycle with an associated refresh rate. A receiving unit (16) of the lidar apparatus also scans the capture region (13) with the refresh rate synchronously to the transmission unit (14) and converts laser beams reflected by objects in surroundings (11) into electric signals. A control unit (20) evaluates the electric signals, wherein the control unit (20) uses an object identification algorithm to classify an object as an identified object if the object was determined several times based on defined criteria. According to the invention, following a first-time detection of an object, the control unit (20) controls the transmission and receiving unit (14, 16) such that they capture the direction of the first detection again at least once, preferably multiple times, within a shorter period, which is shorter than the scanning cycle.

Description

[0001] Lidar device and method for operating a lidar device

[0002] The present invention relates to a method for operating a lidar device according to the preamble of claim 1, as well as a lidar device according to the preamble of claim 10.

[0003] In a generic method for operating a lidar device of a vehicle, a transmitting unit of the lidar device emits laser beams into a detection area of ​​the lidar device and scans the detection area within a scanning cycle with an associated frame rate.

[0004] A receiver unit of the lidar device also scans the detection area at the frame rate synchronously with the transmitter unit and converts laser beams reflected from objects in the surrounding area into electrical signals. These electrical signals are evaluated by a control unit of the lidar device, which uses an object recognition algorithm to classify an object as an identified object if it has been detected multiple times according to defined criteria.

[0005] Object detection means that the control unit uses an algorithm to determine, based on the propagation time of a reflected laser pulse emitted by the transmitting unit and reflected by an object, that the object is a potential object. An object can be another vehicle, a pedestrian, a road barrier, or an obstacle.

[0006] Known vehicles with a lidar device use it to scan an environment and, in particular, to detect objects in that environment. Lidar devices for automotive applications typically emit laser beams in a wavelength range invisible to the human eye. This is usually infrared radiation with a wavelength of a few nanometers in the spectral range of 800 nm to 2500 nm. If the individual laser beam is reflected back to the lidar device by an object in the environment, such as a vehicle, the travel time of the individual laser beam can be used to determine the distance between the lidar system and the object.

[0007] By repeatedly emitting the laser beam in different directions, an area of ​​the environment can be scanned or sampled. The scanned area of ​​the environment is also called the field of view or detection range of the lidar system. The reflection points in the environment that have reflected the laser beam from the lidar device are combined into a so-called lidar point cloud. This lidar point cloud serves as a representation of the environment.

[0008] The range of such a lidar device is one of the factors that limit the driving speed in autonomous driving. In autonomous driving, for example, in Level 3 mode, the driver does not need to constantly monitor the system. The vehicle performs functions such as activating the turn signals, changing lanes, and keeping in lane independently. The driver can attend to other tasks but will be prompted by the system to resume driving if necessary within a pre-warning period.

[0009] When it comes to range, a distinction must be made between the initial detection range and the range of an identified and / or classified and stably usable object. The latter is also referred to as the "range of identified objects." The initial detection range is the range within which a potential object is initially detected. This means that laser beams emitted by a transmitter unit of a lidar device into a detection area of ​​the lidar device are initially reflected by an object, detected by a receiver unit of the lidar device, and converted into an electrical signal. Such an initial detection range is a hardware property and depends, for example, on the laser power with which the lidar device emits laser beams.For static scenes, i.e., a stationary vehicle or stationary objects, this initial detection range is independent of the lidar device's repetition rate or frame rate. The repetition rate or frame rate is the number of images captured by the lidar device during cyclic scanning of the detection area within a specified time frame. The frame rate is usually specified in frames per second (FPS) or in Hertz (Hz).

[0010] For dynamic scenes, i.e., for a moving vehicle and / or moving objects, the frame rate is highly relevant for object identification. The "range to identified objects" depends significantly on this frame rate. The more often the detection area of ​​the lidar device is scanned per second, the faster a potential object can be verified and identified. A second factor for the "range to identified objects" is the criterion used by an object detection algorithm to confirm a potential object as an identified object. Typically, criteria such as a specified number of detections of a potential object within a similarly specified number of consecutively recorded individual images of the detection area are used here.Such possible criteria applicable to the present invention are specified as, for example, "2 times out of 3 frames," abbreviated 2oo3, "4 times out of 5 frames," abbreviated 4oo5, "3 times out of 4 frames," abbreviated 3oo4. Alternative criteria can also be 3oo7, 3oo5. Thus, a potential object is only classified as an identified object after multiple repeated detections.

[0011] The frame rate is therefore correspondingly critical, since, for example, with a 4oo5 criterion, the detection area must be scanned at least four times. The frame rates used in the present invention can be between 10 and 50 FPS, preferably between 15 and 25 FPS. With a frame rate of, for example, 15 FPS, an image is captured approximately every 66 ms, and it therefore takes 264 ms until a potential object can be classified as an identified object at the earliest. This optimal case corresponds to detection of the potential object in all four consecutive images captured within the 264 ms. From these considerations, it is clear that, in addition to the laser power of the lidar device, this frame rate is also critical if the "range to identified objects" is to be increased. However, the frame rate is limited by the transmitting and receiving units of the lidar device.These units must scan the detection area completely and synchronously for each image or frame, which is reaching the mechanical limits of the mechanical elements used so far with movable mirrors for deflecting the emitted and reflected laser beams.

[0012] It is therefore the object of the present invention to improve known methods for operating a lidar device and known lidar devices in such a way that the “range to identified objects” is increased.

[0013] This object is achieved by a method according to the characterizing part of claim 1 and by a lidar device according to the characterizing part of claim 10.

[0014] In a method according to the invention, following an initial detection of an object, the control unit controls the transmitting and receiving units in such a way that the transmitting and receiving units detect and / or scan the direction of the initial detection at least once, preferably several times, within a shortened period of time that is shorter than the scanning cycle.

[0015] If, in a method according to the invention, an object is detected for the first time in a direction or pixel of the resulting image, the transmitting and receiving units of the lidar device are controlled in such a way that the relevant pixel is advantageously detected again "very promptly," i.e., at least earlier than the next acquired frame. Due to the resulting, very fast (fast compared to the normal frame rate) repeated scanning, the algorithmically set criterion can be met earlier than without such fast, repeated scanning. This advantageously results in an increase in the "range to identified objects" of such a lidar device in a method according to the invention.In a lidar device according to the invention, the control unit is designed to control the transmitting and receiving units following an initial detection of an object in such a way that they detect and / or scan the direction of the initial detection at least once, preferably several times, within a shortened period of time that is shorter than the scanning cycle.

[0016] Preferred embodiments of the present invention emerge from the subclaims.

[0017] In a first preferred embodiment of the present invention, following the initial detection of the object, the control unit controls the transmitting and receiving units such that, until the object is classified as an identified object, they reduce the detection area to a reduced detection area that includes the direction of the initial detection. The transmitting and receiving units scan this reduced detection area more quickly with a shortened scanning cycle.

[0018] This offers the advantage of (temporarily) reducing the detection range of the lidar device around the initial detection area of ​​the potential object in favor of a higher frame rate. This allows the object detection algorithm to classify a potential object as an identified object more quickly, since the specified criteria (e.g., 3oo4 or 4oo5) can be met more quickly. The lidar device thus temporarily zooms in on the initial detection area.

[0019] Particularly preferred is an inventive method in which, following the initial detection of the object, the control unit controls the transmitting and receiving units such that the transmitting unit emits several laser pulses in the direction of the initial detection before the expiration of the shortened period, i.e., before the expiration of the normal scanning cycle, and the receiving unit is simultaneously aligned to receive reflected laser radiation from this direction. A method according to the invention can thus be designed such that the unchanged detection area is scanned, preferably also at the unchanged frame rate, thus ensuring that the detection area is not reduced.However, during the regular scanning of the detection area, several additional laser pulses are emitted in the direction of the initial detection in order to be able to reach the criterion for classifying an object as an identified object earlier by considering laser pulses reflected from the object as (further) detections of the object.

[0020] The emission of the laser pulses in the direction of the initial detection and the reception of the reflected laser radiation can thus take place during a scan of the entire detection area and within a regular scanning cycle.

[0021] In a further preferred embodiment of the present invention, the object recognition algorithm identifies an object as an identified object and classifies it as such if a minimum number of x detections results from a number of y scanning cycles or y laser pulses emitted in the direction of the object, where x < y. For example, in the above examples of 3oo4 and 4oo5, in the first case, x = 3 and y = 4, and in the second case, x = 4 and y = 5.

[0022] In a particularly preferred embodiment of a method according to the invention, the transmitting unit and / or the receiving unit comprise a non-mechanical scanner. A non-mechanical scanner is understood to mean a scanner that does not contain any mechanically adjustable deflection elements, such as mechanically movable mirrors (in particular, galvanometer mirrors or mirrors with drive motors). However, a micro-electromechanical element integrated into a chip will be understood as non-mechanical in the following.

[0023] Such a non-mechanical scanner preferably includes phased array optics. Phased array optics is a technology for controlling the phase amplitude of light waves that are emitted, reflected, or detected (received) by a two-dimensional surface with adjustable surface elements. An optical phased array is the optical analogue of a radio wave phased array. The dynamic control of the optical properties of a surface on a microscopic scale makes it possible to vary the direction of the emitted laser beams and the orientation of the receiving unit without the need for moving parts. The absence of moving parts significantly accelerates not only the frame rate but also, in particular, the targeted multiple emission and reception of laser beams toward an initial detection.A phased array optics system can contain one or more laser light sources and phase shifters in the area of ​​the lidar device's transmitting unit. Such phase shifters can also be provided in the lidar device's receiving unit. The receiving unit can also have nanophotonic antennas to evaluate received light with phase shifts and amplitude weighting. Corresponding signals are added to a receive beam, and by adjusting the phase shift, the receive beam can be directed in different directions and capture reflected light with angle resolution. While phased array optics are known in principle, their use in lidar devices for vehicles is not.

[0024] As an alternative to phased array optics, a further method according to the invention can use a lidar device that includes a micro-electromechanical lidar system (MEMS lidar). While current lidar systems for autonomous driving have movable and / or rotating mirrors (particularly galvanometer mirrors or mirrors with drive motors), such MEMS lidar systems include microscanner mirrors in which a mirror plate is moved via electromagnetic, electrostatic, thermoelectric, or piezoelectric mechanisms. Corresponding MEMS mirrors are typically integrated on a silicon chip.

[0025] The invention is explained in more detail below using the figures as examples. These figures show:

[0026] Fig. 1 is a schematic representation of a vehicle having a lidar device; and

[0027] Fig. 2: a schematic representation of a method according to the invention. Fig. 1 shows a schematic representation of a vehicle 12 having a lidar device 10 according to the invention. The vehicle 12 is designed as a passenger car. The lidar device 10 has a transmitting unit 14 for emitting a laser beam. A detection area 13 of an environment 11 of the vehicle 12 is scanned using laser pulses. The lidar device 10 further includes a receiving unit 16, which serves to receive the laser pulses reflected back to the lidar device 10 by an object in the environment 11 and to convert them into electrical signals using a detector.

[0028] The lidar device 10 also includes an evaluation and control unit 20 for determining a representation of the environment 11 in the detection area 13 based on the electrical signals. The electrical signals are fed from the receiving unit 13 to the evaluation and control unit 20 via a signal line 18. The evaluation and control unit 20 is further connected to the transmitting unit 14 via a control line 22. Both the transmitting unit 14 and the receiving unit 16 have means for scanning the detection area 13. For example, using phased array optics, outgoing laser beams from the transmitting unit 14 can scan the detection area 13 in a raster pattern.Laser pulses reflected back from an object in the environment 11 are received by the receiving unit 16 through synchronous scanning (for example, also using phased array optics) and converted into an electrical signal, which is fed to the evaluation and control unit 20 via the signal line 18. The scanning process performed by the transmitting unit 14 and the receiving unit 16 is controlled by the evaluation and control unit 20.

[0029] Figure 2 shows the differences of a method according to the invention for operating a lidar device compared to methods known from the prior art.

[0030] The upper half of Figure 2 (referred to as "state of the art") explains an object detection algorithm based on a 4oo5 criterion (4 out of 5). A method is shown as an example using a refresh rate of 15 FPS (frames per second). The 15 frames per second correspond (reciprocal) to a time interval of 66 ms between the acquisition of two consecutive images or frames.

[0031] On the x-axis, which represents the time axis, the points in time at which a corresponding image is constructed with such a refresh rate are shown as vertical lines, i.e. after 66 ms, 132 ms, 198 ms, etc. Furthermore, the repeated detection of a potential object is symbolized by circles.

[0032] This example illustrates how, in the prior art method, the potential object is repeatedly detected during the first, second, fourth, and fifth capture of an image. Detection does not occur during the third image. Based on the 4oo5 criterion, the potential object is classified as an identified object by an object recognition algorithm after the fourth detection. As shown, a time of 264 ms (4x66 ms) elapses between the scanning of the five images to classify the object as an identified object. If the potential object had been captured during the first, second, third, and fourth frames, this required time would be reduced to 198 ms.

[0033] The method according to the present invention is shown in the lower half of Figure 2.

[0034] Here, following the initial detection of the object, according to the invention, both the transmitting unit 14 and the receiving unit 16 are controlled by the control unit 20 in such a way that the direction of the initial detection is detected or scanned several times again within a shortened period of time which is shorter than the scanning cycle of 66 ms.

[0035] For this purpose, the transmitting unit 14 transmits several laser pulses in the direction or solid angle of the initial detection (first circle symbol on the left) before the end of the scanning cycle, and the receiving unit 16 is simultaneously aligned to receive reflected laser radiation from this direction.

[0036] In the case shown, before the second scan of the entire detection area after a total of 132 ms, the angular range of the initial detection is recorded multiple times, at least three times, and the potential object is detected three times in the process. By recording the object four times between the creation of the first and second image of the detection area, the 4oo5 criterion is met even before the entire detection area has been recorded twice after 132 ms, resulting in a significant time saving. In the case shown, the time saving is greater than three cycles. If the potential object had already been detected in four consecutive cycles using the prior art method (and not missed in the third cycle), the time saving would still be greater than two cycles, i.e., greater than 132 ms at the current frame rate of 15 FPS.

[0037] By more quickly classifying potential objects as identified objects, the method according to the invention significantly improves the "range to identified objects" of a corresponding lidar device. This advantageously results in a higher possible driving speed, particularly in autonomous driving, with a method according to the invention for operating a lidar device.

[0038] List of reference symbols

[0039] 10 Lidar device

[0040] 11 Environment 12 Vehicle

[0041] 13 Detection range

[0042] 14 Transmitter unit

[0043] 16 Receiving unit

[0044] 18 Signal line 20 Control unit

[0045] 22 Control line

Claims

Claims 1. A method for operating a lidar device (10) of a vehicle (12), wherein in the course of the method: a transmitting unit (14) of the lidar device (10) emits laser beams into a detection area (13) of the lidar device, and scans the detection area (13) within a scanning cycle with an associated refresh rate, a receiving unit (16) of the lidar device scans the detection area (13) with the refresh rate synchronously with the transmitting unit (14), and converts laser beams reflected by objects in an environment (11) of the vehicle (12) into electrical signals, a control unit (20) evaluates the electrical signals, wherein the control unit (20) classifies an object as an identified object by means of an object recognition algorithm if it has been detected several times according to defined criteria, characterized in thatthat the control unit (20) following an initial detection of an object controls the transmitting and receiving units (14, 16) in such a way that they detect and / or scan the direction of the initial detection at least once, preferably several times, within a shortened period of time which is shorter than the scanning cycle.

2. Method according to claim 1, characterized in that the control unit (20) following the initial detection of the object controls the transmitting and receiving units (14, 16) in such a way that they reduce the detection area (13) until the object is classified as an identified object to a reduced detection area which includes the direction of the initial detection, and the transmitting and receiving units (14, 16) scan this reduced detection range with a shortened scanning cycle.

3. Method according to claim 2, characterized in that the control unit (20) controls the transmitting and receiving units (14, 16) following the initial detection of the object in such a way that the transmitting unit (14) emits one or more laser pulses in the direction of the initial detection before the end of the scanning cycle and the receiving unit (16) is simultaneously aligned to receive reflected laser radiation from this direction.

4. Method according to claim 3, characterized in that the emission of the laser pulses in the direction of the initial detection and the reception of the reflected laser radiation takes place during a scanning of the entire detection area.

5. Method according to claim 4, characterized in that the scanning of the entire detection area takes place within a regular scanning cycle.

6. Method according to one of the preceding claims, characterized in that the object recognition algorithm classifies an object as an identified object if a minimum number of x detections results from a number of y scanning cycles or from y laser pulses emitted in the direction of the object, where x is less than y.

7. Method according to one of the preceding claims, characterized in that the transmitting unit (14) and / or the receiving unit (16) has a non-mechanical scanner.

8. The method according to claim 7, characterized in that the non-mechanical scanner includes a phased array optics.

9. Method according to one of the preceding claims, characterized in that the lidar device includes a micro-electro-mechanical system lidar (MEMS lidar).

10. Lidar device (10) for a vehicle (12), wherein the lidar device comprises: a transmitting unit (14) configured to emit laser beams into an entire detection area (13) of the lidar device, and to scan the detection area (13) within a scanning cycle at an associated frame rate, a receiving unit (16) configured to scan the detection area (13) at the frame rate synchronously with the transmitting unit (14) and to convert laser beams reflected by objects in an environment (11) into electrical signals, a control unit (20) configured to evaluate the electrical signals, wherein the control unit (20) includes an object recognition algorithm by means of which an object can be classified as an identified object if it has been detected several times according to defined criteria, characterized in that the control unit (20) is configured toFollowing an initial detection of an object, the transmitting and receiving units (14, 16) are controlled in such a way that they detect and / or scan the direction of the initial detection at least once, preferably several times, within a shortened period of time which is shorter than the scanning cycle.

Images