Lidar system and method for sensing surroundings

Abstract

The invention relates to a lidar system (10) and a method for sensing surroundings. The lidar system (10) comprises: a transmitting unit (12) for emitting light (L), a receiving unit (14), which has an optical receiving sensor (22) for receiving the reflected light (L) in a pixel-based manner and for outputting a sensor signal, which comprises pixels (Px), an evaluation unit (16), which is designed to determine evaluation data which depend on a superpixel-based evaluation of the sensor signal, wherein the evaluation data are additionally dependent on a pixel-based evaluation and / or a further superpixel-based evaluation, wherein pixel binning of a plurality of pixels (Px) is provided to generate the at least one superpixel (24). The invention further relates to a use of the lidar system (10) in a vehicle (100).

Description

[0001] LIDAR SYSTEM AND METHOD FOR ENVIRONMENTAL SENSING

[0002] Technical area

[0003] The application relates to a lidar system and a method for environmental detection, in particular for a vehicle, as well as to the use of the lidar system in a vehicle.

[0004] background

[0005] Modern vehicles (cars, vans, trucks, motorcycles, etc.) are equipped with a multitude of sensor systems whose data is used to inform drivers and / or provide them to driver assistance systems. These sensor systems record the vehicle's surroundings and other road users. Based on the recorded data, a model of the vehicle's environment can be created, and changes in this environment can be responded to.

[0006] Sensor systems are constantly being developed for various functions, e.g., for capturing environmental information in the near and far range of vehicles, such as passenger cars or commercial vehicles. Sensor systems can also be used for driver assistance systems, particularly assistance systems for autonomous or semi-autonomous vehicle control. They can be used, in particular, to detect obstacles and / or other road users in the front, rear, or blind spot area of ​​a vehicle. Sensor systems can be based on various sensor principles, such as radar, ultrasound, and optics.

[0007] An important optical sensor principle for environmental detection, e.g., of vehicles, is Lidar technology (Lidar, or Light Detection and Ranging). A Lidar system has an optical transmitter unit and an optical receiver unit. The transmitter unit can emit an optical transmission signal in the form of light, which can be continuous or pulsed. The optical transmission signal can also be modulated. In a Lidar sensor, light in the form of laser beams in the ultraviolet, visible, or infrared range can be used. The receiver unit can receive the light after it has been reflected from an object in a detection area in the vicinity of the Lidar sensor. The optical reception signal can be evaluated using the optical transmission signal, e.g., using a time-of-flight method, and the spatial position and distance of the object from which the reflection occurred can be determined.In this context, reflection or reflected light is understood to mean any light that is thrown back and is intended to include, in particular, light that is thrown back by scattering or absorption-emission.

[0008] DE102019107115A1 describes a vehicle lidar system with a transmitting unit and a receiving unit. The receiving unit has an optical sensor that generates individual pixels and an evaluation unit for pixel-based evaluation of the sensor signal from the optical sensor. The optical sensor and / or the evaluation unit performs pixel binning to form at least one superpixel and performs a superpixel-based evaluation to obtain a mode with an extended detection range of the lidar system.

[0009] Overview

[0010] A lidar system for environmental detection comprises: a transmitting unit for emitting light, a receiving unit which has an optical receiving sensor for pixel-based reception of the reflected light and for outputting a sensor signal comprising pixels, an evaluation unit which is designed to determine evaluation data which depend on a superpixel-based evaluation of the sensor signal, wherein the evaluation data additionally depend on a pixel-based evaluation and / or a further superpixel-based evaluation, wherein pixel binning of a plurality of pixels is provided for generating the at least one superpixel. In particular, the evaluation can therefore depend on the pixel-based evaluation and the superpixel-based evaluation. It is also possible for the evaluation to depend on the superpixel-based evaluation and / or the further superpixel-based evaluation.In particular, it may be provided to form more than one superpixel, so that the evaluation data depend on a pixel-based evaluation and on several superpixel-based evaluations.

[0011] This allows two or more binning states to be used simultaneously for the evaluation, either by combining unbinned pixels with superpixels and evaluating them together, or by combining superpixels with different pixel arrangements and / or different sizes and evaluating them together.

[0012] The receiving sensor has an array of pixels, which can be designed, for example, to receive light and convert it into an electrical sensor signal. For this purpose, the respective pixel can have, for example, photosensitive elements, e.g. photosensitive semiconductor elements. A respective pixel or a respective group of pixels of the receiving sensor can in particular be designed to receive light from a specific direction. This directional information can also be referred to as angular information, since it indicates the angular direction in space from which the received light was reflected. A model of the environment of the lidar system can be compiled from, for example, the direction and distance of the location where the reflection occurred.

[0013] The sensor signal is an electrical signal emitted by the receiving sensor. The sensor signal comprises pixels, which contain the respective electrical sensor signal of the respective pixel of the receiving sensor. A pixel can also be referred to as a picture element. In pixel binning, two or more pixels are combined into a superpixel. The pixels combined into a superpixel are evaluated together.

[0014] The evaluation data is determined by the evaluation unit using the transmitted light and the sensor signal. The sensor signal can be evaluated on a pixel-based and additionally superpixel-based basis. Alternatively or additionally, the sensor signal can be evaluated on a superpixel-based basis using a plurality of different superpixels. The pixel-based evaluation comprises the evaluation of at least one pixel of the sensor signal, i.e. the evaluation of at least one respective electrical sensor signal of the at least one pixel. The superpixel-based evaluation comprises the evaluation of at least one superpixel, i.e. the pixels combined to form at least one superpixel are evaluated together. In the further superpixel-based evaluation, superpixels are evaluated that differ from the superpixels of the superpixel-based evaluation, e.g. in the number and / or arrangement of the pixels forming the superpixel.

[0015] The creation of superpixels represents a compromise between spatial resolution and detection range. Superpixels enable the detection of more distant objects at the cost of losing fine details of the imaged scene. In contrast, pixel-based analysis has the advantage of high resolution. This allows even small objects within the detection range to be detected. The lidar system described here allows both advantages to be combined and utilized.

[0016] By binning pixels into superpixels and evaluating superpixels, however, the detection range can be increased, allowing even more distant and / or darker objects to be detected. The use of superpixels to increase the range is described in DE102019107115A1. However, the use of superpixels can reduce the resolution, making it too low for the detection of smaller objects.

[0017] The described lidar system offers the advantage that, for example, bright and / or nearby objects can be detected with high resolution, especially without an internal sensor, while dark and / or distant objects can be detected with a lower resolution using an internal sensor. Detection can include, for example, the recognition of an object, the direction in space in which the object is located, and / or the distance to the object.

[0018] In one embodiment of the lidar system, the evaluation data for at least one pixel depends on the pixel-based evaluation and the superpixel-based evaluation. This means that the same pixel of the receiving sensor can be evaluated once on a pixel-based basis and again as part of a superpixel. Alternatively or additionally, it is also possible to evaluate the same pixel multiple times as part of different superpixels. This allows both a high resolution and a high detection range to be achieved for such multiple-evaluated pixels.

[0019] In one embodiment, the lidar system has an output interface for outputting the evaluation data determined by the evaluation unit. The evaluation data can comprise evaluation data from multiple binning states. In particular, the evaluation data dependent on the pixel-based evaluation and the evaluation data dependent on the superpixel-based evaluation can be output as a single output data set. This enables the information present in the lidar system to be output with the highest possible quality, e.g., for further processing. A particular advantage is that the output data set can already contain the evaluation data from the pixel-based evaluation and the superpixel-based evaluation. The lidar system therefore has an operating mode in which evaluation data comprises both pixel-based evaluation data and superpixel-based evaluation data.This evaluation data is then available to the receiver upon receipt of the output data set, without requiring any further requirements to be placed on the lidar system or without having to change the operating mode. Similarly, according to one embodiment, the output data set can comprise superpixel-based evaluation data combined with further superpixel-based evaluation data, wherein the further superpixels on the basis of which the further superpixel-based evaluation is performed differ from the superpixels on the basis of which the superpixel-based evaluation is performed.

[0020] Binning can be performed in the receiving sensor in the analog domain. In such an embodiment, the sensor signal can therefore comprise electrical signals for at least one pixel and for at least one superpixel.

[0021] Binning can also occur in the evaluation unit in the digital domain. In such an embodiment, the sensor signal comprises pixels with the associated electrical signal. Binning, i.e., the combination into at least one superpixel, then occurs in the evaluation unit in the digital domain. In this embodiment, binning occurs as so-called software binning following a high-resolution pixel-based measurement without binning.

[0022] Optionally, the lidar system can be equipped with a deflection device that can deflect the transmitted light in different directions. By gradually changing the direction, the detection area can be scanned.

[0023] The evaluation data can, in particular, include distance data that can be determined by the evaluation unit using an indirect time-of-flight method. Time-of-flight methods, also known as time-of-flight methods, measure the time it takes for light to travel a distance. From this time, the distance traveled and the distance to the point of reflection can then be determined. With indirect time-of-flight methods, the time is not measured directly, but indirectly, for example, by determining a phase shift of the received light compared to the transmitted light. The evaluation data forms a preliminary stage of the information obtained for the indirect time-of-flight method, in particular the phase information.These precursors of information can therefore be obtained from the combined evaluation of multiple binning states, in particular the combination of pixel-based evaluation and superpixel-based evaluation. These precursors of information can, in particular, be correlation values ​​of light signals using the transmitted and received light.

[0024] In one embodiment of the lidar system, superpixel-based evaluation is provided for distances above a predeterminable threshold. In such an embodiment, it can be provided that, for distances below the predeterminable threshold, pixel-based evaluation is carried out with high resolution, for example. This allows the near-field below the threshold to always be detected with high resolution. Above the threshold, superpixel-based evaluation can then be carried out alternatively or in addition to the pixel-based evaluation. This allows the detection range of the lidar system to be increased and the high resolution in the near field to be maintained. At the same time, several such thresholds can be provided, above which the size of the superpixel increases with increasing distance. This gradually increases the detection range and gradually decreases the resolution.

[0025] In one embodiment, the evaluation unit is configured to perform the superpixel-based evaluation for at least one pixel depending on the pixel-based evaluation. In this embodiment, a superpixel-based evaluation, e.g., including surrounding pixels, can be performed only for pixels for which no measurement result is available in the pixel-based evaluation, in order to still be able to detect any dark or distant objects that may be present.

[0026] In one embodiment of the lidar system, a detection range of the lidar system in the environment to be detected is divided into regions. The evaluation unit is configured to perform the superpixel-based evaluation for at least one predeterminable region of the detection range. This allows, for example, areas in which distant objects are expected to be specifically evaluated using superpixel-based evaluation. Such areas can be, for example, areas in which the horizon and / or buildings on the horizon are expected to be visible.

[0027] In one embodiment, the superpixel has between 2 and 9 pixels, particularly between 2 and 6 pixels. At this pixel size, the range can be increased while still maintaining an acceptable resolution.

[0028] In embodiments of the lidar system, at least one pixel can be part of more than one superpixel. In such an embodiment, the superpixel can be shifted, for example, by a small number of pixels in the pixel array. This acts like a blur in the resolution, but increases the range, and the number of superpixels can correspond to the number of pixels if the shift is chosen accordingly.

[0029] The described lidar system can be used, for example, in a vehicle. The data output by the lidar system, e.g., the output data set, can then be further used by the vehicle's control units. In particular, further processing and use in autonomous or semi-autonomous driving systems can take place. In particular, output to a central processing unit of the vehicle is also possible, where information about the vehicle and its surroundings is centrally processed.

[0030] A method for environmental detection using the described lidar system comprises:

[0031] Emitting light, pixel-based receiving of the reflected light by means of a receiving sensor and outputting a sensor signal which comprises pixels,

[0032] Determining evaluation data by evaluating the sensor signal, comprising a superpixel-based evaluation and additionally a pixel-based evaluation and / or a further superpixel-based evaluation, wherein pixel binning of a plurality of pixels is provided for generating the at least one superpixel.

[0033] The method offers the advantage that, for example, bright and / or nearby objects can be detected with high resolution without using an internal sensor, while dark and / or distant objects can be detected with a lower resolution using an internal sensor. Detection can include, for example, identifying an object, the direction in space in which the object is located, and / or the distance to the object. Alternatively, it is possible to detect bright or nearby objects with a superpixel with a small number of pixels and to detect dark or distant objects with another superpixel that is larger than the superpixel for bright or nearby objects.

[0034] In one embodiment of the method, the evaluation data for at least one pixel is determined as a function of the superpixel-based evaluation and additionally of the pixel-based evaluation and / or the further superpixel-based evaluation. In such an embodiment, the sensor signal for a respective pixel can be evaluated multiple times, as an individual pixel or as a component of one or more superpixels.

[0035] In one embodiment of the method, the evaluation data dependent on the pixel-based evaluation and the evaluation data dependent on the superpixel-based evaluation are output as a single output data set. This allows the output data of the lidar system to have both high resolution and a high detection range. In one embodiment of the method, pixel binning is performed digitally. This so-called software binning has the advantage that the hardware side, i.e., the receiving sensor, does not need to be modified, compared to unbinned use.

[0036] In one embodiment of the method, a distance measurement is performed using an indirect time-of-flight method using the evaluation data. In particular, phase information on transmitted and received light can be obtained by combining the evaluation of pixels with multiple binning states.

[0037] Fiourenliste

[0038] In the following, embodiments of this application are further explained and described with reference to the figures.

[0039] Fig. 1 schematically shows a vehicle with Lidar system and detection range,

[0040] Fig. 2 schematically shows an embodiment of a receiving sensor with pixels and superpixels,

[0041] Fig. 3 schematically shows another embodiment of a receiving sensor with pixels and superpixels,

[0042] Fig. 4 schematically shows a method for operating the lidar system.

[0043] The same reference numerals are used throughout the figures to refer to identical or similar elements. Representations in the figures may not be to scale.

[0044] Figure description

[0045] Figure 1 schematically shows a vehicle 100, for example a passenger car, with a lidar system 10. The lidar system 10 is arranged in the front area of ​​the vehicle 100. The detection area 20 of the lidar system 10 is located in front of the vehicle in the direction of travel. An object O is schematically shown in the detection area 20. The lidar system 10 has a transmitting unit 12 for emitting light L. The transmitting unit 12 can, in particular, have a light source for emitting laser light. The lidar system 10 further has a receiving unit 14 for receiving the reflected light L. In an evaluation unit 16, the transmitted light and received light can be evaluated in order to detect the object O in the receiving area 20.

[0046] The illustrated lidar system 10 is designed as a scanning lidar system. The lidar system 10 has a deflection device 18, which deflects light L transmitted by the transmitting unit 12 and deflects light L reflected from the detection area 20 toward the receiving unit 14. The deflection device 18 can change the angle at which the light L is deflected, thus enabling, for example, a step-by-step scanning of the detection area 20. A possible scanning movement of the light L is shown in Figure 1 by the arrow 26.

[0047] The receiving unit 14 has a receiving sensor 22 for receiving the light L. The receiving sensor 22 has pixels Px, by means of which the light L can be converted into an electrical sensor signal.

[0048] Figure 1 also shows a computing unit 30 of the vehicle 100, which can be configured, for example, as the central computing unit 30 of the vehicle 100. The computing unit 30 is connected to the lidar system 10 via a data interface, via which output data from the lidar system 10 can be output to the computing unit 30 of the vehicle 100.

[0049] The computing unit 30 can, for example, be configured as a central computing unit 30 of the vehicle 100, in which data from multiple sensor systems of the vehicle 100 can be received, evaluated, and further processed. The computing unit 30 can, for example, be used to implement autonomous or semi-autonomous driving functions.

[0050] The evaluation unit can evaluate the light L that is transmitted into the detection area 20 and received from the detection area 20. The evaluation unit can also monitor and control the transmission process in the transmission unit 12, the reception process in the reception unit 14, and, if applicable, the deflection effect of the deflection device 18.

[0051] In the example shown, an area in front of the vehicle 100 in the direction of travel can be monitored. It is also possible to arrange the lidar system 10 in other areas of the vehicle 100, for example, in the rear area and / or in the side areas. It is also possible to arrange multiple lidar systems 10 on the vehicle 100, in particular in corner areas of the vehicle 100.

[0052] With the lidar system 10, stationary or moving objects O, in particular vehicles, persons, animals, plants, obstacles, road surface irregularities, in particular potholes or stones, road markings, traffic signs, open spaces, in particular parking spaces, precipitation or the like, can be detected in the detection area 20.

[0053] Figure 2 shows a receiving sensor 22 with the pixels Px. The pixels Px can convert received light L into electrical signals and output them as electrical sensor signals. The electrical sensor signals can then be evaluated, for example, in relation to the individual pixels Px. Such an evaluation can, for example, be carried out in relation to a threshold, whereby it is then detected whether the amount of light received, i.e. the number of photons received, exceeds the certain threshold value or not. The result of the evaluation can then, for example, receive the value 0 or 1. Alternatively or additionally, superpixels 24 can be formed from a group of pixels Px. In a superpixel 24, the pixels Px that belong to the superpixel 24 are evaluated together. This can take place, for example, in the digital domain in the evaluation unit 16.For evaluation in the digital domain, the sensor signals for each individual pixel Px are transferred to the evaluation unit 16 and evaluated there on a pixel-based and / or superpixel-based basis.

[0054] In the example shown in Figure 2, each superpixel 24 comprises four pixels Px, and the individual superpixels 24 do not overlap. The example shown corresponds to 2x2 binning. Asymmetric binning NxM is also possible, for example, where values ​​between 1 and 3 for N and M are advantageous, because the resolution remains quite good despite the binning. Example superpixels 24 comprise 2x2 pixels Px, 2x3 pixels Px, 2x1 pixel Px, or 1x3 pixels Px.

[0055] Figure 3 shows another example of a receiving sensor 22, in which superpixels 24 are also formed from pixels Px. As in Figure 2, the superpixels 24 have a size of 2x2. However, as in Figure 2, other sizes are also conceivable. In the exemplary embodiment of Figure 3, the superpixels 24 overlap. They are each shifted by one pixel Px relative to one another. A single pixel Px can therefore belong to several different superpixels 24. Here, shifting the superpixel 24 by one pixel Px corresponds to blurring the detection. However, the total number of superpixels 24 can be at least approximately the same size as the number of pixels Px of the receiving sensor 22.

[0056] Figure 4 schematically illustrates a method for environmental detection using the described lidar system 10. In step S1, light L is emitted by the transmitting unit 12. The light L reflected from the detection area 20 is received pixel-by-pixel in step S2. In step S3, the sensor signal is output, which comprises pixels Px. This means that a sensor signal is assigned to the respective pixels Px. In step S4, the sensor signal is evaluated. The evaluation in step S4 comprises the superpixel-based evaluation and additionally a pixel-based evaluation and / or another superpixel-based evaluation.

[0057] The superpixel 24 was generated by binning multiple pixels Px, as described with respect to Figure 2 or Figure 3. The additional superpixel was also generated by binning multiple pixels Px, whereby the size and / or shape of the additional superpixel differs from the superpixel 24.

[0058] The binning of the multiple pixels Px is preferably carried out digitally in the evaluation unit 16 using software. This has the advantage that the sensor signal of the receiving sensor 22 can comprise pixels Px, which are then combined later in the evaluation unit 16 to form superpixels 24 and possibly further superpixels for evaluation.

Claims

CLAIMS 1. Lidar system (10) for environmental detection, comprising: a transmitting unit (12) for emitting light (L), a receiving unit (14) which has an optical receiving sensor (22) for pixel-based receiving of the reflected light (L) and for outputting a sensor signal which comprises pixels (Px), an evaluation unit (16) which is designed to determine evaluation data which depend on a superpixel-based evaluation of the sensor signal, wherein the evaluation data additionally depend on a pixel-based evaluation and / or a further superpixel-based evaluation, wherein pixel binning of a plurality of pixels (Px) is provided for generating the at least one superpixel (24).

2. Lidar system according to claim 1, wherein the evaluation data for at least one pixel (Px) depend on the superpixel-based evaluation and additionally on the pixel-based evaluation and / or the further superpixel-based evaluation.

3. Lidar system according to claim 1 or 2, comprising an output interface for outputting the evaluation data determined by the evaluation unit (16), wherein it is provided that the evaluation data dependent on the pixel-based evaluation and the evaluation data dependent on the superpixel-based evaluation are output as an output data set.

4. Lidar system according to one of the preceding claims, wherein the evaluation unit (16) is provided for carrying out pixel binning.

5. Lidar system according to one of the preceding claims, wherein an increased detection range of the lidar system (10) can be achieved by the superpixel-based evaluation.

6. Lidar system according to one of the preceding claims, wherein the evaluation data comprises distance data.

7. Lidar system according to claim 6, wherein the evaluation unit (16) is designed to determine the distance data by means of an indirect light transit time method.

8. Lidar system according to claim 6 or 7, wherein the superpixel-based evaluation is provided for distances above a predeterminable threshold value.

9. Lidar system according to one of the preceding claims, wherein the evaluation unit (16) is designed to carry out the superpixel-based evaluation for at least one pixel (Px) as a function of the pixel-based evaluation.

10. Lidar system according to one of the preceding claims, wherein a detection area (20) of the lidar system (10) is divided into areas, and the evaluation unit is designed to carry out the superpixel-based evaluation for at least one predeterminable area of the detection area (20).

11. Lidar system according to one of the preceding claims, wherein the superpixel (24) has between 2 and 9 pixels (Px), in particular between 2 and 6 pixels (Px).

12. Lidar system according to one of the preceding claims, wherein at least one pixel (Px) is part of more than one superpixel (24).

13. Use of the lidar system (10) according to one of the preceding claims in a vehicle (100).

14. A method for environmental detection using a lidar system (10), comprising: Emitting light (L), pixel-based receiving of the reflected light (L) and outputting a sensor signal comprising pixels (Px), Determining evaluation data by evaluating the sensor signal, comprising a superpixel-based evaluation and additionally a pixel-based evaluation and / or a further superpixel-based evaluation, wherein pixel binning of a plurality of pixels (Px) is provided for generating the at least one superpixel (24).

15. The method according to claim 13, wherein the evaluation data for at least one pixel (Px) are determined as a function of the superpixel-based evaluation and additionally of the pixel-based evaluation and / or the superpixel-based evaluation.

16. The method according to claim 14 or 15, wherein the evaluation data dependent on the pixel-based evaluation and the evaluation data dependent on the superpixel-based evaluation are output as an output data set.

17. The method according to any one of claims 14 to 16, wherein the pixel binning is performed in the digital domain.

18. Method according to one of claims 14 to 17, wherein a distance measurement is carried out by means of an indirect light transit time method using the evaluation data.

Images