Correction of a measured data set of a lidar sensor system

Abstract

The aim of the invention is to correct a measured data set (15) of a lidar sensor system (3) which comprises a detector array having multiple pixels, wherein the measured data set (15) contains measured values including one energy value and one distance value. This aim is achieved in that a cluster (10) of pixels is identified, the energy values of which pixels are greater than or equal to a limit value. On the basis of the distance values, a distance value of the cluster (10) is determined, and pixels in the surroundings (11a, 11b) of the cluster (10), the distance values of which pixels lie within a distance range around the distance value of the cluster (10), are identified as potentially defective. On the basis of the curve of the energy values of the potentially defective pixels, the potentially defective pixels are classified as defective, and the measured values thereof are removed from the measured data set (15) or are labelled as defective.

Description

[0001] Correction of a measurement data set of a lidar sensor system

[0002] The present invention relates to a computer-implemented method for correcting a measurement data set of a lidar sensor system, which has a detector array with a plurality of pixels, and wherein the measurement data set contains measured values ​​that include an energy value and a distance value, at least for the pixels of the plurality of pixels. The invention is further directed to a computer-implemented method for object detection and / or object tracking, which includes such a computer-implemented method for correcting a measurement data set of a lidar sensor system, as well as to a computer-implemented method for distance determination, which includes such a computer-implemented method for correcting a measurement data set of a lidar sensor system.The invention also relates to a method for at least partially automatically guiding a motor vehicle, which includes such a computer-implemented method for correcting a measurement data set from a lidar sensor system. Furthermore, the invention relates to a data processing device for carrying out such computer-implemented methods, to an electronic vehicle guidance system for a motor vehicle having such a data processing device, and to a corresponding computer program product.

[0003] Lidar sensor systems can be used for a wide range of driver assistance functions as well as other functions for at least partially automated driving of a motor vehicle. For this purpose, a lidar sensor system can be mounted on the motor vehicle, for example, so that measurement data sets generated by the lidar sensor system represent the external environment of the motor vehicle. It is also possible for a lidar sensor system to be mounted in an infrastructure external to the motor vehicle, for example, in a parking infrastructure, and the measurement data set represents the environment of the lidar sensor system, including, for example, the motor vehicle, other components of the infrastructure device, and / or other motor vehicles.Based on the measurement data set or based on a plurality of consecutively recorded measurement data sets of corresponding consecutive frames, control signals for at least partially automatically guiding the motor vehicle and / or assistance information to support a person driving the motor vehicle can be generated. In particular, such measurement data sets from lidar sensor systems contain information about the position of objects in the vicinity of the lidar sensor system and, in particular, about their distance from the lidar sensor system. For this purpose, the lidar sensor system emits laser pulses and, using the detector array, detects portions of the emitted laser pulses reflected by the objects in the vicinity. Known methods for direct or indirect time-of-flight measurement can be used to determine the distance of the objects accordingly.

[0004] The reflected portions of the laser pulses detected by the detector array can be distorted or distorted. This distortion or distortion results, for example, from the absorption of a small portion of the light power by the medium in the optical path, which can influence the refractive index and thus distort the laser beam. This effect is particularly noticeable when the reflected portions of the laser pulses have high optical power, as can occur, for example, when the reflecting objects are highly reflective, such as so-called retroreflectors, which can be found in traffic signs, vehicle exterior mirrors, and so on.It can happen that the detected reflected portions of the laser pulses activate not only those pixels of the detector array that reflect the actual position of the reflection point in the surrounding area, but also incorrectly activate pixels in their surroundings, so that the object is, in a sense, detected with more measurement points than would be correct or expected. This effect is known as blooming.

[0005] In camera-based image capture, approaches to detecting and eliminating blooming are known. These involve identifying overexposed areas and applying image processing filters to sharpen the camera image around these areas. However, in lidar sensor systems, not only the optical energy captured by the pixels is relevant, but also, and especially, the radial distance of the object, which can also be distorted by blooming. In particular, blooming also renders otherwise correct measurements at the position of the affected area unusable.

[0006] It is an object of the present invention to provide a possibility to correct measurement data sets generated by a lidar sensor system in order to at least partially compensate for the effects of blooming.

[0007] This problem is solved by the subject matter of the independent claim.

[0008] Advantageous further developments and preferred embodiments are the subject of the dependent claims. The invention is based on the idea of ​​identifying a cluster of pixels of the detector array whose energy values, according to the measurement data set, are greater than or equal to a predetermined limit value, and of identifying, in the vicinity of the cluster, those pixels as potentially defective whose distance values, according to the measurement data set, are similar to the distance values ​​of the pixels of the cluster. Depending on a spatial distribution of the energy values ​​of the potentially defective pixels, these are classified as actually defective, and the measurement data set is corrected by removing the corresponding measured values ​​of the pixels classified as defective or marking them as defective.

[0009] According to one aspect of the invention, a computer-implemented method for correcting a measurement data set of a lidar sensor system, in particular a lidar sensor system of a motor vehicle or an electronic vehicle guidance system for the motor vehicle, is specified. The lidar sensor system has a detector array with a plurality of pixels. The measurement data set is obtained, in particular from the lidar sensor system, and the measurement data set contains respective measurement values ​​for at least the pixels of the plurality of pixels of the detector array. The measurement values ​​include a respective energy value and a respective distance value. A cluster of one or more adjacent pixels of the detector array is identified, wherein the respective energy values ​​of the pixels of the cluster are each greater than or equal to a predetermined limit value.Based on the respective distance values ​​of the pixels of the cluster, a distance value of the cluster is determined, and pixels in an environment, in particular a predefined environment, of the cluster are identified as potentially defective pixels if their respective distance values ​​lie within a distance range, in particular a predefined distance range, around the distance value of the cluster. The potentially defective pixels are classified as defective, in particular as actually defective, depending on a spatial progression of the energy values ​​of the potentially defective pixels. The measurement data set is corrected by removing the measured values ​​of the pixels in the environment of the cluster classified as defective from the measurement data set or by marking them as defective in the measurement data set. The corrected measurement data set is, in particular, saved.

[0010] Unless otherwise stated, all steps of the computer-implemented method can be performed by a data processing device having at least one computing unit, in particular by a data processing device of the motor vehicle. In particular, the at least one computing unit is configured or adapted to perform the steps of the computer-implemented method. For this purpose, the at least one computing unit can, for example, store a computer program containing instructions that, when executed by the at least one computing unit, cause the at least one computing unit to execute the computer-implemented method.

[0011] All computing units of the at least one computing unit can be part of the motor vehicle. However, it is also possible for all computing units of the at least one computing unit to be part of an external computing system outside the motor vehicle, for example, a backend server or a cloud computing system. It is also possible for the at least one computing unit to comprise both at least one vehicle computing unit of the motor vehicle and at least one external computing unit of the external computing system. The at least one vehicle computing unit can, for example, be comprised of one or more control units (ECUs) (electronic control units), and / or one or more zone control units (ZCLIs), and / or one or more domain control units (DCLIs) of the motor vehicle and / or the lidar sensor system.

[0012] For each embodiment of a computer-implemented method according to the invention, a corresponding embodiment of a method for correcting a measurement data set of a lidar sensor system is obtained, which is not purely computer-implemented, by including a method step of generating the measurement data set by means of the lidar sensor system.

[0013] A well-known type of lidar system is a so-called laser scanner, in which a laser beam is deflected by a light deflection device, allowing different deflection angles of the laser beam to be achieved. The light deflection device can, for example, contain a rotatably mounted mirror. Alternatively, the light deflection device can have a mirror element with a tiltable and / or pivotable surface. The mirror element can, for example, be designed as a microelectromechanical system (MEMS). The emitted laser beams can be partially reflected in the environment, and the reflected portions can in turn strike the laser scanner, in particular the light deflection device, which can direct them onto a detector unit of the laser scanner. Each optical detector of the detector unit generates, in particular, an associated detector signal based on the portions detected by the respective optical detector.Based on the spatial arrangement of the respective detector, together with the current position of the light deflection device, in particular its rotational position or its tilt and / or pivot position, the direction of incidence of the detected reflected components can be determined. An evaluation unit can also, for example, perform a time-of-flight measurement to determine the radial distance of the reflecting object. Alternatively or additionally, a method can be used to determine the distance by evaluating a phase difference between emitted and detected light.

[0014] Other types of lidar systems are flash lidar systems. These are non-scanning systems that do not require such a light deflection arrangement. The laser light generated by the light source is scattered by an optical element, so that it is emitted in a single flash across a wide angle.

[0015] The measurement data set can, in particular, correspond to a frame of the lidar sensor system—in the case of a laser scanner, for example, a complete scan run. In particular, the measurement data set contains precisely one set of measurement values ​​for at least some of the plurality of pixels, which includes the respective distance value and the respective energy value.

[0016] The distance value corresponds, in particular, to a radial distance of the respective reflecting object or the point in the surrounding area that leads to the respective reflection, the reflected portions of which are detected by the corresponding pixel. The distance value can be determined, for example, by applying a method for direct time-of-flight measurement or indirect time-of-flight measurement. The energy value is, in particular, a measure of the optical energy detected by the respective pixel based on the corresponding reflected portion of the laser pulses.However, the energy value does not have to be given directly as energy, but can also be a value of another quantity that concerns the detected energy or depends on it, such as an echo pulse width, EPW, of a corresponding signal pulse of a detector signal of the respective pixel, an area under the signal pulse or a part of the area under the signal pulse and so on.

[0017] It is possible that individual pixels of the detector array do not detect reflected portions of emitted laser pulses. Nevertheless, corresponding energy values ​​or distance values ​​can also be stored in the measurement data set for such pixels. These values ​​can then be standard values ​​such as 0, °°, or n / a, or the like, and have no physical or technical significance regarding an object in the vicinity of the lidar sensor system.

[0018] As mentioned above, a cluster contains one or more clusters. A broad understanding of the term "cluster" is used, meaning that, unless otherwise stated, even a single pixel can form a cluster. If the cluster consists of a single pixel, the distance value of the cluster is equal to the distance value of the corresponding pixel. If the cluster contains two or more pixels, the distance value of the cluster is calculated from the individual distance values ​​of its pixels, for example, by averaging or calculating a median. The distance value of the cluster can also correspond to a minimum distance value or a maximum distance value of the distance values ​​of its pixels.

[0019] To identify the cluster, in particular a pixel of the detector array is identified whose energy value is greater than or equal to the limit value, whereby it can be specified, for example, that the pixel is located in a certain defined sub-area of ​​the detector array, for example in a certain column or within several adjacent columns of the detector array. Once such a pixel has been identified, other pixels located next to the identified pixel are checked to see whether their energy value is also greater than or equal to the limit value. If this is the case, these can be assigned to the cluster, provided that all pixels in the resulting cluster are adjacent to one another. This process can continue until no more pixels are found, particularly within the defined sub-area, whose energy value is greater than the limit value, or until no more such directly adjacent pixels are found.

[0020] The fact that a first pixel and a second pixel of the detector array are adjacent to one another can be understood here and below to mean that the second pixel lies in a predefined neighborhood of the first pixel, for example in a neighborhood of four, an eight, or a Manhattan neighborhood. The additional condition that the first and second pixels lie within the defined sub-area of ​​the detector array, in particular in one or more adjacent columns, may apply. To identify potentially defective pixels, all pixels in the predefined vicinity of the cluster are identified whose distance values ​​lie within the predefined distance range around the distance values ​​of the cluster.The environment of the cluster can, for example, be defined such that only pixels with a distance from one of the clusters that is smaller than a predetermined maximum distance are located in the environment and / or only pixels within the same sub-area of ​​the detector array, in particular the same column or the same adjacent columns, as the cluster itself.

[0021] The environment can also be defined such that all pixels of the cluster and the pixels identified as potentially faulty are adjacent to each other and / or within the same sub-area of ​​the detector array.

[0022] In particular, potentially defective pixels are identified as potentially defective based on their distance values ​​and, for example, independently of their respective energy values. In other words, the energy values ​​of potentially defective pixels are not necessarily greater than or equal to the threshold value and are generally smaller than the threshold value. This particularly takes into account the fact that, while blooming occurs in the vicinity of pixels with very high energy input, the pixels affected by blooming themselves may also have a lower energy input.

[0023] The distance range can in particular be predefined in such a way that a value interval with a defined width is specified and the distance range corresponds to an interval for the distance values ​​with this width, wherein the distance value of the cluster lies within, for example in the middle, of the distance range.

[0024] To mark the measured values ​​of pixels classified as faulty in the measurement data set, a corresponding flag can be set for each pixel classified as faulty, for example, which identifies the measured values ​​of this pixel as faulty or invalid. Pixels not classified as faulty are accordingly not marked, or their flag is not set.

[0025] Removing the measured values ​​of pixels classified as faulty can also be referred to as filtering the measurement data set. If the measured values ​​of a pixel classified as faulty are removed from the measurement data set, this can be understood in particular as meaning that the corresponding pixels in the corrected measurement data set are treated in the same way as if no reflected portions of the emitted laser pulses had been detected at the corresponding positions. The measured values ​​are therefore replaced, in particular, by predefined standard values ​​as mentioned above.

[0026] The invention is based in particular on two essential findings. Firstly, the finding that the distance values ​​determined for pixels affected by blooming are generally relatively close to the distance value of those pixels that correspond to the measurement of a real, highly reflective object in the surrounding area and are accordingly identified as pixels of the cluster. Accordingly, according to the invention, the pixels in the vicinity of the cluster within the predefined distance range are classified as potentially faulty. However, since it cannot be ruled out per se that real objects are actually located at the corresponding locations, this is combined with the further finding that the spatial distribution of the energy values ​​of pixels affected by blooming exhibits characteristic signatures.Accordingly, the potentially defective pixels are classified as defective depending on the spatial course of the energy values ​​of the potentially defective pixels, i.e., in particular, they are only classified as defective if, on the one hand, they were previously classified as potentially defective and their energy value corresponds to a spatial course of a course expected for blooming.

[0027] This ensures, in particular, that pixels affected by blooming can be identified as such with increased reliability and, on the other hand, that the risk of pixels not affected by blooming being classified as affected by blooming or as faulty is reduced. In other words, both the risk of false positive detections of faulty pixels and the risk of false negative detections of such pixels are reduced.

[0028] The corrected measurement data set can be used by downstream functions, also known as hierarchically higher-level functions, in the same way as an uncorrected measurement data set from the lidar sensor system would be used. If correcting the measurement data set involves removing the measured values ​​of the pixels classified as faulty, this automatically ensures that the measured values ​​of the pixels classified as faulty are not taken into account when performing the downstream functions. If the measured values ​​of the pixels classified as faulty are marked as faulty, this marking can be taken into account when performing the downstream function, in particular by not using these values ​​to perform the function. In both cases, this increases the reliability of the downstream function.If this is a function for at least partially automatic driving of the motor vehicle, this will in particular increase driving safety.

[0029] According to at least one embodiment, the plurality of pixels of the detector array are arranged corresponding to a plurality of subregions of the detector array. To generate the measurement data set, the subregions of the plurality of subregions are exposed one after the other in a predetermined order until all subregions of the plurality of subregions are exposed. The cluster is identified within a first subregion of the plurality of subregions, and the surroundings of the cluster lie within the first subregion.

[0030] In other words, all pixels of the cluster and all pixels classified as potentially defective lie within the same first sub-area. It should be noted that the term "first" merely specifies the corresponding sub-area, but is to be understood independently of the order of exposure of the plurality of sub-areas.

[0031] The sub-areas of the plurality of sub-areas are different from one another, in particular, they are all different from one another. The sub-areas of the plurality of sub-areas therefore do not overlap.

[0032] If the lidar sensor system is configured as a laser scanner, the sequence of exposure of the plurality of sub-areas corresponds, in particular, to the scanning direction. If the lidar sensor system is configured as a flash lidar sensor system, the sequence can be determined, for example, by using a shutter, in particular an electronic shutter.

[0033] Such embodiments take into account the fact that a high energy input to pixels of the cluster due to highly reflective objects only leads to blooming in neighboring pixels in the surrounding area if they are illuminated simultaneously or approximately simultaneously. This further reduces the risk of false positive detection of blooming.

[0034] According to at least one embodiment, the plurality of pixels is arranged corresponding to a plurality of columns and a plurality of rows of the detector array, and each of the sub-regions of the plurality of sub-regions is defined by a column of the plurality of columns or by a plurality of adjacent, in particular directly adjacent, columns of the plurality of columns.

[0035] It should be noted that a single pixel of the detector array does not necessarily consist of a single optical detector. Rather, it is also possible for a group of several adjacent optical detectors to form a pixel. The latter is particularly possible when using single-proton avalanche photodiodes (SPADs). This means that even a single column of pixels does not necessarily consist of a single column of optical detectors.

[0036] In other embodiments, however, it is also possible for a pixel to consist of exactly one optical detector, for example a single photodiode or avalanche photodiode, APD (English: Avalanche Photodiode).

[0037] According to at least one embodiment, the first subregion is formed by exactly one column of the plurality of columns.

[0038] This allows for particularly precise localization of the pixels affected by blooming. In particular, in some embodiments, all subregions are each formed by exactly one column of the plurality of columns.

[0039] According to at least one embodiment, the limit value corresponds to a saturation limit value.

[0040] The saturation threshold can be determined, for example, by a corresponding limitation of the optical detectors of the pixels of the detector array and / or by corresponding limitations of electronic components for signal acquisition or signal evaluation, for example, an analog-to-digital converter (ADC). Therefore, measuring an energy value above the saturation threshold is not possible or is not permitted due to a corresponding lack of reliability. In the latter case, for example, the energy values ​​of pixels whose energy input would actually lead to a value above the saturation threshold can be set to the saturation threshold. In such embodiments, the energy values ​​of the pixels of the cluster are therefore equal to the saturation threshold as a predetermined threshold.In this way, the cluster is determined in such a way that it only contains pixels with particularly high energy input, which ultimately reduces the probability of false positive detections of pixels affected by blooming.

[0041] According to at least one embodiment, the potentially defective pixels are classified as defective only if all pixels of the potentially defective pixels and all pixels of the cluster together form a contiguous region of the detector array.

[0042] In other words, the potentially defective pixels are only classified as defective if all potentially defective pixels and all pixels of the cluster are adjacent to each other in the sense described above.

[0043] This takes into account the fact that blooming typically occurs continuously in a specific vicinity of pixels in the cluster. In corresponding embodiments, the risk of false positive detections of faulty pixels is further reduced.

[0044] According to at least one embodiment, the cluster contains two or more adjacent pixels of the detector array, for example three or more.

[0045] In other words, potentially defective pixels are only identified and classified as defective if the cluster contains two or more adjacent pixels or three or more.

[0046] This also further reduces the risk of false positive detections of faulty pixels.

[0047] According to at least one embodiment, a first further cluster consisting of one or more adjacent pixels of the detector array is identified within a second sub-region of the plurality of sub-regions, wherein the respective energy values ​​of the pixels of the first further cluster are each greater than or equal to the predetermined limit value. Based on the respective distance values ​​of the pixels of the first further cluster, a distance value of the first further cluster is determined, and the pixels in an environment of the first further cluster are identified as further potentially defective pixels if their respective distance values ​​lie within the distance range around the distance value of the first further cluster. Depending on a spatial progression of the energy values ​​of the first further potentially defective pixels, the first further potentially defective pixels are classified as defective.Correcting the measurement data set involves removing the measured values ​​of the pixels classified as faulty in the vicinity of the first additional cluster from the measurement data set or marking them as faulty in the measurement data set.

[0048] The above explanations, advantages, and statements regarding the cluster can be applied analogously to the first additional cluster. Furthermore, further embodiments of the computer-implemented method arise by performing the aforementioned steps for further, in particular all, sub-areas of the plurality of sub-areas.

[0049] For example, the distance range around the distance value of the first additional cluster can correspond to an interval whose width is identical to the distance range around the distance value of the cluster. However, the distance value of the first additional cluster is not necessarily equal to the distance value of the cluster.

[0050] In such embodiments, the blooming-affected pixels of the detector array for the measurement data set can be more completely detected.

[0051] According to at least one embodiment, a second further cluster is identified within the first sub-region, comprising one or more adjacent pixels of the detector array that lie outside the cluster, wherein the respective energy values ​​of the pixels of the second further cluster are each greater than or equal to the predetermined limit value. Based on the respective distance values ​​of the pixels of the second further cluster, a distance value of the second further cluster is determined, and the pixels in an environment of the second further cluster are identified as second further potentially defective pixels if their respective distance values ​​lie within a distance range around the distance value of the second further cluster. Depending on a spatial progression of the energy values ​​of the second further potentially defective pixels, the second further potentially defective pixels are classified as defective.Correcting the measurement data set involves removing the measured values ​​of the pixels classified as faulty in the vicinity of the second additional cluster from the measurement data set or marking them as faulty in the measurement data set.

[0052] The above explanations, advantages, and statements regarding the cluster can be applied analogously to the second additional cluster. For example, the distance range around the distance value of the second additional cluster can correspond to an interval whose width is identical to the distance range around the distance value of the cluster.

[0053] In such embodiments, the blooming-affected pixels of the detector array for the measurement data set can be more completely detected.

[0054] According to at least one embodiment, a cluster energy distribution function is determined for the cluster based on a predetermined pixel energy distribution function. The spatial distribution of the energy values ​​of the potentially defective pixels is compared with the cluster energy distribution function, and the potentially defective pixels are classified as defective depending on the result of the comparison.

[0055] The pixel energy distribution function is, in particular, a so-called point spread function of the lidar sensor system. The point spread function results, in particular, from the optical components of the lidar sensor system. The point spread function describes, in particular, deviations from optimal imaging of a point light source by the optical components of the lidar sensor system. The point spread function describes, in particular, how an idealized point object would be imaged by the lidar sensor system on the detector array, and in particular, how the spatial distribution of the energy values ​​on the detector array would be expected. The point spread function is also referred to as the point response and takes into account, for example, diffraction effects, aberrations, and so on. The actual distribution of the energy values ​​on the detector array corresponds, in particular, to a convolution of the actual shape of the light sources with the point spread function.In other words, the point spread function smears the image, or rather the distribution of energy values ​​on the detector array. For example, the point spread function of the lidar sensor system is approximately radially symmetric. In particular, the point spread function has a maximum value in the center and decreases towards the outside, particularly very rapidly and sharply. If the cluster consists of a single pixel, the cluster energy distribution function is equal to the pixel energy distribution function. Otherwise, the cluster energy distribution function is determined depending on the pixel energy distribution function and, in particular, depending on the corresponding pixel positions of the individual pixels in the cluster.

[0056] The spatial distribution of the energy values ​​of the potentially defective pixels corresponds, in particular, to a distribution on the detector array. The comparison of the spatial distribution of the energy values ​​of the potentially defective pixels with the cluster energy distribution function involves, in particular, calculating a deviation value that quantifies the deviation of the spatial distribution of the energy values ​​from the cluster energy distribution function, for example, a mean square error or the like. The result of the comparison includes, in particular, the deviation value determined in this way. For example, the potentially defective pixels are classified as defective if the deviation value is less than or equal to a predetermined maximum deviation, in particular only if.

[0057] Such embodiments exploit the fact that the energy values ​​of the pixels affected by blooming are generally dominated by blooming and therefore decrease with increasing distance from the cluster, according to the point spread function. This allows for particularly reliable identification of the pixels affected by blooming.

[0058] The pixel energy distribution function, in particular the point spread function, can be determined for the lidar sensor system during calibration, for example, at the end of the lidar sensor system's production. The pixel energy distribution function can be determined piece-specifically for each individual lidar sensor system, type-specifically for all lidar sensor systems of a certain type, batch-specifically for all lidar sensor systems of a certain batch or production period, and so on.

[0059] According to at least one embodiment, for each pixel of the cluster, the pixel energy distribution function is shifted according to a position of the respective pixel, in particular a pixel position on the detector array, of the respective pixel, and the cluster energy distribution function is determined by combining the shifted pixel energy distribution functions, in particular all shifted pixel energy distribution functions of the pixels of the cluster, with one another, in particular by convolving them with one another.

[0060] Depending on the embodiment of the computer-implemented method, the combination of the shifted pixel energy distribution functions may also include a weighting or multiplication by an amplitude correction factor, particularly in addition to the convolution. This can be particularly advantageous if the energy values ​​of the individual pixels in the cluster are different. The amplitude correction factors can be equal to or proportional to the energy value of the corresponding pixel.

[0061] If the threshold for the energy values ​​corresponds to the saturation threshold, the energy values ​​of all pixels in the cluster are equal to the saturation threshold, as mentioned above. In this case, for example, the combination of the shifted pixel energy distribution functions may also include clipping to the saturation threshold, so that values ​​above the saturation threshold are set to the saturation threshold.

[0062] Except for the weighting factor or amplitude correction factor, if applicable, and the shift corresponding to the position of the respective pixel, the shifted pixel energy distribution functions are therefore identical, in particular, their shape is identical. The shift is performed such that a center of the respective shifted pixel energy distribution function corresponds to the position of the respective pixel, in particular, its center position.

[0063] According to at least one embodiment, a first further cluster energy distribution function is determined for the first further cluster based on the pixel energy distribution function. The spatial distribution of the energy values ​​of the first further potentially defective pixels is compared with the first further cluster energy distribution function, and the first further potentially defective pixels are classified as defective depending on the result of the comparison.

[0064] According to at least one embodiment, a second further cluster energy distribution function is determined for the second further cluster based on the pixel energy distribution function. The spatial distribution of the energy values ​​of the second further potentially defective pixels is compared with the second further cluster energy distribution function, and the second further potentially defective pixels are classified as defective depending on the result of the comparison.

[0065] The explanation relates to the pixel energy distribution function and the cluster energy distribution function can be transferred analogously to the corresponding embodiments with the first and / or second further cluster.

[0066] For applications or application situations that may arise in a method according to the invention and which are not explicitly described herein, it may be provided that, according to the method, an error message and / or a request to enter user feedback is output and / or a standard setting and / or a predetermined initial state is set.

[0067] According to a further aspect of the invention, a computer-implemented method for object detection and / or object tracking is provided. A computer-implemented method according to the invention is performed for correcting a measurement data set of a lidar sensor system, and using the corrected measurement data set, an object detection algorithm is performed for detecting an object in the environment of the lidar sensor system and / or an object tracking algorithm is performed for tracking the object.

[0068] The object detection algorithm or the object tracking algorithm is in particular a known algorithm which, according to the invention, uses the corrected measurement data set instead of the measurement data set.

[0069] According to a further aspect of the invention, a computer-implemented method for distance determination is provided. A computer-implemented method according to the invention is used to correct a measurement data set of a lidar sensor system. Using the corrected measurement data set, a distance of an object in an environment of the lidar sensor system from the lidar sensor system is determined. The determination of the distance of the object can be determined, in particular, based on the measured values ​​of the corrected measurement data set that are not marked as erroneous or that have not been removed from the measurement data set.

[0070] According to a further aspect of the invention, a method for at least partially automatically driving a motor vehicle is specified. In this case, a computer-implemented method according to the invention for correcting a measurement data set or a computer-implemented method according to the invention for object recognition and / or object tracking or a computer-implemented method according to the invention for distance determination is carried out. Using the corrected measurement data set, optionally in particular using a result of the object recognition algorithm and / or object tracking algorithm or the distance of the object, at least one control signal for at least partially automatically driving the motor vehicle is generated, in particular by means of at least one computing unit, and / or assistance information is generated for assisting a person driving the motor vehicle, in particular for assisting in driving the motor vehicle.

[0071] The at least one control signal can, for example, be provided to one or more actuators of the motor vehicle, including, for example, one or more brake actuators and / or one or more steering actuators and / or one or more drive motors of the motor vehicle. The one or more actuators can influence a longitudinal and / or lateral control of the motor vehicle depending on the at least one control signal in order to guide the motor vehicle at least partially automatically.

[0072] The assistance information can be output to the person driving the motor vehicle via an output device of the motor vehicle, for example via a display and / or an audio output system and / or a haptic output system.

[0073] According to a further aspect of the invention, a data processing device with at least one computing unit is specified. The at least one computing unit is configured to carry out a computer-implemented method according to the invention for correcting a measurement data set, or a computer-implemented method according to the invention for object recognition and / or object tracking, or a computer-implemented method according to the invention for distance determination. A computing unit can be understood, in particular, as a data processing device that contains a processing circuit. The computing unit can therefore, in particular, process data for carrying out computing operations. This may also include operations for carrying out indexed access to a data structure, for example a look-up table (LUT).

[0074] The computing unit may, in particular, contain one or more computers, one or more microcontrollers, and / or one or more integrated circuits, for example one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), and / or one or more single-chip systems (SoCs). The computing unit may also contain one or more processors, for example one or more microprocessors, one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more signal processors, in particular one or more digital signal processors (DSPs). The computing unit may also include a physical or virtual network of computers or other of the aforementioned units.

[0075] In various embodiments, the computing unit includes one or more hardware and / or software interfaces and / or one or more memory units.

[0076] A memory unit can be a volatile data memory, for example a dynamic random access memory (DRAM) or a static random access memory (SRAM), or a non-volatile data memory, for example a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory or flash EEPROM, a ferroelectric random access memory (FRAM), a magnetoresistive random access memory,MRAM (magnetoresistive random access memory) or phase-change random access memory (PCRAM).

[0077] According to a further aspect of the invention, an electronic vehicle guidance system for a motor vehicle is provided. The electronic vehicle guidance system comprises a data processing device according to the invention. The at least one computing unit is configured to generate, using the corrected measurement data set, at least one control signal for at least partially automatically guiding the motor vehicle and / or to generate assistance information for assisting a person driving the motor vehicle.

[0078] An electronic vehicle guidance system can be understood as an electronic system designed to guide a vehicle fully automatically or autonomously, in particular without requiring driver intervention. The vehicle automatically performs all required functions, such as steering, braking, and / or acceleration maneuvers, monitoring and detecting road traffic, and corresponding reactions. In particular, the electronic vehicle guidance system can implement a fully automatic or fully autonomous driving mode of the motor vehicle according to level 5 of the SAE J3016 classification. An electronic vehicle guidance system can also be understood as an advanced driver assistance system (ADAS), which supports the driver in partially automated or semi-autonomous driving.In particular, the electronic vehicle guidance system can implement a partially automated or semi-autonomous driving mode according to levels 1 to 4 of the SAE J3016 classification. Here and below, "SAE J3016" refers to the corresponding standard in the April 2021 version.

[0079] The at least partially automated vehicle guidance may therefore include driving the vehicle according to a fully automated or fully autonomous driving mode of Level 5 according to SAE J3016. The at least partially automated vehicle guidance may also include driving the vehicle according to a partially automated or semi-autonomous driving mode according to Levels 1 to 4 according to SAE J3016.

[0080] According to at least one embodiment, the electronic vehicle guidance system includes the lidar sensor system. According to at least one embodiment, the lidar sensor system is configured as a laser scanner.

[0081] According to at least one embodiment, each pixel of the detector array includes one or more optical detectors.

[0082] According to at least one embodiment, the optical detectors of the detector array are each designed as single-photon avalanche photodiodes.

[0083] According to at least one embodiment, the plurality of pixels of the detector array are arranged according to a plurality of columns and a plurality of rows. The number of rows is in particular between 10 and 1000, for example between 50 and 1000. The number of columns is in particular between 10 and 1000, in particular between 50 and 1000, for example between 100 and 1000.

[0084] Further embodiments of the electronic vehicle guidance system according to the invention follow directly from the various embodiments of the computer-implemented methods according to the invention and vice versa.

[0085] In particular, individual features and corresponding explanations, as well as advantages relating to the various embodiments of the computer-implemented methods according to the invention, can be transferred analogously to corresponding embodiments of the electronic vehicle guidance system according to the invention. In particular, the electronic vehicle guidance system according to the invention is designed or programmed to carry out a computer-implemented method according to the invention. In particular, the electronic vehicle guidance system according to the invention carries out a computer-implemented method according to the invention.

[0086] According to a further aspect of the invention, a computer program product with instructions is specified. When the instructions are executed by a data processing device, in particular a data processing device according to the invention, the instructions cause the data processing device to perform a computer-implemented method according to the invention. The instructions can be present, for example, as program code. The program code can be provided, for example, as binary code or assembly code and / or as source code of a programming language, for example, C, and / or as a program script, for example, Python.

[0087] According to a further aspect of the invention, a further computer program with further instructions is provided. Upon execution of the further instructions by an electronic vehicle guidance system according to the invention, in particular by the data processing device of the electronic vehicle guidance system, the further instructions cause the electronic vehicle guidance system to carry out a method according to the invention for at least partially automatically guiding a motor vehicle.

[0088] The additional instructions can be provided, for example, as program code. The program code can be provided, for example, as binary code or assembly code and / or as source code of a programming language, for example, C, and / or as a program script, for example, Python.

[0089] According to a further aspect of the invention, a computer-readable storage medium is provided which stores a computer program according to the invention and / or a further computer program according to the invention.

[0090] The computer program, the further computer program and the computer-readable storage medium are each computer program products with the instructions and / or the further instructions.

[0091] Further features of the invention emerge from the claims, the figures and the description of the figures. The features and combinations of features mentioned above in the description as well as the features and combinations of features mentioned below in the description of the figures and / or shown in the figures can be encompassed by the invention not only in the respectively specified combination, but also in other combinations. In particular, the invention can also encompass embodiments and combinations of features that do not have all the features of an originally formulated claim. Furthermore, the invention can encompass embodiments and combinations of features that go beyond the combinations of features set out in the backreferences to the claims or deviate from them.

[0092] The invention is explained in more detail below using specific embodiments and associated schematic drawings. In the figures, identical or functionally equivalent elements may be provided with the same reference numerals. The description of identical or functionally equivalent elements may not necessarily be repeated for different figures.

[0093] The figures show:

[0094] Fig. 1 is a schematic representation of a motor vehicle with an exemplary embodiment of an electronic vehicle guidance system according to the invention;

[0095] Fig. 2 is a schematic representation of various aspects of an exemplary embodiment of a computer-implemented method according to the invention for correcting a measurement data set of a lidar sensor system;

[0096] Fig. 3 is a schematic representation of a pixel energy distribution function according to another exemplary embodiment of a computer-implemented method according to the invention for correcting a measurement data set of a lidar sensor system;

[0097] Fig. 4 is a schematic flow diagram of another exemplary embodiment of a computer-implemented method according to the invention for correcting a measurement data set of a lidar sensor system; and

[0098] Fig. 5 is a schematic flow diagram of a further exemplary embodiment of a computer-implemented method according to the invention for correcting a measurement data set of a lidar sensor system.

[0099] Fig. 1 schematically shows a motor vehicle 1 with an exemplary embodiment of an electronic vehicle guidance system 2 according to the invention. The electronic vehicle guidance system 2 contains a data processing device according to the invention with at least one computing unit 4. The motor vehicle 1, for example the electronic vehicle guidance system 2, has a lidar sensor system 3, which can be configured, for example, as a laser scanner. The lidar sensor system 3 has a detector array with a plurality of pixels, which are arranged, in particular, according to a plurality of rows and a plurality of columns.

[0100] By means of the electronic vehicle guidance system 2, a method according to the invention for at least partially automatically guiding the motor vehicle 1 can be carried out. The at least one computing unit 4 carries out a computer-implemented method according to the invention for correcting a measurement data set 15 of the lidar sensor system 3. For this purpose, the at least one computing unit 4 receives a measurement data set 15 from the lidar sensor system 3 and generates a corrected measurement data set 16 based thereon. The measurement data set 15 contains respective measurement values, including a respective distance value and a respective energy value, for each pixel of the plurality of pixels.

[0101] The at least one computing unit 4 generates, using the corrected measurement data set 16, i.e. depending on the corrected measurement data set 16, at least one control signal for at least partially automatically driving the motor vehicle 1 and / or assistance information for assisting a person driving the motor vehicle 1.

[0102] Fig. 4 shows a schematic flow diagram of an exemplary embodiment of the computer-implemented method according to the invention for correcting the measurement data set 15.

[0103] After the at least one computing unit 4 has received the measurement data set 15, it identifies in step 400 a cluster 10 from one or more adjacent pixels of the detector array, wherein the respective energy values ​​of the pixels of the cluster 10 are greater than or equal to a predetermined limit value, for example equal to a saturation limit value.

[0104] In step 420, the at least one computing unit 4 determines a distance value of the cluster 10 based on the respective distance values ​​of the pixels of the cluster 10, for example as an average of the distance values ​​of the pixels of the cluster 10. Furthermore, the at least one computing unit 4 identifies those pixels as potentially defective pixels that lie in a predefined environment 11a, 11b of the cluster 10 and whose distance values ​​lie within a predefined distance range around the distance value of the cluster 10.

[0105] In step 440, the at least one computing unit 4 classifies the potentially defective pixels as defective depending on a spatial progression of the energy values ​​of the potentially defective pixels and generates the corrected measurement data set 16 by removing the measurement values ​​of the pixels classified as defective in the environment 11 a, 11 b of the cluster 10 from the measurement data set 15 or by marking them as defective in the measurement data set 15.

[0106] Fig. 5 shows a schematic flow diagram of a further exemplary embodiment of the computer-implemented method according to the invention for correcting the measurement data set 15.

[0107] After the at least one computing unit 4 has received the measurement data set 15, a column of the detector array is selected in step 500. In step 520, the at least one computing unit 4 checks whether a cluster 10 consisting of one or more, preferably two or more, adjacent pixels of the detector array is present within the column, wherein the respective energy values ​​of the pixels of the cluster 10 are greater than or equal to the predetermined limit value, for example, equal to the saturation limit value. If this is not the case, the process continues with step 500 by selecting the next column of the detector array. Otherwise, the process continues with step 540.

[0108] In step 540, the at least one computing unit 4 determines a distance value of the cluster 10 based on the respective distance values ​​of the pixels of the cluster 10, for example, as an average of the distance values ​​of the pixels of the cluster 10. Furthermore, the at least one computing unit 4 identifies those pixels as potentially defective pixels that lie in a predefined environment 11a, 11b of the cluster 10 and whose distance values ​​lie within a predefined distance range around the distance value of the cluster 10. If no such potentially defective pixels are identified, the process continues with step 520 by searching within the same column for another cluster whose pixels have energy values ​​that are greater than or equal to the predetermined limit value. Otherwise, the process continues with step 560.For example, in step 540, the at least one computing unit 4 can also check whether all of the potentially defective pixels and all of the pixels of cluster 10 together form a contiguous region of the detector array, i.e., a contiguous section within the column. For example, the process continues with step 560 only if this is the case, and otherwise with step 520.

[0109] In Fig. 2, a schematic illustration of the environment of the lidar sensor system 3 is shown on the right. The illustration shows, in particular, an object 9, for example a part of a traffic sign, in the environment of the lidar sensor system 3. In an area 6 of the object 9, this object is highly reflective, which leads, for example, to energy values ​​of the pixels of the cluster 10 being equal to the saturation limit. This is shown schematically on the left in Fig. 2, where the pixel positions of the column under consideration are plotted in the vertical direction and the corresponding energy values ​​in the horizontal direction. In areas 7a, 7b vertically above and below area 6, the object 9 is not highly reflective, but the pixels in the column under consideration in the associated areas 11a, 11b above and below the pixels of the cluster 10 are affected by blooming.

[0110] In step 560, the at least one computing unit 4 determines a cluster energy distribution function for the cluster 10 based on a predefined point spread function 13 and compares the spatial distribution of the energy values ​​of the potentially defective pixels with the cluster energy distribution function. If a deviation value between the spatial distribution of the energy values ​​of the potentially defective pixels and the cluster energy distribution function is less than or equal to a predefined maximum deviation, the at least one computing unit 4 classifies the potentially defective pixels as defective. If this is the case, the at least one computing unit 4 generates the corrected measurement data set 6 by removing the measurement values ​​of the pixels classified as defective in the environment 11a, 11b of the cluster 10 from the measurement data set 15 or by marking them as defective in the measurement data set 15.Otherwise, the process continues with step 520 by searching for another cluster within the same column whose pixels have energy values ​​that are greater than or equal to the predetermined threshold.

[0111] Fig. 3 schematically illustrates the point spread function 13 and a pixel energy distribution function 14, which corresponds to the point spread function 13 limited to a maximum value. To determine the cluster energy distribution function, the at least one computing unit 4 can shift the pixel energy distribution function 14 for each pixel of the cluster 10 according to a position of the respective pixel and combine, in particular convolve, the shifted pixel energy distribution functions.

[0112] The method was described with reference to Fig. 5 in such a way that a column-by-column analysis of the pixels is performed. This is particularly advantageous when the individual columns are exposed sequentially to generate the measurement data set 15. However, it is also possible for the detector array to be exposed sub-area by sub-area according to a different scheme. The method can then be adapted accordingly so that the analyzed columns are replaced by the sub-areas. The sub-areas can, for example, be groups of a defined number of adjacent columns.

[0113] As described, in particular with reference to the figures, the invention makes it possible to correct measurement data sets generated by a lidar sensor system in order to at least partially correct the effects of blooming.

[0114] Advantageously, no tracking algorithms are required for the inventive correction of the measurement data set.

[0115] In various embodiments where the cluster comprises two or more pixels, the accuracy of the method is increased by calculating the cluster energy distribution function corresponding to multiple pixels, thus reducing the risk of misclassifications.

[0116] In various embodiments, the point spread function inherent in the optical path of the lidar sensor system is used to identify the faulty points. This increases the robustness of the method for use with different lidar sensor systems.

Claims

Patent claims 1. A computer-implemented method for correcting a measurement data set (15) of a lidar sensor system (3) having a detector array with a plurality of pixels, and wherein the measurement data set (15) for the pixels of the plurality of pixels contains measured values ​​that include an energy value and a distance value, wherein a cluster (10) is identified from one or more adjacent pixels of the detector array, wherein the respective energy values ​​of the pixels of the cluster (10) are greater than or equal to a predetermined limit value; based on the respective distance values ​​of the pixels of the cluster (10), a distance value of the cluster (10) is determined, and pixels in an environment (11 a, 11 b) of the cluster (10) are identified as potentially defective pixels if their distance values ​​lie within a distance range around the distance value of the cluster (10);the potentially defective pixels are classified as defective depending on a spatial progression of the energy values ​​of the potentially defective pixels; and the measurement data set (15) is corrected by removing the measured values ​​of the pixels classified as defective in the environment (11a, 11b) of the cluster (10) from the measurement data set (15) or by marking them as defective in the measurement data set (15); 2. Computer-implemented method according to claim 1, wherein the plurality of pixels is arranged corresponding to a plurality of sub-regions of the detector array, the sub-regions are exposed one after the other to generate the measurement data set, the cluster (10) is identified within a first sub-region and the surroundings (11 a, 11 b) of the cluster (10) lie within the first sub-region.

3. A computer-implemented method according to claim 2, wherein the plurality of pixels are arranged according to a plurality of columns and a plurality of rows and each of the sub-areas is formed by one or more adjacent columns of the plurality of columns.

4. The computer-implemented method of claim 3, wherein the first sub-area is formed by exactly one column of the plurality of columns.

5. Computer-implemented method according to one of claims 2 to 4, wherein within a second sub-area of ​​the plurality of sub-areas, a first further cluster is identified from one or more adjacent pixels of the detector array, wherein the respective energy values ​​of the pixels of the first further cluster are greater than or equal to the predetermined limit value; based on the respective distance values ​​of the pixels of the first further cluster, a distance value of the first further cluster is determined, and pixels in an environment of the first further cluster are identified as first further potentially defective pixels if their distance values ​​lie within a distance range around the distance value of the first further cluster; depending on a spatial progression of the energy values ​​of the first further potentially defective pixels, the first further potentially defective pixels are classified as defective;and correcting the measurement data set (15) includes removing the measured values ​​of the pixels classified as faulty in the vicinity of the first further cluster from the measurement data set (15) or marking them as faulty in the measurement data set (15); 6. Computer-implemented method according to one of claims 2 to 5, wherein within the first sub-area a second further cluster is identified from one or more adjacent pixels of the detector array which lie outside the cluster, wherein the respective energy values ​​of the pixels of the second further cluster are greater than or equal to the predetermined limit value; based on the respective distance values ​​of the pixels of the second further cluster, a distance value of the second further cluster is determined and pixels in an environment of the second further cluster are identified as second further potentially defective pixels if their distance values ​​lie within a distance range around the distance value of the second further cluster; depending on a spatial progression of the energy values ​​of the second further potentially defective pixels, the second further potentially defective pixels are classified as defective; and correcting the measurement data set (15) involves removing the measured values ​​of the pixels classified as defective in the vicinity of the second further cluster from the measurement data set (15) or marking them as defective in the measurement data set (15).

7. A computer-implemented method according to any one of the preceding claims, wherein the limit value corresponds to a saturation limit value.

8. Computer-implemented method according to one of the preceding claims, wherein the potentially defective pixels are classified as defective only if all of the potentially defective pixels and all pixels of the cluster (10) together form a contiguous region of the detector array.

9. A computer-implemented method according to any one of the preceding claims, wherein the cluster (10) contains two or more adjacent pixels of the detector array.

10. Computer-implemented method according to one of the preceding claims, wherein a cluster energy distribution function for the cluster (10) is determined based on a predetermined pixel energy distribution function (14), the spatial course of the energy values ​​of the potentially defective pixels is compared with the cluster energy distribution function, and the potentially defective pixels are classified as defective depending on a result of the comparison.

11. A computer-implemented method according to claim 9 and claim 10, wherein for each pixel of the cluster (10), the pixel energy distribution function (14) is shifted according to a position of the respective pixel; and the cluster energy distribution function is determined by combining the shifted pixel energy distribution functions.

12. Computer-implemented method according to one of claims 10 or 11, wherein the pixel energy distribution function (14) corresponds to a point spread function of the lidar sensor system (3).

13. A computer-implemented method for object detection and / or object tracking, wherein a computer-implemented method according to one of the preceding claims is carried out and, using the corrected measurement data set (16), an object detection algorithm for detecting an object (9) in an environment of the lidar sensor system (3) and / or an object tracking algorithm for tracking the object (9) is carried out.

14. A computer-implemented method for determining distance, wherein a computer-implemented method according to one of claims 1 to 12 is carried out and a distance of an object (9) in an environment of the lidar sensor system (3) from the lidar sensor system (3) is determined using the corrected measurement data set (16).

15. A method for at least partially automatically driving a motor vehicle (1), wherein a computer-implemented method according to one of the preceding claims is carried out and, using the corrected measurement data set (16), at least one control signal for at least partially automatically driving the motor vehicle (1) is generated; and / or assistance information for assisting a person driving the motor vehicle (1) is generated.

16. Data processing device with at least one computing unit (4) which is configured to carry out a computer-implemented method according to one of claims 1 to 14.

17. Electronic vehicle guidance system (2) for a motor vehicle (1), comprising a data processing device according to claim 16, wherein the at least one computing unit (4) is configured to, and using the corrected measurement data set (16), to generate at least one control signal for at least partially automatically driving the motor vehicle (1); and / or To generate assistance information to assist a person driving the motor vehicle (1).

18. Electronic vehicle guidance system (2) according to claim 17, wherein the electronic vehicle guidance system (2) includes the lidar sensor system (3).

19. Computer program product comprising - instructions which, when executed by a data processing device, Cause the data processing device to carry out a computer-implemented method according to one of claims 1 to 14; and / or further instructions which, when executed by an electronic vehicle guidance system (2) according to one of claims 17 or 18, cause the electronic vehicle guidance system (2) to carry out a method according to claim 15.

Images