Method and system for improving the quality of image information from a 3D recognition unit for use in a vehicle

Abstract

Method for improving the reliability of image information of a 3D recognition unit (110) for use on a vehicle (1, 1a, 1b, 1c), wherein the 3D recognition unit (110) serves to illuminate the object and to create a 3D image from the light reflected from the object, which contains 3D information, comprising a step in which the 3D recognition unit (110) is calibrated for a specific position of the 3D recognition unit (110) on the vehicle, characterized by a step for calibrating the 3D recognition unit (110) comprising the following substeps: Detection (S1) of markers (S10, S20, S30) using different integration / exposure times, wherein the markers (S10, S20, S30) have different known minimum reflectances as the lowest supported reflectance corresponding to different object types and representing the minimum number of pixels returned by the said markers to be detected; Extraction (S2) of reliable information from the aforementioned marker detection; Creation (S3) of different masks based on the reliable information thus obtained for different markings that correspond to different object types, so that the different masks for the different object types are created.

Description

Technical area

[0001] One subject of the invention is a method for improving the reliability of image information from a 3D recognition unit for use in a vehicle according to the preamble to claim 1. The invention also relates to a system for improving the reliability of image information from a 3D recognition unit for use in a vehicle. The invention relates to a vehicle. Another subject of the invention is a computer program and a computer program product. State of the art

[0002] Image quality measurement and comparison methods exist for 2D cameras. For time-of-flight cameras, different measurement methods can be used to measure the quality of the distance data, such as measuring the variance of distance measurements. Based on this measurement data, the camera's exposure time can be automatically adjusted to achieve good image quality across the entire image.

[0003] When mounting a time-of-flight camera on a vehicle, the camera's distance from the ground depends on the vehicle type. Furthermore, due to its limited illuminance, the camera cannot illuminate the entire field of view, so the camera's automatic exposure function may attempt to maximize the view, resulting in overexposure for the camera, creating other visual problems.

[0004] WO 2011 / 54971 A2 describes a method and system for detecting 3D objects in a specific area. A camera is calibrated with respect to the ground based on data about the distance to the ground. An object detection volume is created in which all points with a distance of, for example, 10 cm or more from the ground are excluded.

[0005] The paper "Road Marking Detection Using LIDAR Reflective Intensity Data and its Application to Vehicle Localization" by Hata and Wolf, presented at the IEEE 17th International Conference on Intelligent Transportation Systems (ITSC) in 2014 (pp. 584-589), describes a test experiment investigating the detection of road markings using a vehicle's LIDAR system. Since LIDAR data from road markings exhibits greater intensity than LIDAR data from the road surface, a threshold is set before road marking detection is performed to localize the vehicle on the road.

[0006] Furthermore, the article “Analysis and characterization of the PMD camera for application in mobile robotics” by Wiedemann, Sauer et al., presented at the 17th World Congress of the International Federation of Automatic Control in 2008 (pp. 13689-13694), describes the effect of adjusting an integration time on the data quality in 3D image acquisition using a time-of-flight camera. Objects of the invention

[0007] An object of the present invention is to provide a flexible and efficient method for improving the reliability of image information of a 3D recognition unit for use in a vehicle.

[0008] Another object of the present invention is to provide a flexible and efficient system for improving the reliability of image information of a 3D recognition unit for use in a vehicle. Summary of the invention

[0009] These and other objects, which will become apparent from the following description, are achieved by a method, a system, a vehicle, a computer program, and a computer program product according to the appended independent claims. Preferred embodiments of the method and system are specified in the appended dependent claims.

[0010] Specifically, one object of the invention is achieved by a method for improving the reliability of image information from a 3D recognition unit for use in a vehicle, in which the 3D recognition unit serves to illuminate the object and create a 3D image from the light reflected from the object, which contains 3D information. The method comprises a step in which the 3D recognition unit is calibrated for a specific 3D recognition unit position on the vehicle.The calibration step of the 3D recognition unit includes the following substeps: detecting markers with different known minimum reflectances (the lowest supported reflectance) corresponding to different object types using different integration times / exposure times; obtaining reliable information from said marker detection; creating different masks based on the reliable information thus obtained for different markers corresponding to different object types. The same 3D recognition unit can be used for different object types, thus reducing costs and improving flexibility and efficiency.Furthermore, the quality of the information / data in the image of different objects is improved, allowing data processing to lead to better identification of the detected objects. The amount of required information / data is minimized by creating different masks for different objects. This allows the maximum object detection distance for the 3D recognition unit to be determined for different objects. This enables a generic solution for a vehicle fleet using a single variant of the 3D recognition unit instead of multiple preset variants or manually creating a mask for each 3D recognition unit depending on the object.

[0011] According to one embodiment, the method comprises a step in which, based on the reliable information thus obtained, different masks are created for different 3D recognition unit positions, including the height above said markings and the direction of the 3D recognition unit. This allows the same 3D recognition unit to be used for different vehicle configurations that require different positions of the 3D recognition unit, including different heights and / or directions for identifying an object, thus reducing costs. Furthermore, the quality of the information / data of the image of the object is improved, so that processing the data leads to better identification of the detected object regardless of different positions, i.e., different heights / directions of the 3D recognition unit.This improves the reliability of the image information from a 3D recognition unit. The amount of required information / data is minimized by creating different masks for the different positions of the 3D recognition unit. This allows the maximum object detection distance for the 3D recognition unit to be determined for the different positions of the 3D recognition unit. This enables a generic solution for a vehicle fleet using a single variant of the 3D recognition unit instead of multiple preset variants or manually creating a mask for each 3D recognition unit depending on its position on the vehicle.By assigning different masks to different positions of a 3D recognition unit on a vehicle in this way, these different positions can be identifiable so that when a 3D recognition unit is placed at a specific location on a vehicle, the 3D recognition unit is controlled to use the mask assigned to the identified position.

[0012] According to one embodiment of the method, the step of detecting markings with a known minimum reflectivity is performed for different illuminance levels of the 3D recognition unit. This allows different masks to be created for different illuminance levels. This allows different masks to be created for different illuminance levels for the respective position of the 3D recognition unit. This allows different masks to be created for different illuminance levels for different objects. By assigning different masks to different illuminance levels of a 3D recognition unit in this way, such a 3D recognition unit with variable illuminance, which is placed on a vehicle, can be controlled to use the mask assigned to the set illuminance level.

[0013] According to one embodiment of the method, the step of detecting markings with a known minimum reflectivity is performed for different illumination patterns of the 3D recognition unit. This allows different masks to be created for different illumination patterns. Consequently, different masks can be created for different illumination patterns for the respective position of the 3D recognition unit. Consequently, different masks can be created for different illumination patterns for different objects. By assigning different masks to different illumination patterns for a 3D recognition unit in this way, such a 3D recognition unit with variable illumination patterns, which is placed on a vehicle, can be controlled such that the mask assigned to the set illumination pattern is used.

[0014] According to one embodiment of the method, the step of creating a mask comprises excluding information from the 3D recognition unit corresponding to pixels deemed unreliable. This improves the quality of the image information / data. Since not all pixels need to be processed, the object detection and image processing speed are increased.

[0015] According to one embodiment, the method comprises the following steps: detecting markers with a known maximum reflectivity as the highest expected reflectivity using different integration times / exposure times; determining saturation characteristics, including a saturation level, from said marker detection to ensure reliable detection. This allows for determining when a particular object is not detectable due to its reflectivity / saturation.

[0016] According to one embodiment of the method, the step of detecting markers with known maximum reflectance comprises a step of detecting markers with different known maximum reflectances as the highest expected reflectance corresponding to different object types using different integration times / exposure times. Saturation characteristics, including a saturation level, are determined from said marker detection to ensure reliable detection of said different object types. This allows for determining when different objects are not detectable due to their reflectance / saturation.

[0017] According to one embodiment of the method, the step of detecting markers with a known maximum reflectivity is performed for different illuminances of the 3D recognition unit. This makes it possible to determine when a specific object is not detectable due to its reflectivity / saturation for different illuminances of the 3D recognition unit.

[0018] According to one embodiment of the method, the step of detecting markers with a known maximum reflectivity is performed for different illumination patterns of the 3D recognition unit. This makes it possible to determine when a specific object is unrecognizable due to its reflectivity / saturation for different illumination patterns of the 3D recognition unit.

[0019] According to one embodiment, the method comprises a step of storing the masks thus obtained. This allows such masks to be used for validation by comparison with another mask of another 3D recognition unit mounted on a different vehicle with corresponding features, i.e., with the 3D recognition unit positioned in the same way. This allows such masks to be used in another 3D recognition unit of the same type without requiring calibration of that 3D recognition unit, although some correlation may be preferable.

[0020] According to one embodiment, the method comprises a step of storing the saturation characteristics determined thereby. This allows such saturation characteristics to be used for validation by comparing them with saturation characteristics of another 3D recognition unit mounted on a different vehicle with corresponding features, i.e., with the 3D recognition unit positioned in the same manner. This allows such saturation characteristics to be used in another 3D recognition unit of the same type without requiring calibration of that 3D recognition unit, although some correlation may be preferred.

[0021] Specifically, one purpose of the invention is achieved by a system for improving the reliability of the image information of a 3D recognition unit for use in a vehicle. The 3D recognition unit serves to illuminate the object and, from the light reflected from the object, which contains 3D information, to create a 3D image suitable for performing the method described above.

[0022] The system according to the invention offers the advantages according to the corresponding method claims. Brief description of the drawings

[0023] The present invention will be better understood by reading the following detailed description with reference to the accompanying drawings, in which like references refer to the same parts throughout the drawings. Fig. Figure 1a is a schematic side view of a vehicle according to the present invention. Fig. 2 is a schematic diagram of a system for improving the reliability of image information of a 3D recognition unit for use in a vehicle according to an embodiment of the present invention. Fig. 3 is a schematic representation of a block diagram of a method for improving the reliability of image information of a 3D recognition unit for use in a vehicle according to an embodiment of the present invention. Fig. Figure 4a is a schematic plan view of a vehicle with a 3D recognition unit according to the present invention. Fig. Figure 4b is a schematic side view of a vehicle with a particular configuration for a particular position of a 3D recognition unit according to the present invention. Fig. Figure 4c is a schematic side view of a vehicle with a particular configuration for a particular position of a 3D recognition unit according to the present invention. Fig. 5 is a schematic diagram of a computer according to an embodiment of the present invention. Detailed description

[0024] In the following, the term “connection” refers to a communication connection, which may be a physical connection, such as an optoelectronic communication connection, or a non-physical connection, such as a wireless connection in the form of a radio or microwave connection.

[0025] In the following, the term "2D" means two-dimensional. In the following, the term "3D" means three-dimensional.

[0026] In the following, the term "marker" refers to any suitable marking with a known minimum reflectance as the lowest supported reflectance corresponding to a specific object. The term "marker" can also refer to any suitable marking with a known maximum reflectance as the highest expected reflectance. A "marker" can be any suitable element, such as one or more stripes. The term "marker" can refer to a continuous marking and / or to multiple markings forming a pattern. The term "marker" can refer to different markings with different known minimum reflectances as the lowest supported reflectance corresponding to different object types.The markers can correspond to the ground, but also to other objects, such as pedestrians, animals, cyclists, reflective objects in the form of road signs, road markings, guardrails, etc. The markers / mask can take into account the expected object size and the probability of detecting a sufficient number of signals / pixels of the object in a specific area of ​​the 3D recognition unit's field of view. For example, a nearby object may generate 100 pixels / signals due to strong signals / reflections and larger dimensions, while a more distant object will be detected as smaller due to a weaker signal / reflection and its geometry, thus generating 20 pixels, for example.

[0027] In the following, the term “3D recognition unit position” means the height of the 3D recognition unit and the direction of the 3D recognition unit.

[0028] The term "3D recognition unit direction" refers to the direction in which the 3D recognition unit is set and thus detects objects. The direction of the 3D recognition unit includes the angle along the X, Y, and Z axes.

[0029] The term "integration time / exposure time" is used below. With reference to a 3D recognition unit in the form of a time-of-flight camera, such as a continuous wave (PMD) sensor, the term "integration time" is used, meaning that such a time-of-flight camera can capture four images during the integration time.

[0030] Fig. 1 is a schematic representation of a vehicle 1 according to an embodiment of the present invention. The vehicle 1 used as an example is a heavy vehicle in the form of a truck. The vehicle according to the present invention may be any suitable vehicle, such as a bus, a passenger car, a train, or the like. The vehicle comprises a system for improving the reliability of image information of a 3D recognition unit 110 for use with a vehicle according to the present invention. The vehicle comprises a 3D recognition unit 110. The 3D recognition unit 110 can recognize objects that may vary. The 3D recognition unit 110 is mounted on the vehicle, here on the driver's cab, in a specific position, including a specific height H and a specific recognition direction D.The height and / or direction may vary depending on the vehicle configuration, for example, the cab configuration and / or the chassis configuration. The position, including the direction D, includes an angle on the X, Y, and Z axes that must be determined, i.e., the roll angle, the pitch angle, and the yaw angle. If the 3D recognition unit 110 is tilted by 1 degree around the Y axis, with the Y axis pointing in the forward direction of the vehicle, the result differs compared to a 3D recognition unit 100 that has no roll offset from the horizontal line. A vehicle can have several possible mounting positions.

[0031] Fig. 2 is a schematic diagram of a system I for improving the reliability of image information of a 3D recognition unit 110 for use in a vehicle according to an embodiment of the present invention.

[0032] System I comprises an electronic control unit 100.

[0033] The 3D recognition unit 110 is used to illuminate an object O using light L1 and to create a 3D image from the light L2 reflected by the object, which contains 3D information. The 3D recognition unit 110 is configured to detect reflected light using the time-of-flight technique. The 3D recognition unit 110 includes active light L1 for illuminating the object O. The light L1 can be any suitable light, such as infrared light, visible light, laser light, or the like. According to one embodiment, the 3D recognition unit 110 includes a means for illuminating an object O using infrared light.

[0034] According to one embodiment, the 3D recognition unit 110 comprises a time-of-flight camera unit. The time-of-flight camera can be any sensor that uses airborne signals and correlates the received signal to determine a distance to one or more measurement points. The time-of-flight (ToF) camera unit can be any ToF camera, based, for example, on the principles of continuous wave (CW, such as PMD, Swissranger, etc.), pulsed wave (such as TDC), or range gating (Obzerv). Time-of-flight cameras typically operate with a light-emitting diode that illuminates the entire scene simultaneously. According to one embodiment, the 3D recognition unit 110 comprises a LIDAR, i.e., a laser scanner unit.

[0035] The 3D recognition unit 110 is used to recognize a 3D image of an object O.

[0036] The system I comprises a means 112, 112a, 112b, 120, 130, 140 for calibrating the 3D recognition unit 110 for a specific 3D recognition unit position on a vehicle.

[0037] The means for calibrating the 3D recognition unit comprises a means 112 for detecting markers. The means 112 for detecting markers comprises a means 112a for detecting markers with a known minimum reflectivity as the lowest supported reflectivity using different integration times / exposure times for different 3D recognition unit positions. The different 3D recognition unit positions include the height above said markers and the direction of the 3D recognition unit. Said means 112 for detecting markers is included in said 3D recognition unit 110. The 3D recognition unit 110 is thus capable of detecting said markers.The system I and consequently the 3D recognition unit 110 comprises said means 112a for detecting markers with known minimum reflectivity as the lowest supported reflectivity using different integration times / exposure times.

[0038] According to one embodiment, the means 112 for detecting marks comprises means for detecting marks with a certain known reflectivity between the lowest and the highest reflectivity as a certain assisted reflectivity using different integration times / exposure times.

[0039] Consequently, the marker detection means 112 captures images with different integration times / exposure times for each position of the 3D recognition unit 110 to obtain distance data for each position.

[0040] The means 112 for detecting marks comprises a means 112b for detecting marks with different known minimum reflectances as the lowest supported reflectance corresponding to different object types using different integration times / exposure times.

[0041] The means for calibrating the 3D recognition unit 110 comprises a means 120 for obtaining reliable information from said marker detection. The system I thus comprises a means 120 for obtaining reliable information from said marker detection. The means 120 for obtaining reliable information from said marker detection comprises applying a quality measurement to each distance pixel for which the 3D recognition unit 110, i.e., the means 112a for detecting markers with a known minimum reflectivity, can estimate which pixels / measurements can be considered reliable. According to one embodiment, a margin is added to increase the robustness of the object detection.

[0042] The means 112a, 112b for detecting markings with known minimum reflectivity can operate with different illumination intensities of the 3D detection unit 110.

[0043] The means 112a, 112b for detecting markings with known minimum reflectivity can operate with different illumination patterns of the 3D detection unit 110.

[0044] The means 120 for obtaining reliable information from said mark detection comprises obtaining reliable information from different known minimum reflectivities as the lowest supported reflectivities corresponding to different object types.

[0045] When obtaining reliable information from different known minimum reflectivities as the lowest supported reflectivities corresponding to different object types, the object size is taken into account. According to one embodiment, the marker detection means 112 for object detection can operate with a specific pixel threshold, for example, 30 pixels, so that a cluster of pixels can be included in the object detection. Consequently, markers / objects that return a number of pixels below the threshold, for example, 20 pixels, are not detected. By taking the object size into account when creating a mask in this way, the quality / reliability can be significantly increased.

[0046] The detected pixel density can be used as a measure for setting the cutoff distance threshold for the mask. For example, if only 25% of the pixels provide sufficiently strong signals for a marker, and the pixel density at that distance is, say, 10 cm, the physical distance between two adjacent pixels is 10 cm, and the detectable object must have a size of 30 cm x 30 cm. This means that with 100% detection, 3x3 pixels, or 9 pixels, can be detected. If only 25% can be detected, this corresponds to 2 pixels. If the threshold is set to 5 pixels, the object cannot be detected at that distance.

[0047] The means for calibrating the 3D recognition unit 110 comprises a means 132 for creating different masks on the basis of the reliable information thus obtained for different 3D recognition unit positions, including the height above said markings and the direction of the 3D recognition unit. The system I thus comprises a means 132 for creating different masks on the basis of the reliable information thus obtained for different 3D recognition unit positions, including the height above said markings and the direction of the 3D recognition unit. When creating the masks, only the reliable pixels are taken into account. As soon as the 3D recognition unit 110, ieOnce the means 120 for obtaining reliable information from said marker detection has decided which pixels can be considered reliable, each mask is stored and assigned to the required integration time and the type of object used.

[0048] The means for calibrating the 3D recognition unit 110 comprises a means 134 for creating different masks based on the reliable information thus obtained for different markings corresponding to different object types. System I thus comprises a means 134 for creating different masks based on the reliable information thus obtained for different markings corresponding to different object types.

[0049] Means 132 and means 134 are included in a means 130 for creating masks. System I thus comprises a means 130 for creating masks based on reliable information for markings.

[0050] The means 130 for creating a mask comprises a means 130a for excluding information from the 3D recognition unit corresponding to pixels deemed unreliable. The means 130a for excluding information from the 3D recognition unit corresponding to pixels deemed unreliable may comprise any suitable filtering means.

[0051] The markers don't need to be continuous, as the 3D recognition unit, such as a time-of-flight camera, can interpolate / extrapolate a mask. The markers should enable the 3D recognition unit to create the information for the entire field of view, limited only by its illumination.

[0052] The marking detection means 112 comprises a means 112c for detecting markings with a known maximum reflectivity as the highest expected reflectivity using different integration times / exposure times. The system I, and consequently the 3D recognition unit 110, comprises said means 112c for detecting markings with a known maximum reflectivity as the highest expected reflectivity using different integration times / exposure times.

[0053] The means 112 for detecting marks comprises a means 112d for detecting marks with different known maximum reflectances as the highest expected reflectances corresponding to different object types, using different integration times / exposure times.

[0054] The means 112c, 112d for detecting markings with a known maximum reflectivity can operate with different illumination intensities of the 3D detection unit 110.

[0055] The means 112c, 112d for detecting markings with a known maximum reflectivity can operate with different illumination patterns of the 3D detection unit 110.

[0056] The system I comprises a means 140 for determining saturation characteristics, including a degree of saturation, from said mark detection to determine reliable detection.

[0057] The means 140 for determining saturation characteristics, including a degree of saturation, from said mark detection serves to ensure reliable detections for said different object types.

[0058] By determining saturation characteristics, including a degree of saturation, from said marker detection with said means 140 in this way, when the integration time for detecting an object with relatively low reflectivity, for example, the ground, is increased, the system I, including the 3D detection unit 110, knows when to expect that another object with relatively higher reflectivity, for example, a pedestrian, will not be detectable due to saturation. According to one embodiment, the system I, including the 3D detection unit 110, can be controlled such that the 3D detection unit 110 switches to the multiple exposure mode, the so-called HDR (High Dynamic Range), at a specific time to ensure that the most critical object type, for example, a pedestrian, is detectable at all times.

[0059] System I comprises means 100, 150 for storing the masks thus created. The means 100, 150 for storing the masks thus created comprises means 150 for externally storing the masks thus created, including an external storage means. Said means 150 for externally storing masks may comprise any suitable server unit, any suitable computer, or the like.

[0060] System I comprises means 100, 160 for storing the saturation characteristics thus determined. The means 100, 160 for storing the saturation characteristics thus determined comprises an external means 160 for storing the saturation characteristics thus determined. Said external means 160 may comprise any suitable server unit, any suitable computer, or the like.

[0061] The electronic control unit is connected via a connection 12a for signal transmission to means 112a, which serves to detect markers with a known minimum reflectivity as the lowest supported reflectivity using different integration times / exposure times, including the direction of the 3D recognition unit and the height above said markers. The electronic control unit 100 can receive a signal from said means 112a via connection 12a containing 3D image data for said markers for different integration times / exposure times for the various positions of the 3D recognition unit 110.

[0062] The electronic control unit 100 is connected via a connection 12b for signal transmission to means 112b, which serves to detect markers with different known minimum reflectances as the lowest supported reflectances corresponding to different object types using different integration times / exposure times. The electronic control unit 100 can receive a signal from said means 112b via connection 12b containing 3D image data for said markers for different integration times / exposure times for the different object types.

[0063] The electronic control unit 100 is connected via a connection 120a for signal transmission to the means 120, which serves to obtain reliable information from the aforementioned marker detection. The electronic control unit 100 can transmit a signal to the aforementioned means 120 via the connection 120a, which signal contains 3D image data for the aforementioned markers for different integration times / exposure times for the various positions of the 3D detection unit 110 and / or 3D image data for the aforementioned markers for different integration times / exposure times for the various object types.

[0064] The electronic control unit 100 is connected via a connection 120b for signal transmission to the means 120 for obtaining reliable information from said marker detection. The electronic control unit 100 can receive a signal from said means 120 via the connection 120b containing data regarding reliable information from said marker detection, including data regarding reliable information from the marker detection for different integration times / exposure times for the different positions of the 3D detection unit 110 and / or data regarding reliable information from the marker detection for different integration times / exposure times for the different object types.

[0065] The electronic control unit 100 is connected via a connection 132a to the means 132 for signal transmission, which serves to create different masks based on the thus obtained reliable information for different 3D recognition unit positions, including the direction of the 3D recognition unit and the height above said markings. The electronic control unit 100 can transmit a signal to said means 132 via the connection 132a, which signal contains mask creation data for the thus obtained reliable information for different 3D recognition unit positions.

[0066] The electronic control unit 100 is connected via a connection 132b to the means 132 for signal transmission, which serves to create different masks based on the thus obtained reliable information for different 3D recognition unit positions. The electronic control unit 100 can receive a signal from the means 132 via the connection 132b containing mask data for different masks based on the thus obtained reliable information for different 3D recognition unit positions.

[0067] The electronic control unit is connected via a connection 134a to the means 134 for signal transmission, which serves to create different masks based on the reliable information thus obtained for different markings corresponding to different object types. The electronic control unit 100 can transmit a signal containing mask creation data for the reliable information obtained for different object types to said means 134 via the connection 134a.

[0068] The electronic control unit is connected via a connection 134b to the means 134 for signal transmission, which serves to create different masks based on the reliable information thus obtained for different markings corresponding to different object types. The electronic control unit 100 can transmit a signal to the means 134 via the connection 134b, which signal contains mask data for different masks based on the reliable information thus obtained for different object types.

[0069] The electronic control unit 100 is connected via a connection 30a for signal transmission to means 130a, which serves to exclude information from the 3D recognition unit corresponding to pixels deemed unreliable. The electronic control unit 100 can transmit a signal to said means 130a via the connection 30a, which signal contains filtering data for excluding information from the 3D recognition unit corresponding to pixels deemed unreliable.

[0070] The mask data transmitted by said means 132 are filtered mask data when the means 130a has excluded information from the 3D recognition unit corresponding to pixels considered unreliable.

[0071] The mask data transmitted by said means 134 are filtered mask data when the means 130a has excluded information from the 3D recognition unit corresponding to pixels considered unreliable.

[0072] The electronic control unit 100 is connected via a connection 12c for signal transmission to means 112c, which serves to detect markers with a known maximum reflectivity as the highest expected reflectivity using different integration times / exposure times. The electronic control unit 100 can receive a signal from said means 112c via connection 12c containing 3D image data for said markers for different integration times / exposure times.

[0073] The electronic control unit 100 is connected via a connection 12d for signal transmission to means 112d, which serves to detect markers with different known maximum reflectances as the highest expected reflectances corresponding to different object types using different integration times / exposure times. The electronic control unit 100 can receive a signal from said means 112d via the connection 12d containing 3D image data for said markers for different integration times / exposure times for the different object types.

[0074] The electronic control unit 100 is connected via a connection 140a for signal transmission to the means 140, which serves to determine saturation characteristics, including a saturation level, from the aforementioned marker detection to ensure reliable detection. The electronic control unit 100 can transmit a signal containing 3D image data for the aforementioned markers, including 3D image data for the aforementioned markers for different integration times / exposure times for the different object types, to the aforementioned means 140 via the connection 140a.

[0075] The electronic control unit 100 is connected via a connection 140b for signal transmission to the means 140, which serves to determine saturation characteristics, including a saturation level, from said mark detection in order to ensure reliable detection. The electronic control unit 100 can receive a signal from said means 140 via the connection 140b, which contains data regarding the saturation from said mark detection, including saturation levels for different objects, in order to ensure reliable detection.

[0076] The means 130 for generating masks based on reliable information for markings is connected via a connection 150a for signal transmission to the means 150, which serves to store the masks thus generated. The means 130 can transmit a signal containing mask data for masks for different 3D recognition unit positions and / or mask data for masks for different objects to the means 150 via the connection 150a, wherein the means 150 serves to store the said masks.

[0077] According to one embodiment, the electronic control unit 100 can store masks for different 3D recognition unit positions, which are supplied to it as mask data from the means 132 via the connection 132b, and / or store masks for different object mask data, which are supplied to it from the means 134 via the connection 134b.

[0078] The means 130 for creating masks based on reliable information for markings is connected via a connection 150b for signal transmission to the means 150, which serves to externally store the masks thus created. The means 130 can receive a signal from said means 150 via the connection 150b containing mask data for masks for different 3D recognition unit positions and / or mask data for masks for different objects stored externally for a different 3D recognition unit and / or a different vehicle configuration, wherein the means 130 can validate said masks with masks created by the means 130.

[0079] The storage means 150, for example a server unit for external storage of masks, can store a plurality of masks for a plurality of positions and a plurality of objects, and according to one embodiment also for different illumination intensities and different illumination patterns, wherein the electronic control unit 100 can, according to one embodiment, participate in sorting out the masks so that mask data for a reduced number of masks are transmitted to the electronic control unit 100.

[0080] According to one embodiment, the electronic control unit 100 may request accessible masks based on the position, make an initial estimate, and then select the most suitable mask from the provided mask data.

[0081] According to one embodiment, the electronic control unit 100 makes an estimate and transmits the estimate, along with the position and any other relevant information, such as the object type and the type of 3D recognition unit, to the storage means 150, for example, an external server unit. The storage means 150 evaluates the masks and transmits data regarding the most suitable masks to the electronic control unit, which makes a further estimate that is correlated with the received data. If the correlation result is incorrect, another estimate is made.

[0082] The means 140 for determining saturation characteristics, including a saturation level, from said mark detection to ensure reliable detection is connected via a connection 160a to the means 160 for signal transmission, which serves to store the saturation characteristics thus determined. The means 140 can transmit a signal containing data regarding the saturation from said mark detection to said means 160 via the connection 160a to said means 160 to ensure reliable detection, wherein the means 160 can store said saturation level for said marks, including saturation levels for different objects. This can be particularly relevant when the illuminance and / or the illumination patterns are variable.

[0083] According to one embodiment, the electronic control unit 100 can store saturation levels for said markings, including saturation levels for different objects.

[0084] The means 140 for determining the saturation characteristics, including a degree of saturation, from said marker detection to ensure reliable detection is connected via a connection 160b for signal transmission to the means 160, which serves to store the saturation characteristics thus determined. The means 140 can receive a signal from said means 160 via the connection 160b containing data regarding the saturation from said marker detection that has been stored externally for another 3D recognition unit and / or another vehicle configuration, wherein the means 130 can validate said masks with masks created by the means 130.

[0085] Fig. Figure 3 is a schematic block diagram of a method for improving the reliability of image information from a 3D recognition unit for use in a vehicle. The 3D recognition unit illuminates the object and creates a 3D image containing 3D information from the light reflected from the object.

[0086] According to one embodiment, the method for improving the reliability of image information of a 3D recognition unit for use in a vehicle comprises a step S1. In this step, markers with different known minimum reflectivities are detected as the lowest supported reflectivities corresponding to different object types using different integration times / exposure times.

[0087] According to one embodiment, the method for improving the reliability of image information of a 3D recognition unit for use in a vehicle comprises a step S2. In this step, reliable information is obtained from the aforementioned marker recognition.

[0088] According to one embodiment, the method for improving the reliability of image information of a 3D recognition unit for use in a vehicle comprises a step S3. In this step, different masks are created based on the reliable information thus obtained for different markings corresponding to different object types.

[0089] Fig. Figure 4a is a schematic top view of a vehicle 1a with a 3D recognition unit according to the present invention. Different recognition areas R3, R4 are shown. The different markings S10, S20, S30 correspond to different objects, for example, ground / road for marking S10, pedestrians for marking S20, and highly reflective objects, such as road signs, for S30.

[0090] The markers don't need to be continuous, as the 3D recognition unit, such as a time-of-flight camera, can interpolate / extrapolate a mask. The markers / patterns should enable the 3D recognition unit to create the information for the entire field of view, limited only by its illumination.

[0091] The 3D recognition unit 110 is used to recognize markers S10, S20, S30 with different known minimum reflectivity as the lowest supported reflectivity corresponding to different object types, using different integration times / exposure times.

[0092] Then, reliable information is obtained from the aforementioned mark detection and different masks M1, M2, M3 are created for the aforementioned different object types based on the thus obtained reliable information.

[0093] Here, the markers S10, S20, and S30, which correspond to different object types, are shown simultaneously in one field for illustrative purposes. Markers corresponding to one object type can be recognized and then replaced by a marker corresponding to another object for recognition, and so on.

[0094] Fig. Figure 4b is a schematic side view of a vehicle 1b with a particular configuration for a particular position of a 3D recognition unit 110 according to the present invention. Fig. 4c is a side view of a vehicle 1c with a specific configuration for a specific position of a 3D recognition unit 110. Since the vehicle 1b is higher than the vehicle 1a, a correspondingly higher position of the 3D recognition unit 110 results.

[0095] The 3D recognition unit position at vehicle 1a is located at a height H1 above the ground O1. The ground or road forms an object O1. The 3D recognition unit position at vehicle 1a is directed toward the ground in a direction D1, which corresponds to a specific angle in the X, Y, Z planes.

[0096] The 3D recognition unit position at vehicle 1b is located at a height H2 above the ground O1. The ground or road forms an object O1. The 3D recognition unit position at vehicle 1b is directed toward the ground in a direction D2, which corresponds to a specific angle in the X, Y, Z plane.

[0097] The detection range R1 of the 3D recognition unit 110 mounted on the vehicle 1a may be different from the detection range R2 of the 3D recognition unit 110 mounted on the vehicle 1b.

[0098] The 3D recognition unit 110 is used to recognize markings S10 with a known minimum reflectivity as the lowest supported reflectivity using different integration times / exposure times. The known reflectivity of said markings S10 corresponds to a specific object O1, for example, the ground / road O1. The said recognition is performed for different 3D recognition unit positions, which are represented here as a position on vehicle 1a and a position on vehicle 1b.

[0099] Then, reliable information is obtained from the aforementioned marker detection, and different masks are created for the aforementioned other 3D detection unit positions based on the thus obtained reliable information.

[0100] Fig. 5 shows a diagram of a device 500. The device 500 shown in FIG. Fig.The control unit 100 described above may, according to one embodiment, include device 500. Device 500 includes a non-volatile memory 520, a data processing device 510, and a read / write memory 550. Non-volatile memory 520 has a first memory element 530 in which a computer program, such as an operating system, is stored to control the function of device 500. Device 500 also includes a bus controller, a serial communication port, I / O means, an A / D converter, a time and date acquisition and transmission unit, an event counter, and an interrupt controller (not shown). Non-volatile memory 520 also has a second memory element 540.

[0101] A computer program P is provided that includes routines for improving the reliability of image information from a 3D recognition unit for use in a vehicle. The program P includes routines for detecting markings with different known minimum reflectances as the lowest supported reflectances corresponding to different object types, using different integration times / exposure times. The program P includes routines for obtaining reliable information from said marking detection. According to one embodiment, the program P includes routines for creating different masks based on the reliable information thus obtained for different markings corresponding to different object types. The program P can be stored in executable or compressed form in a separate memory 560 and / or in the read / write memory 550.

[0102] When it is stated that the computing device 510 performs a particular function, this is to be understood as meaning that the computing device 510 executes a particular part of the program stored in a separate memory 560 or executes a particular part of the program stored in a read / write memory 550.

[0103] The data processing device 510 can communicate with a data communication port 599 via a data bus 515. The non-volatile memory 520 is used to communicate with the data processing device 510 via a data bus 512. The separate memory 560 is used to communicate with the data processing device 510 via a data bus 511. The read / write memory 550 is used to communicate with the data processing device 510 via a data bus 514. For example, the connections connected to the control unit 100 can be connected to the data communication port 599.

[0104] When data arrives at data port 599, it is temporarily stored in the second memory portion 540. Once the received input data has been temporarily stored, the data processing device 510 is ready to execute code as described above. The signals arriving at data port 599 can be used by the device 500 to detect marks with different known minimum reflectances as the lowest supported reflectances corresponding to different object types, using different integration times / exposure times. The signals arriving at data port 599 can be used by the device 500 to obtain reliable information from the aforementioned mark detection.The signals arriving at the data port 599 can be used by the device 500 to create different masks based on the reliable information thus obtained for different markings corresponding to different object types.

[0105] Portions of the methods described herein may be performed by device 500 using data processing device 510, which executes the program stored in separate memory 560 or read / write memory 550. When device 500 executes the program, portions of the methods described herein are performed.

[0106] The above description of the preferred embodiments of the present invention is provided for purposes of illustration and explanation. It is not intended to be an exhaustive description and is not intended to limit the invention to the described variants. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to best explain the principles of the invention and their practical applications, and to enable those skilled in the art to understand the invention for various embodiments and with various modifications as are suited to the intended use.

Claims

[1] A method for improving the reliability of image information of a 3D recognition unit (110) for use in a vehicle (1, 1a, 1b, 1c), in which the 3D recognition unit (110) serves to illuminate the object and to create a 3D image from the light reflected by the object, which contains 3D information, with a step in which the 3D recognition unit (110) is calibrated for a specific position of the 3D recognition unit (110) on the vehicle, characterized by a step for calibrating the 3D recognition unit (110) with the following sub-steps: Detecting (S1) markers (S10, S20, S30) using different integration times / exposure times, wherein the markers (S10, S20, S30) have a different known minimum reflectivity as the lowest supported reflectivity corresponding to different object types and representing the minimum number of pixels returned by said markers to be detected; Obtaining (S2) reliable information from said marker detection; Creation (S3) of different masks based on the reliable information thus obtained for different markings corresponding to different object types, so that the different masks are created for the different object types. [2] System (I) for improving the reliability of image information of a 3D recognition unit (110) for use in a vehicle (1, 1a, 1b, 1c), in which the 3D recognition unit (110) serves to illuminate the object and to create a 3D image from the light reflected by the object, which contains 3D information, with a means for calibrating the 3D recognition unit (110) for a specific position of the 3D recognition unit (110) on the vehicle, characterized by that the means for calibrating the 3D recognition unit (110) comprises: a means (112) for detecting markings (S10, S20, S30); and a means (112b) for detecting markers (S10, S20, S30) using different integration times / exposure times, wherein the markers (S10, S20, S30) have a different known minimum reflectivity as the lowest supported reflectivity corresponding to different object types and representing the minimum number of pixels returned by said markers to be detected; and characterized by : means (120) for obtaining reliable information from said marker detection; and means (134) for creating different masks on the basis of the reliable information thus obtained for different markings corresponding to different object types, so that the different masks are created for the different object types. [3] The system of claim 2, wherein the means (112) for detecting markers comprises means (112a) for detecting markers with known minimum reflectivity as the lowest supported reflectivity using different integration times / exposure times for different 3D recognition unit positions, including the height above said markers and the direction of the 3D recognition unit; wherein the means (120) is provided for obtaining reliable information from said marker detection; and wherein a means (132) is provided for creating different masks based on the reliable information thus obtained for different 3D recognition unit positions. [4] System according to claim 2 or 3, wherein the means (112a, 112b) for detecting markers with known minimum reflectivity can operate with different illuminances of the 3D recognition unit (110). [5] System according to one of claims 2-4, wherein the means (112a, 112b) for detecting markings with known minimum reflectivity is designed with different illumination patterns of the 3D recognition unit (110). [6] System according to one of claims 2-5, wherein the means (132, 134) for creating a mask comprises means (130a) for excluding information of the 3D recognition unit corresponding to pixels considered unreliable. [7] A system according to any one of claims 2-6, wherein the means (112) for detecting markers comprises means (112c) for detecting markers with known maximum reflectivity as the highest expected reflectivity corresponding to the maximum number of pixels returned from said markers using different integration times / exposure times; and means (140) for determining saturation characteristics, including a saturation level, from said marker detection to ensure reliable detection. [8] A system according to any one of claims 2-7, wherein the means (112) for detecting markers comprises means (112d) for detecting markers having different known maximum reflectances as the highest expected reflectances corresponding to different object types using different integration times / exposure times; and wherein means (140) is provided for determining saturation characteristics, including a saturation level, from said marker detection to ensure reliable detections for said different object types. [9] System according to one of claims 2-8, wherein the means (112c, 112d) for detecting markings with a known maximum reflectivity is designed with different illuminances of the 3D detection unit (110). [10] System according to one of claims 2-9, wherein the means (112c, 112d) for detecting markings with known maximum reflectivity is designed with different illumination patterns of the 3D recognition unit (110). [11] System according to one of claims 2-10, comprising means (150) for storing the masks thus created. [12] System according to one of claims 7-10, comprising means (160) for storing the saturation characteristics thus determined. [13] Vehicle (1) with a system (I) according to one of claims 2-12. [14] A computer program (P) for improving the reliability of image information of a 3D recognition unit (110) for use in a vehicle, comprising program code which, when executed by an electronic control unit (100) or by another computer (500) connected to the electronic control unit (100), causes the electronic control unit to perform the steps of claim 1. [15] Computer program product comprising a digital storage medium in which the computer program according to claim 14 is stored.

Images