Improved Periphery Detection Method

Abstract

To improve a periphery detecting method for vehicle to enhance a degree of adaptation of a generated periphery model.SOLUTION: A method by a system 7 comprises the steps of: using a first periphery detection sensor 1 which is an optical sensor to detect a periphery of a vehicle; generating a first periphery model; using an image evaluation unit 3 to calculate a region with a low degree of adaptation of the periphery model; taking a selected measure to improve a possibility of detection of the region with the low degree of adaptation; detecting a periphery of the vehicle again; comparing sensor data of the optical sensor before and after the measure is taken; and merging the sensor data after the measure is taken with the existing sensor data so as to generate a second periphery model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an improved perimeter detection method. [Background technology]

[0002] In the prior art, systems are known that calculate a model of the surroundings based on various sensor data and map data, and there is also the concept of a system that validates this model of the surroundings with external sources (other vehicles, infrastructure objects, vehicle-to-vehicle and road-to-vehicle communications) or extends it in a spatially narrow (static) area.

[0003] It is already possible at the current state of the art to validate or extend the accuracy of the characteristics of the surroundings detected by the target vehicle (ego vehicle). If the detection field of the sensor based on the current surroundings contains gaps or uncertainties, it is not possible at the current state of the art for the ego vehicle to reduce these gaps or uncertainties on its own. Summary of the Invention [Problem to be solved by the invention]

[0004] It is therefore an object of the present invention to provide an improved method for perimeter detection, thereby improving the fitness of the resulting generated perimeter model. [Means for solving the problem]

[0005] This problem is solved by the subject matter of the independent claims 1 and 9. Further advantageous configurations and embodiments are the subject matter of the dependent claims.

[0006] The surrounding model represents a framework for a (highly) automated vehicle capable of planning vehicle movements (trajectories). First, initial investigations based on the present invention confirmed that if the surrounding model contains inadequacies due to the current situation (poorly illuminated roadway areas, prediction range due to sensor range smaller than required for reliable trajectory planning), the (highly) automated vehicle can only compute a reduced number of possible vehicle trajectories, increasing the probability that it cannot provide an ideal, i.e. sufficiently comfortable or reliable trajectory from the real vehicle surroundings. Therefore, compared to the known prior art, the aim was to create the possibility of expanding the coverage and compatibility of the vehicle surroundings so that the surrounding model is more consistent with the real vehicle surroundings, thus significantly improving the probability of finding a sufficiently comfortable or sufficiently reliable trajectory.

[0007] Therefore, according to the invention, there is provided an improved perimeter detection method using at least one first perimeter detection sensor of a vehicle, the at least one first perimeter detection sensor being an optical sensor, - detecting the surroundings of the vehicle using at least one first surroundings detection sensor; - creating a first surroundings model based on sensor data of a first surroundings detection sensor; - calculating areas of low fit in the surrounding model by evaluating the optical sensor data using an image evaluation device; - improving the detectability of areas with low fitness by implementing selected measures; - detecting again the surroundings of the vehicle by means of at least one optical sensor; - comparing sensor data of the optical sensor after the implementation of the action with sensor data of the optical sensor before the implementation of the action; - fusing the post-measurement sensor data with existing sensor data of at least one first surroundings detection sensor (1) to create a second surroundings model.

[0008] Calculating areas with low fitness is advantageous because these areas cause uncertainty and inaccuracy in the surrounding model, which is disadvantageous for trajectory planning. Based on the calculation of these areas, measures or at least one measure can be implemented, which improves the detectability of these areas. This is advantageous for reducing the uncertainty or inaccuracy and thus increasing the fitness of the surrounding model.

[0009] In improving the detectability of areas with low goodness of fit, detectability is improved, especially by optical sensors.

[0010] During fusion, objects of the surroundings model detected by the optical sensor after processing are fused with the objects present. It is also conceivable to fuse the surroundings model from two sensors. In this case, a radar sensor, a lidar sensor or an ultrasonic sensor may be used as the second sensor. When using a second sensor, for example, the sensor data after processing may be fused with sensor data of a second surroundings detection sensor, which allows a more accurate surroundings model to be generated and thus the fitness of the surroundings model to be further improved.

[0011] In a particularly preferred embodiment, the measure comprises illuminating the area with low compatibility.In the context of the present invention, the concept of illumination is understood to mean not only illuminating the area with light in the visible band, but also illuminating with invisible light or illuminating with varying wavelengths.Particularly preferably, illumination with invisible light is first carried out.This is advantageous because in this way the driver is not stimulated or distracted by illumination.

[0012] In order to illuminate areas with a low degree of conformity, at least one illumination device of the vehicle is particularly preferably controlled.

[0013] In a further preferred embodiment, the at least one illumination device comprises a headlight, a high beam, a fog light, a matrix light, a laser light, an infrared light source and / or an additional light source mounted on the vehicle, which may for example emit light rays outside the visible band. In general, all of the above light sources are mounted on the vehicle such that they emit these light rays in a detection area of ​​at least one optical sensor mounted on the vehicle.

[0014] By targeted activation or brightening of headlights and fog lights, targeted parts of the detection area are illuminated by the forward-directed optical sensors, thus providing additional information about the surrounding characteristics and object distances in situations where the contrast ratio in natural lighting conditions contains insufficient information content.

[0015] The current state of the art does not take into account the use of light sources fixedly mounted on the vehicle, which actively illuminate the vehicle surroundings on demand. By actively controlling the mounted light sources and the actuators connected thereto, targeted illumination of under-illuminated areas in the detection field of the optical sensor is achieved without additional hardware costs. Obstacles are detected early, previously poorly illuminated areas can be used for trajectory planning, and by using the light sources already mounted, current systems can also be expanded in this respect.

[0016] In a further preferred embodiment, the additional light source is a light source configured to emit light rays at selectable frequencies, such that objects in the detected vicinity having known reflective properties, identified by the particular frequency, are highlighted.

[0017] By using different frequencies or wavelengths when actively illuminating the surroundings, it is also possible to infer the roadway type and roadway characteristics. For this purpose, impulses of specific wavelengths, for example infrared or ultraviolet light, are irradiated at predefined intervals and a unique image for each wavelength is detected using an image sensor with appropriate filters for these wavelengths. By alternating pulsing or impulse illumination with different wavelengths in subsequent image fusion, additional image or contrast information can be obtained, so that roadway and lane boundaries can be more clearly defined and information can also be extracted from the sensor images in cases where the sensor is not detectable in the visible light band, for example by oncoming vehicles.

[0018] For light of a different frequency or wavelength, or light of a different wavelength content than that in standard floodlights or natural light, the contrast difference will appear differently. For example, a scene in which a light gray road is separated by a light gray concrete fence has almost no contrast in natural lighting, but shows strong contrast under infrared light illumination.

[0019] Active illumination or pulse / impulse control of the on-board illumination means allows sudden roadway changes, such as holes in the road, holes dug by construction, jumps in the coefficient of friction, roadway boundaries, roadway defects, harbors, etc., to be better detected based on their reflection properties and taken into account in the surrounding model.

[0020] It is also conceivable to equip the vehicle with illumination means that take into account the pigments used for the lane markings and their reflection properties, and therefore with wavelengths adapted to the pigments. This allows the lane and lane markings to be highlighted specifically for the optical sensor by pulse-controlled or impulse illumination, and allows the recognition of different types of markings (normal ones, white markings, yellow construction markings and various common colors). Furthermore, clear lane markings for automated vehicles may be defined with reflection properties that are only visible to the camera of the (highly) automated vehicle with the corresponding illumination means. This allows individual roads or lanes to be prepared specifically for (highly) automated vehicles, without the human driver having to adjust to these change or additional markings. This is possible even without special pigments, since for example yellow markings a corresponding wavelength or frequency for yellow light is selected. In this way, the yellow lines appear brighter in the camera image.

[0021] Furthermore, preferably after the step of comparing the sensor data, it is checked whether there are further regions with low fitness, which will check whether the measures have been sufficient and whether the previously difficult to detect regions with low fitness now have a higher fitness. As further regions with low fitness, either new regions or previously already detected regions with low fitness can be understood.

[0022] Particularly preferably, the selected measures are repeated if further areas with a low degree of conformance are present. It is also conceivable to introduce other measures if further areas with a low degree of conformance are present or if the same areas still have a low degree of conformance. For example, data on these areas may be captured from the infrastructure or from other traffic participants.

[0023] Particularly preferably, when assessing the fitness of the optical sensor data, the contrast, the illumination of the detection area, the wavelength of the captured light beam and / or the distribution of the different detected wavelengths over the detection area are analyzed. This analysis is advantageous, since in this way it is possible to determine which illumination method or generally which measures are suitable for improving the detectability of areas with a low fitness.

[0024] Furthermore, according to the invention, a system for use in a vehicle is proposed, the system being adapted to carry out the method according to the invention, comprising at least one first and second perimeter detection sensor for detecting the perimeter of the vehicle, the first perimeter detection sensor being an optical sensor and the second perimeter detection sensor being a radar sensor, a lidar sensor or an ultrasonic sensor, an image evaluation device for evaluating and comparing the optical sensor data, at least one actuator for controlling at least one illumination device, and a computing unit for creating a perimeter model and fusing the sensor data, and the computing unit also sets parameters on how to control the illumination device by means of the actuator in order to target and illuminate the corresponding area.

[0025] Further advantageous embodiments can be seen from the drawings. [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 shows a schematic flow diagram of one embodiment of the present invention. [Diagram 2] FIG. 2 shows a schematic diagram of a system according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] FIG. 1 shows a schematic flow diagram of one embodiment of the invention. The flow diagram shows an improved method for perimeter detection using at least one first perimeter detection sensor of a vehicle, the at least one first perimeter detection sensor being an optical sensor. In step S1, the perimeter of the vehicle is detected using the at least one first perimeter detection sensor. In step S2, a first perimeter model is created based on sensor data of the at least one first perimeter detection sensor. In a next step S3, regions with a low fit in the perimeter model are calculated by evaluating the optical sensor data using an image evaluation device. In step S4, selected measures are taken to improve the detectability of the regions with a low fit. In a next step S5, the perimeter of the vehicle is detected again using at least the optical sensor. Then in step S6, the sensor data of the optical sensor after the measures are taken are compared with the sensor data of the optical sensor before the measures are taken. Finally, in step S7, the sensor data after the measures are fused with the existing sensor data of the at least one first perimeter detection sensor to create a second perimeter model.

[0028] 2 shows a schematic diagram of a system 7 according to one embodiment of the invention, where the system 7 is provided for use in a vehicle, the system is configured to perform the method according to the invention and comprises at least one first periphery detection sensor 1 and a second periphery detection sensor for detecting the surroundings of the vehicle, the first periphery detection sensor 1 being an optical sensor and the second periphery detection sensor 2 being a radar, lidar or ultrasonic sensor, an image evaluation device 3 for evaluating and comparing the optical sensor data, at least one actuator 5 for controlling at least one illumination device 6 and a computing unit 4 for creating a periphery model and fusing the sensor data. This application relates to the invention described in the claims, but also includes the following as other aspects. 1. An improved perimeter detection method using at least one first perimeter detection sensor (1) of a vehicle, said at least one first perimeter detection sensor (1) being an optical sensor, - detecting (S1) the surroundings of the vehicle using the at least one first surroundings detection sensor (1); - creating a first surroundings model based on sensor data of the surroundings detection sensor (1) (S2); - calculating (S3) areas of low fit in said surrounding model by evaluating optical sensor data using an image evaluation device (3); - a step (S4) of improving the detectability of said regions having a low fitness by implementing selected measures; - a step (S5) of detecting again the surroundings of the vehicle by means of at least one optical sensor (1); - a step (S6) of comparing the sensor data of the optical sensor (1) after the implementation of the measure with the sensor data of the optical sensor (1) before the implementation of the measure; - a step (S7) of fusing the sensor data after the measure with existing sensor data of the at least one first surroundings detection sensor (1) to create a second surroundings model. 2. 2. The method of claim 1, wherein said treatment comprises irradiating said areas having a low degree of fitness. 3. 3. The method according to claim 2, wherein at least one illumination device (6) of the vehicle is controlled to illuminate the areas having a low degree of conformity. 4. 4. The method according to claim 3, wherein the at least one illumination device (6) comprises a headlamp, a fog lamp, an infrared light source and / or an additional light source mounted on the vehicle. 5. 5. The method of claim 4, wherein the additional light source is a light source configured to emit light at a selectable frequency, such that objects in the detected vicinity having known reflective properties, identified by a particular said frequency, are highlighted. 6. 2. The method of claim 1, further comprising checking whether there are further regions with a low degree of fit after the step of comparing the sensor data. 7. 7. The method according to claim 6, wherein if there are further regions with low fitness, the selected measures are repeated or modified and applied. 8. 2. The method according to claim 1, wherein when assessing the fitness of the optical sensor data, the contrast, the illumination of the detection area, the wavelength of the captured light beam, the distribution of the different detected wavelengths in relation to the detection area and / or exceedance of a minimum greyscale threshold in the image area are analysed. 9. A system (7) for use in a vehicle, the system (7) being configured to carry out the methods described in 1 to 8 above, comprising at least one first periphery detection sensor (1) for detecting the periphery of the vehicle, the first periphery detection sensor (1) being an optical sensor, an image evaluation device (3) for evaluating and comparing the optical sensor data, at least one actuator (5) for controlling at least one illumination device (6), and a calculation unit (4) for creating a periphery model and fusing the sensor data. [Explanation of symbols]

[0029] 1 First peripheral detection sensor 2 Second peripheral detection sensor 3 Image Evaluation Unit 4 Calculation Unit 5 Actuators 6 Irradiation device 7. System S1~S7 Method steps

Claims

1. An improved perimeter detection method using at least one first perimeter detection sensor (1) of a vehicle, said at least one first perimeter detection sensor (1) being an optical sensor, - detecting (S1) the surroundings of the vehicle by means of said at least one first surroundings detection sensor (1); - creating (S2) a first surroundings model based on sensor data of said surroundings detection sensor (1); - a step (S3) of calculating areas in said surrounding model that have a low degree of conformance, i.e. are difficult to detect, by evaluating the optical sensor data using an image evaluation device (3); - a step (S4) of improving the detectability of said regions having a low fitness by implementing selected measures; - a step (S5) of detecting again said surroundings of said vehicle by means of at least one optical sensor (1), a step (S6) of comparing the sensor data of the optical sensor (1) after the implementation of the measure with the sensor data of the optical sensor (1) before the implementation of the measure, - fusing (S7) said sensor data after said measure with existing sensor data of said at least one first surroundings detection sensor (1) to create a second surroundings model, the step includes irradiating the area having a low degree of conformity; At least one illumination device (6) of the vehicle is controlled to illuminate the areas having a low degree of conformity, The at least one illumination device (6) includes headlights, fog lights, infrared light sources and / or additional light sources mounted on the vehicle; and the additional light source being configured to emit light at selectable frequencies, such that objects in the detected vicinity having known reflective properties, identified by a particular frequency, are highlighted; A method comprising:

2. The method of claim 1 , further comprising: after the step of comparing the sensor data, determining whether there are additional regions with a low goodness of fit.

3. The method of claim 2 , wherein if there are further regions with low fitness, the selected action is repeated or modified and applied.

4. The method of claim 1 , wherein when assessing the fitness of the optical sensor data, contrast, illumination of the detection area, wavelength of the captured light beam, distribution of the different detected wavelengths in relation to the detection area and / or exceedance of a minimum greyscale threshold in the image area are analysed.

5. A system (7) for use in a vehicle, the system (7) being configured to carry out the method according to claims 1 to 4, comprising at least one first periphery detection sensor (1) for detecting the periphery of the vehicle, the first periphery detection sensor (1) being an optical sensor, an image evaluation device (3) for evaluating and comparing optical sensor data, at least one actuator (5) for controlling at least one illumination device (6), and a computing unit (4) for creating a periphery model and fusing the sensor data.

Images