Method for detecting an incorrectly detected lane marking, method for indicating an incorrectly detected lane marking and control unit

Abstract

The invention relates to a method for detecting an incorrectly detected road marking, a method for indicating an incorrectly detected road marking and a control device (20) which is configured to carry out one or both of the said methods. It is intended that road markings that are only recognizable depending on the time of day are not taken into account as recognized road markings, since they are only incorrectly recognized as road markings that only arise due to lighting conditions that depend on the time of day.

Description

[0001] The invention relates to a method for detecting an incorrectly detected road marking, a method for indicating an incorrectly detected road marking and a control device which is configured to carry out one or both of the said methods.

[0002] Driver assistance systems are ubiquitous in today's vehicles. Driver assistance systems, for example, access the vehicle's own sensors, process the sensor data received, and intervene accordingly in the vehicle's driving operation. For example, lane departure warning systems intervene by steering the vehicle if the driver is about to leave the lane or keep the vehicle permanently in the center of the lane. For this purpose, lane markings detected by a vehicle camera are primarily used. If lane markings are detected where none actually exist, this usually leads to incorrect interventions by the respective driver assistance system. However, incorrect interventions in the vehicle's driving operation can confuse the driver and significantly reduce user acceptance of the driver assistance system.

[0003] Document DE 10 2021 001 451 A1 therefore proposes a device for lane estimation for a navigation system in a vehicle, in which contamination and fogging in the area of an optical axis of the vehicle's camera and the visibility conditions in the vehicle's surroundings are determined based on a contrast measure in the recorded camera image. Weighted environmental data are used for lane estimation, which are processed accordingly based on the determined contamination, fogging, and visibility conditions.

[0004] From DE 10 2009 039 450 A1 a method for lane recognition is known, whereby it is provided to determine not only lane markings and road edge markings but also reflector markings and to check the plausibility or correct the lane recognition based on the arrangement of the reflector markings.

[0005] From DE 10 2022 115 860 B3, a lane guidance assistance system with detection of lane boundaries is known, whereby it is intended to determine, in addition to the environmental sensor data for the actual lane detection, environmental conditions such as time of day and weather and to parameterize the lane detection depending on this.

[0006] The invention is based on the object of providing improved methods and an improved control unit which further reduce the occurrence of faulty interventions by driver assistance systems.

[0007] The object of the invention is achieved by a method for detecting an incorrectly detected lane marking, a method for indicating an incorrectly detected lane marking, and a control unit for operating a motor vehicle according to the independent patent claims. Preferred developments are the subject of the respective dependent claims.

[0008] A first aspect relates to a method for detecting an incorrectly detected lane marking for a motor vehicle. Examples of lane markings are lane markings, such as lane shoulders and median strips, and symbols such as directional arrows, warning signs, speed limits, and the like. The motor vehicle comprises an environment sensor unit configured to detect lane markings and a memory unit with stored information about a plurality of detected lane markings. The environment sensor unit preferably comprises one or a plurality of cameras configured to generate camera images of lane markings and to detect the lane markings based on image processing.

[0009] The stored information about the multitude of detected lane markings includes location information, associated geometric shapes, and associated timestamps for each of the detected lane markings. In other words, the stored information about the multitude of detected lane markings represents swarm data. Swarm data is usually anonymized vehicle environment information provided by a multitude of motor vehicles and collected by the individual motor vehicles. The vehicle environment information includes, for example, camera data continuously generated by the vehicle's own cameras while driving. Usable data such as lane markings along the roadway, associated lane markings, line types, and the like can be obtained from the aggregation and processing of the swarm data.Such usable data is stored at least partially as the information about the plurality of detected lane markings in the memory unit of the motor vehicle.

[0010] In one method step, a lane marking is detected using the environment sensor unit. For this purpose, at least one associated location information item, a geometric shape of the detected lane marking, and the associated timestamp are determined. For example, a lane marking, such as a lane shoulder and / or a median strip, and / or a symbol, such as a directional arrow, a warning sign, a speed limit, and the like, is detected. The detection of lane markings, their positions, and their geometric shapes using an environment sensor unit are generally known and will therefore not be described in detail.

[0011] According to a further step of the method, the detected road marking is rejected under the conditions that the detected road marking matches one of the plurality of road markings in terms of position and geometric shape, and that the matching road marking is only detectable depending on the time of day. "Dependent on the time of day" can refer to a point in time or to a specific period of a day. In other words, a check is carried out to determine whether and at what time of day the road marking detected by the environment sensor unit has already been detected by the swarm, i.e., by other motor vehicles.If, for example, a road marking is repeatedly detected at the same time or within a certain time range, it can be concluded that this road marking was simply mistakenly identified as a road marking that only occurs due to daytime-dependent lighting conditions. For example, problems with the detection of road markings often arise when guardrails with multiple struts are positioned next to the road. When sunlight falls through these struts, a bright, marking-wide band of light often appears on the road between the shadows cast by the struts. Many cameras record the differences in brightness of such bands of light and incorrectly identify the bright bands of light as road markings. Another example is tar seams in the road, which reflect sunlight when the sun is low in the sky, making them appear significantly brighter than the rest of the road.In general, it can be said that in the case of phenomena caused by sunlight that are mistakenly identified as lane markings, a higher ambient brightness can usually be detected. Therefore, the brightness values preferably do not refer exclusively to the brightness values of the lane markings, but additionally or alternatively to the brightness values of the vehicle's surroundings.

[0012] The invention thus takes advantage of the fact that road markings are fundamentally permanently present on the roadway and not phenomena that arise due to daytime-dependent lighting conditions. It is therefore intended that road markings that are recognizable depending on the time of day should not be considered as recognized road markings, i.e. that they were merely incorrectly recognized as road markings. This can further reduce the occurrence of erroneous interventions by driver assistance systems. Most daytime-dependent phenomena are generally dependent on the position of the sun and the resulting lighting conditions. The rejection of the road marking detected by the environment sensor unit therefore preferably occurs under the additional condition that the recognized road marking was detected during the day, i.e. not at night.

[0013] In a preferred embodiment, the information about the plurality of detected road markings further includes markings that indicate whether the respective road marking is recognizable depending on the time of day. The rejection of the road marking detected by the environment sensor unit preferably occurs based on the associated marking of the matching road marking. In other words, the swarm data has already been aggregated and evaluated in advance, so that the usable data further contains the marking, also called markers, that immediately determine whether the road marking is dependent on the time of day or not. Thus, detected road markings can be quickly rejected if the identical road marking detected by the swarm is stored with the marking.This reduces the computing time required in the detection situation and allows an incorrectly detected lane marking to be rejected, thus improving the reaction time of the driver assistance system.

[0014] In a further preferred embodiment, the rejection of the road marking detected by the environment sensor unit occurs under the additional condition that the timestamp of the road marking detected by the environment sensor unit essentially matches the timestamp of the matching road marking or lies within a time period in which the matching road marking was detected. Consequently, only detected road markings that match the road markings detected by the swarm in terms of time of day are rejected.For example, if a lane marking is detected by the environment sensor unit at night, while the swarm has only ever detected this lane marking during the day, the detected lane marking is not discarded, as it is necessary to question whether the detected lane marking could actually be an incorrectly detected lane marking. This further improves the process and further reduces the occurrence of erroneous interventions by driver assistance systems.

[0015] In a further preferred embodiment, the environment sensor unit comprises a brightness sensor, and the road marking is detected based on a detected brightness value that exceeds a predetermined brightness value. The detection of road markings based on brightness or contrast values represents a reliable method.

[0016] In a further preferred embodiment, the information about a plurality of detected lane markings further includes associated brightness values detected (by the swarm), and the lane marking detected by the environment sensor unit is discarded if the detected brightness value of the detected lane marking is greater than or equal to the associated detected brightness value of the matching lane marking. In other words, it is ensured that only detected lane markings are discarded that were detected at least with the reliability of the swarm with regard to the detected brightness value, or even more clearly. This further improves the method and further reduces the occurrence of erroneous interventions by driver assistance systems.

[0017] In a further preferred embodiment, the motor vehicle further comprises a Car2X communication unit configured to transmit and receive Car2X radio signals. Car2X ("Car to x") is communication between vehicles and their environment ("x"), for example with other vehicles and / or with a stationary radio station. According to a preferred method step, the detected lane marking is transmitted with associated location information, geometric shape, and timestamp using the Car2X communication unit. In other words, the motor vehicle forms part of the swarm, which enriches the swarm data with further information, thereby improving the quality of the usable data and ultimately further reducing the occurrence of erroneous interventions by driver assistance systems.Preferably, the Car2X communication unit is used to receive the stored information about the plurality of detected lane markings, i.e. swarm data, preferably the usable data, and to store it in the storage unit and / or to update the stored information about the plurality of detected lane markings.

[0018] In a further preferred embodiment, it is provided that the detected lane marking that has not been rejected is used for a driver assistance system. The detected lane marking that has not been rejected is preferably linked to a further identifier that indicates that the detected lane marking has been checked and classified as not faulty. Consequently, the driver assistance system can grade or weight the detected lane markings used, which are dependent on the further identifier, and thus make better decisions. The driver assistance system is preferably a lane keeping and / or lane control assistant that is configured to keep the motor vehicle in the detected lane and / or to keep the motor vehicle in a predetermined position within the detected lane, for example, in the center of the lane.

[0019] Preferably, a suitable lane marking is selected from the stored information about the plurality of detected lane markings and used for the driver assistance system if no lane marking is detected by the environment sensor unit. This ensures that a lane marking is available to the driver assistance system even if the environment sensor unit is unable to detect a lane marking.

[0020] A further aspect of the invention relates to a further method, namely a method for indicating an incorrectly detected lane marking. The method is preferably carried out by the aforementioned motor vehicle and / or an external server.

[0021] In a further step of the method, a plurality of detected lane markings with associated location information, geometric shapes, and timestamps are received from a plurality of motor vehicles using a Car2X communication unit. In other words, the swarm data from the plurality of vehicles is combined, i.e., aggregated.

[0022] In a further process step, it is checked whether one of the received road markings is only recognizable depending on the time of day based on the associated time stamps. In other words, identical road markings are searched for in the swarm data and the associated time stamps are examined. If the time stamps are widely distributed throughout the day, for example, it can be assumed that the associated road marking is a permanent, i.e., actual, road marking. However, if time stamps are only available for one point in time or a defined period of time during the day, it can be assumed that the associated road marking was incorrectly recognized as a road marking that only appeared to have appeared, for example, due to daytime-dependent lighting conditions.

[0023] In a further step of the method, the lane marking, which is only recognizable depending on the time of day, is linked to a label indicating that the lane marking is only recognizable depending on the time of day. Linking the label thus indicates incorrectly recognized lane markings. Preferably, the lane markings with the linked label are transmitted via Car2X communication, for example, to motor vehicles, where they are stored as usable data in storage units.

[0024] A further aspect of the invention relates to a control unit for a motor vehicle. The control unit is configured to carry out the above-described method for detecting an incorrectly detected lane marking and / or the above-described method for indicating an incorrectly detected lane marking. The motor vehicle is preferably the motor vehicle described above. The advantages achieved with the methods can be achieved analogously with the control unit. The disclosed feature combinations of the methods can be transferred analogously to the control unit. A repeated description of the features and advantages is therefore omitted.

[0025] The individual method steps of the methods according to the invention are further preferably embodied as one or more processes that run on one or more processors in one or more electronic computing devices and are generated during the execution of one or more computer programs. The computing devices are preferably designed to cooperate with other components, in particular a control unit for operating a motor vehicle, in order to implement the functionalities described herein. Likewise, the control unit is preferably embodied as a central (single-part) or as a decentralized (multi-part) component.

[0026] The individual components of the control unit are further preferably embodied as one or more processes that run on one or more processors in one or more electronic computing devices and are generated during the execution of one or more computer programs. The computing devices are preferably designed to cooperate with other components, for example an environmental sensor unit configured to detect lane markings, in order to implement the functionalities described herein. The instructions of the computer programs are also preferably stored in a memory, such as a RAM element. However, the computer programs can also be stored in a non-volatile storage medium, such as a CD-ROM, a flash memory, or the like.

[0027] It will also be apparent to those skilled in the art that the functionalities of multiple computers (data processing devices) may be combined or combined in a single device, or that the functionality of a particular data processing device may be distributed among a plurality of devices in order to carry out the steps of the methods according to the invention without departing from the methods according to the invention.

[0028] A further aspect of the invention relates to a computer program comprising instructions which, when the program is executed by a computer, such as a control unit for operating a motor vehicle, cause the computer to carry out one of the methods mentioned or both of the methods mentioned herein, in particular a method for detecting an incorrectly detected road marking and / or a method for indexing an incorrectly detected road marking.

[0029] Further preferred embodiments of the invention result from the remaining features mentioned in the subclaims.

[0030] The various embodiments of the invention mentioned in this application can be advantageously combined with one another, unless stated otherwise in the individual case.

[0031] The invention is explained below in exemplary embodiments with reference to the accompanying drawings. They show: Fig. 1 a schematic representation of a method for detecting an incorrectly detected road marking for a motor vehicle according to one embodiment, Fig. 2 a schematic representation of a method for indicating an incorrectly detected road marking according to an embodiment and Fig. 3 a schematic representation of a motor vehicle with a control unit for operating the motor vehicle according to an embodiment.

[0032] Fig. Figure 1 shows a schematic representation of a method for detecting an incorrectly detected lane marking for a motor vehicle 10 according to one embodiment. The motor vehicle 10 comprises an environment sensor unit 12 configured to detect lane markings, a memory unit 14, a Car2X communication unit 16 configured to transmit and receive Car2X radio signals, a driver assistance system 18, and a control unit 20 (see Fig. 3). The control unit 20 is connected to the other components 12, 14, 16, and 18, respectively. The environment sensor unit 12 comprises a plurality of cameras configured to generate camera images of lane markings and to detect the lane markings based on image processing. The storage unit 14 comprises stored information about a plurality of detected lane markings, which includes location information associated with the lane markings, associated geometric shapes, and associated time stamps. The Car2X communication unit 16 is configured, in particular, to receive the information about the plurality of detected lane markings or updated information about the plurality of detected lane markings via Car2X communication and to transmit it via the control unit 20 to the storage unit 14 so that it can be stored there.The driver assistance system 18 is a lane keeping assistant configured to keep the motor vehicle 10 in the lane. To this end, the driver assistance system 18 intervenes by steering the driving operation of the motor vehicle 10 if the vehicle is about to leave the lane. To determine whether the motor vehicle 10 is about to leave the lane, the driver assistance system 18 accesses the sensor data relating to the detected lane markings from the environment sensor unit 12. To reduce the occurrence of erroneous interventions by the driver assistance system 18 in the driving operation of the motor vehicle 10, the methods described below and the control unit 20 described below are proposed.

[0033] According to a first method step 50 of the method for detecting an incorrectly detected lane marking, a lane marking is detected using the environment sensor unit 12. For this purpose, at least one associated location information item, a geometric shape of the detected lane marking, and the associated time stamp are determined.

[0034] According to a second step 52 of the method, the detected road marking is rejected under the conditions that the detected road marking matches a road marking from the plurality of road markings in terms of position and geometric shape, and that the matching road marking is only detectable depending on the time of day. In other words, a check is performed to determine whether and at what time of day the road marking detected by the environment sensor unit 12 has already been detected by the swarm, i.e., by other motor vehicles. If, for example, a road marking was repeatedly detected at the same time or within a certain time range, it can be concluded that this road marking was merely mistakenly detected as a road marking that only arises due to daytime-dependent lighting conditions.For example, problems often arise with the detection of lane markings when guardrails with multiple struts are positioned next to the road. When sunlight shines through these struts, a bright, marking-wide band of light often appears on the road between the shadows of the struts. Many cameras detect the brightness differences of such bands of light and incorrectly interpret the bright bands of light as lane markings. Another example is tar seams in the road, which, due to the low position of the sun, reflect sunlight in such a way that they appear significantly brighter than the rest of the road.

[0035] According to a third optional method step 54, the detected lane marking that has not been rejected is transferred to the driver assistance system 18 and used by the driver assistance system 18 to maintain the lane. The transfer of rejected detected lane markings to the driver assistance system 18 is thus prevented in order to reduce the occurrence of erroneous interventions by the driver assistance system 18 in the driving operation of the motor vehicle 10.

[0036] Fig. Figure 2 shows a schematic representation of a method for indexing an incorrectly detected lane marking according to one embodiment. The method is performed by, but is not limited to, an external server.

[0037] In a first step 60 of the further method, a plurality of detected lane markings with associated location information, geometric shapes, and timestamps are received from a plurality of motor vehicles using a Car2X communication unit. In other words, the swarm data from the plurality of vehicles is combined, i.e., aggregated.

[0038] In a second method step 62, a check is performed to determine whether one of the received lane markings is only recognizable depending on the time of day based on the associated time stamps. In other words, identical lane markings are searched for in the swarm data, and the associated time stamps are examined. If, for example, the time stamps are widely distributed throughout the day, it can be assumed that the associated lane marking is a permanent, i.e., actual, lane marking. However, if only time stamps for one point in time or a defined period of the day are present, it can be assumed that the associated lane marking was mistakenly recognized as a lane marking that only supposedly appeared, for example, due to daytime-dependent lighting conditions.

[0039] In a third step 64 of the further method, the lane marking, which is only recognizable depending on the time of day, is linked to a marking indicating that the lane marking is only recognizable depending on the time of day. The linking to the marking thus indicates incorrectly recognized lane markings. The lane markings with linked markings are transmitted to the motor vehicle 10 via Car2X communication according to Fig. 3 to be stored in the storage unit 14 as usable data and for the Fig. 1 described method to be used.

[0040] Fig. Figure 3 shows a schematic representation of the previously described motor vehicle 10 with the control unit 20 for operating the motor vehicle 10 according to an embodiment. The control unit 20 is particularly designed for carrying out the Fig. 1 and / or the method described in Fig.2 procedure. List of reference symbols 10 motor vehicle 12 Environment sensor unit 14 storage unit 16 Car2X communication unit 18 Driver assistance system 20 Control unit 50 first process step - detection of a road marking 52 Second process step - Discarding the detected lane marking 54 Third step - Using the non-rejected lane marking 60 first process step - receiving detected lane markings 62 Second step - Checking the received lane markings 64 third process step - linking the time-of-day dependent road marking

Claims

[1] A method for detecting an incorrectly detected road marking for a motor vehicle (10), wherein the motor vehicle (10) has an environmental sensor unit (12) configured to detect road markings and a memory unit (14) with stored information about a plurality of detected road markings, which includes location information associated with the road markings, associated geometric shapes and associated time stamps, and the method comprises the steps of: - detecting (50) a lane marking using the environment sensor unit (12), - rejecting (52) the detected road marking under the conditions that the detected road marking matches a road marking from the plurality of road markings with regard to position and geometric shape and that the matching road marking is only recognizable depending on the time of day. [2] The method according to claim 1, wherein the information about the plurality of recognized road markings further comprises respective markings indicating whether the respective road marking is recognizable depending on the time of day, and the rejection (52) of the road marking recognized by the environment sensor unit (12) is carried out based on the associated marking of the matching road marking. [3] Method according to one of the preceding claims, wherein the discarding (52) of the road marking detected by the environment sensor unit (12) takes place under the additional condition that the time stamp of the road marking detected by the environment sensor unit (12) substantially matches the time stamp of the matching road marking or lies in a time period in which the matching road marking was detected. [4] Method according to one of the preceding claims, wherein the environment sensor unit (12) comprises a brightness sensor and the road marking is detected based on a detected brightness value that exceeds a predetermined brightness value. [5] The method according to claim 4, wherein the information about a plurality of detected road markings further comprises associated detected brightness values, and the road marking detected by the environment sensor unit (12) is discarded if the detected brightness value of the detected road marking is greater than or equal to the associated detected brightness value of the matching road marking. [6] Method according to one of the preceding claims, wherein the motor vehicle (10) further comprises a Car2X communication unit (16) configured to transmit and receive Car2X radio signals, and the method further comprises the step: - Transmitting the detected lane marking with associated location information, geometric shape and time stamp using the Car2X communication unit (16). [7] Method according to one of the preceding claims, further comprising the step: - Using (54) the non-rejected detected lane marking for a driver assistance system (18) of the motor vehicle (10). [8] Method according to claim 7, wherein a suitable road marking is selected from the stored information about the plurality of detected road markings and used for the driver assistance system (18) if no road marking is detected by the environment sensor unit (12). [9] A method for indexing an incorrectly detected road marking, comprising the steps of: - receiving (60) a plurality of detected lane markings with associated location information, geometric shapes and time stamps from a plurality of motor vehicles (10) using a Car2X communication unit (16), - Checking (62) whether one of the received lane markings is only recognizable depending on the time of day based on the associated time stamps, - linking (64) the road marking which is only visible depending on the time of day with a marking which indicates that the road marking is only visible depending on the time of day. [10] Control device (20) for operating a motor vehicle (10), which is designed to carry out the method according to one of claims 1 to 8 and / or the method according to claim 9.

Images