Method for operating a driver assistance device for a motor vehicle, driver assistance device for a motor vehicle and computer program product

Abstract

The invention relates to a method for operating a driver assistance device for a motor vehicle, wherein environmental data describing an environment (2) of the motor vehicle are recorded at least temporarily by means of an environment detection device of the motor vehicle.It is provided that, based on the environmental data, at least temporarily a road marking (5, 6, 7) of a roadway (3) traveled on by the motor vehicle and a reference object (8, 9) are detected and a relative position of the road marking (5, 6, 7) relative to the reference object (8, 9) is determined, wherein the relative position together with an object data set describing the reference object (8, 9) is stored in a data storage as part of a marking data set describing the road marking (5, 6, 7); and / or that, based on the environmental data, the reference object (8, 9) is detected and the marking data set containing the object data set describing the reference object (8, 9) is read from the data storage and a position of the road marking (5, 6, 7) is determined based on the relative position of the road marking (5, 6, 7) relative to the reference object (8, 9).The invention further relates to a driver assistance device for a motor vehicle and a computer program product.

Description

[0001] The invention relates to a method for operating a driver assistance system for a motor vehicle, wherein environmental data describing the environment of the motor vehicle are acquired, at least temporarily, by means of an environment sensing device of the motor vehicle. The invention further relates to a driver assistance system for a motor vehicle and a computer program product.

[0002] For example, US Patent 11,232,600 B2 is known from the prior art. This document describes a head-up display arrangement for a motor vehicle, wherein the display arrangement comprises: a global positioning system module configured to output geographic location coordinates associated with the motor vehicle; an electronic processing device communicatively coupled to the positioning system module, wherein the electronic processing device is configured to: receive the geographic location coordinates; retrieve lane marking location data stored in conjunction with the received geographic location coordinates from a database; and transmit the retrieved lane marking location data.

[0003] The head-up display arrangement further comprises a head-up display that is communicatively coupled to the electronic processing device and configured to receive the transmitted lane marking location data and to display virtual lane markings depending on the received lane marking location data, wherein the virtual lane markings include a transparent colored highlight.

[0004] Further state of the art is known from publications US 11,248,925 B2, DE 10 2009 046 699 B4 and US 10,633,027 B2.

[0005] The object of the invention is to propose a method for operating a driver assistance device for a motor vehicle which has advantages over known methods, in particular enabling reliable detection of a road marking, preferably independent of environmental conditions.

[0006] According to the invention, this is achieved by a method for operating a driver assistance system for a motor vehicle with the features of claim 1. It is provided that, based on the environmental data, a lane marking of a roadway traveled by the motor vehicle and a reference object are detected at least temporarily, and a relative position of the lane marking relative to the reference object is determined, wherein the relative position, together with an object data set describing the reference object, is stored in a data memory as part of a marking data set describing the lane marking; and / or that, based on the environmental data, the reference object is detected, and the marking data set containing the object data set describing the reference object is read from the data memory, and a position of the lane marking is determined based on the relative position of the lane marking relative to the reference object.

[0007] Advantageous embodiments with expedient further developments of the invention are specified in the dependent claims. It should be noted that the exemplary embodiments described in the description are not limiting; rather, any variations of the features disclosed in the description, the claims, and the figures are possible.

[0008] The method serves to operate a driver assistance system. The driver assistance system, or a control unit of the driver assistance system used to carry out the method, is preferably an integral part of the motor vehicle, but can of course also exist separately, particularly until the driver assistance system or the control unit is mounted on or in the motor vehicle. The driver assistance system serves, for example, to support a user of the motor vehicle while driving and / or at least temporarily to carry out the driving of the motor vehicle itself, which in this case is at least partially autonomous. Partially autonomous driving means that the driver assistance system independently takes over longitudinal and / or lateral control of the motor vehicle, at least temporarily.

[0009] Longitudinal guidance refers in particular to setting the vehicle's speed, for example, by appropriately controlling the vehicle's drive system and / or braking system. The drive system serves to propel the vehicle, thus providing the drive torque for propelling the vehicle. To provide the drive torque, the drive system preferably has at least one drive unit, which can be configured as an internal combustion engine or an electric traction motor. The braking system, on the other hand, serves to brake or decelerate the vehicle. It includes, in particular, a service brake, which exerts a braking force on one or more wheels of the vehicle to slow it down.

[0010] Supporting the user in carrying out driving operations corresponds, for example, to assisted driving according to SAE Level 1. At least partially autonomous driving operations occur in particular within the framework of partial automation according to SAE Level 2, conditional automation according to SAE Level 3, high automation according to SAE Level 4, or full automation according to SAE Level 5. Whenever autonomous driving is mentioned in this description, it always refers to at least partially autonomous driving as defined above.

[0011] The vehicle, and in particular its driver assistance system, is equipped with an environmental sensing system that serves to detect the vehicle's surroundings. This system generates environmental data that describes the vehicle's environment. To detect the environment and provide this data, the environmental sensing system has one or more environmental sensors. Examples of sensors used as environmental sensors include: radar sensor, sonar sensor, lidar sensor, and image sensor or camera.

[0012] The radar sensor uses electromagnetic waves, the sonar sensor uses sound waves, and the lidar sensor uses light to detect the environment, particularly to identify obstacles. The sonar sensor, for example, is designed as an ultrasonic sensor, meaning it uses sound waves in the ultrasonic range. The radar sensor, sonar sensor, and lidar sensor are all reflectivity sensors, meaning they are active sensors that transmit an output signal into the environment and receive the reflected output signal as an input signal.

[0013] For example, the driver assistance system is designed to use environmental data to perform assisted driving and / or autonomous driving, at least temporarily. This means that environmental data is evaluated, particularly regarding one or more obstacles in the vicinity of the vehicle. Based on this, in the case of assisted driving, helpful information is displayed to the vehicle's user, or in the case of autonomous driving, longitudinal and / or lateral guidance is provided. During autonomous driving, standard traffic rules are preferably observed. For example, the vehicle stays in a specific lane, adheres to its lane, and obeys traffic signs and signals. This ensures the safe flow of road traffic.

[0014] Especially for the implementation of a lane keeping assist system, it is necessary to continuously and accurately detect the road markings of the lane being traveled by the vehicle. Road markings include, for example, lane boundaries, lane markings, or guide lines. Road markings, which can also be referred to as road markings, are primarily line-like markings applied to the road surface. Detecting the road markings is performed in the first stage of the process or driver assistance system using environmental data provided by the environmental sensing device.

[0015] Preferably, the image data provided by the image sensor is evaluated to detect the lane markings and determine their position. Position refers specifically to an absolute position in a geodetic context or a relative position to the vehicle. The position can also be, or describe, the path of the lane markings. The path describes the course of the lane markings in the vicinity of the vehicle. Assisted driving or at least partially autonomous driving is carried out based on the lane markings, or rather their position and / or path.

[0016] For example, the detected lane markings are used as the basis for the lane keeping assist system. However, this is only possible if the lane markings are reliably detected. This is particularly the case under good environmental conditions, such as a dry road surface and / or daylight. The lane markings must also be of sufficiently high quality and clearly distinguishable from the road surface so that they can be detected using the environmental data.

[0017] According to the invention, after recognizing the road marking based on the environmental data, the reference object is additionally detected. The reference object is understood to be an object different from the road marking that is present in the vicinity of the motor vehicle. Preferably, the reference object has a three-dimensional shape, in contrast to the quasi-two-dimensional road marking. The reference object thus has, for example, a non-zero vertical extent. For instance, the reference object originates from the road surface or at least from a surface on which the road surface is also located. With this design of the reference object, it remains recognizable even if the road marking is obscured, for example, by dirt, leaves, or snow.

[0018] However, it is also possible to use a different (additional) lane marking as a reference object. For example, it is possible to reconstruct lane markings that are not recognized, at least in sections, using this additional lane marking. In this case, the lane marking is, in particular, a guide line, preferably a center line, and the additional lane marking is a lane boundary, which can also be referred to as an edge line. The lane marking is thus located using any reference object, allowing its position to be determined with high accuracy, even if the lane marking itself is not recognized or is only recognized with low accuracy using the image data. If both the lane marking and the reference object have been detected using the environmental data, the relative position of the lane marking is determined relative to the reference object.This means that, using the environmental data, not only is the presence of the road marking and the reference object detected, but their relative positions are also determined. For example, it is intended to determine both the position of the road marking and the position of the reference object, in particular a relative position of the road marking and a relative position of the reference object, preferably each relative to the motor vehicle. From these positions, the relative position of the road marking with respect to the reference object is then determined.

[0019] In addition to the relative position, the object data record, which describes the reference object, is determined based on the environmental data. At least one parameter of the reference object, such as its geometry, size, or similar, is thus determined from the environmental data and stored in the object data record. The object data record is chosen such that the reference object can be uniquely identified using the environmental detection device and the environmental data it provides. The information obtained, i.e., at least the relative position and the object data record, is stored in the marking data record describing the road marking.

[0020] The marking data set preferably comprises at least one parameter of the road marking, for example, a marking position, a marking type, and / or a marking path. The marking type describes the meaning of the road marking; for example, it indicates whether the road marking is in the form of a lane boundary, a lane marking, and / or a guide line. The marking data set is stored in the data storage device. The data storage device can be part of the driver assistance system or be located separately from the vehicle, in particular as part of a central processing unit, for example, a stationary central processing unit. The central processing unit is preferably understood to be a computing device that has higher computing power and / or a larger storage capacity than the control unit of the driver assistance system.

[0021] In addition to or as an alternative to the first part of the process, a second part involves, at least temporarily, using the surrounding data to detect the reference object and, using the detected reference object, reading the marking data record from the data storage that contains the object data record describing the reference object. In other words, the system checks whether a marking data record exists in the data storage for the detected reference object that contains the object data record belonging to the reference object. If so, the marking data record is read and evaluated. In this process, the marking data record is used to determine the position of the road marking relative to the reference object based on the relative position of the road marking contained within it.

[0022] Ultimately, the road markings are not directly detected using the environmental data; rather, their presence and position are inferred indirectly. For this purpose, the marking data set stored in the data storage is used together with the reference object or its position. Therefore, it is sufficient for the road markings to be detected that the reference object is recorded; direct detection of the road markings based on the environmental data is not necessary. Accordingly, the presence and position of the road markings can still be reliably determined even if the road markings are not described by the environmental data or are only partially described, particularly due to obstructions such as dirt, leaves, or snow.

[0023] It may be possible to perform the two described process steps locally using the driver assistance system. When the vehicle first drives over the roadway, the lane markings and the reference object are detected, and the resulting marking data set is stored in the data memory. On subsequent drives over the roadway, the marking data set is used to recognize the lane markings. For example, this forms the basis for assisted driving and / or autonomous driving. In addition to the marking data set stored in the data memory, environmental data can also be used to detect the lane markings, achieving particularly high accuracy.

[0024] The described method can be used particularly profitably for multiple motor vehicles. The invention also relates to a method for operating a driver assistance system arrangement with multiple driver assistance systems for multiple motor vehicles, wherein, by means of an environment detection device, at least one of the motor vehicles, preferably several environment detection devices of several of the motor vehicles, acquires environmental data describing the environment of the respective motor vehicle, at least temporarily. At least one of the driver assistance systems performs the first part of the method, in which the marking data set is stored in the data memory.A second driver assistance device, in particular a second driver assistance device different from the first, serves to implement the second part of the process, in which the marking data set is read from the data storage and used to determine the position of the road marking. The explanations in this description initially refer to the procedure for operating the driver assistance device, but are of course also applicable analogously to the procedure for operating the driver assistance device assembly.

[0025] A further development of the invention provides that the detection of the lane marking and the reference object is carried out using environmental data by a control unit of the driver assistance system or by a central processing unit. This has already been mentioned. For example, the environmental data is evaluated locally on board the vehicle by the driver assistance system or its control unit. However, it can also be provided that the environmental data, in particular the image data acquired by the image sensor, is transmitted to the central processing unit. In this case, the detection of the lane marking and the reference object is not carried out by the driver assistance system, but by the central processing unit.

[0026] When the lane markings and reference object are detected by the central processing unit, it is ensured that personal information, such as faces or vehicle license plates, is anonymized to comply with all applicable data protection regulations. Detection using the control unit offers the advantage that the driver assistance system learns the lane markings along the vehicle's route over time. Processing by the central processing unit, on the other hand, has the advantage of greater computing power, enabling more precise analysis.

[0027] A further development of the invention provides that the environmental detection device includes an image sensor, and the detection of the lane marking and the reference object is carried out using image data acquired by the image sensor. Compared to other sensor data, the image data has the advantage of enabling reliable detection of the lane marking, particularly through classification, for example, using an artificial neural network or the like. It may be possible to evaluate the image data either in the control unit of the driver assistance system or in the central processing unit. This results in the advantages already mentioned.

[0028] A further development of the invention provides that, during the detection of the road marking and the reference object, the position of the road marking and a position of the reference object are determined, and the relative position is calculated from these. Thus, it is not only intended to identify the road marking and the reference object, but also to determine their positions. As already explained, the positions can be either absolute positions or relative positions with respect to the motor vehicle or its position. From these positions, the relative position of the road marking relative to the reference object is subsequently determined. This results in the advantages described above.

[0029] A further development of the invention provides that the environmental sensing device includes a reflection sensor, and the position of the reference object is determined using this reflection sensor. The reflection sensor can be, for example, a radar sensor, a sonar sensor, a lidar sensor, or a combination of several such sensors. Preferably, the lane marking and the reference object are detected using the image data or the image sensor, respectively. Subsequently, the position of the reference object is determined, or at least determined more precisely, using the reflection sensor. For example, it is possible to estimate the position of the reference object based on the image data and then refine the estimated position using the reflection sensor. This approach achieves high accuracy in determining the position of the reference object.

[0030] A further development of the invention provides that the environmental sensing device includes a positioning device, and that the position of the motor vehicle determined by the positioning device is used to determine the position of the lane marking and the position of the reference object. The positioning device is, in particular, an end device of a navigation satellite system, especially a global navigation satellite system, for example, an end device for Navstar GPS, Galileo, or the like. The global position of the motor vehicle, i.e., its geodetic absolute position, is determined by means of the positioning device. This position is subsequently used to determine the positions of the lane marking and the reference object.For example, it is additionally or alternatively planned to use the vehicle's position when reading the marking data record from the data storage, particularly for pre-selecting potential marking data records. This enables a rapid determination of relevant marking data records in the data storage.

[0031] A further development of the invention provides that at least one of the following objects is used as a reference object: traffic sign, signaling system, street lighting system, building, and additional road marking. A traffic sign is understood to be a traffic sign or a symbol applied to the roadway. The signaling system is, in particular, a traffic light system, for example, a traffic light. The street lighting system serves to illuminate the roadway in poor lighting conditions. Additional road marking is a road marking different from the road marking. For example, if the road marking is a lane boundary, a guide line can be used as an additional road marking. It can also be provided that several reference objects are assigned to the road marking in order to achieve particularly precise localization of the road marking.In any case, accurate recording of the road markings is possible even under unfavorable conditions.

[0032] A further development of the invention provides that, in determining the position of the road marking, environmental data is used in addition to the marking data set. Thus, the road marking is not determined solely based on the marking data set, nor is its position determined solely on the basis of the marking data set; rather, environmental data is also taken into account. It may also be possible to first determine the road marking using the environmental detection device or the environmental data, and only supplement it with the marking data set where it is not detected with sufficient accuracy or is not continuous. By using several different data sets, the road marking is detected with high accuracy.

[0033] A further development of the invention provides that the road marking is used, at least temporarily, for the implementation of assisted driving and / or at least partially autonomous driving of the motor vehicle. This, as well as the resulting advantages, has already been mentioned.

[0034] The invention further relates to a driver assistance device for a motor vehicle, in particular for carrying out the method according to the explanations in this description, wherein the driver assistance device is provided and designed to capture environmental data describing the environment of the motor vehicle at least temporarily by means of an environment detection device of the motor vehicle.The driver assistance system is further designed and configured to use environmental data to detect, at least temporarily, a lane marking of a roadway traveled by the motor vehicle and a reference object, and to determine a relative position of the lane marking relative to the reference object, whereby the relative position, together with an object data set describing the reference object, is stored in a data memory as part of a marking data set describing the lane marking; and / or to detect the reference object using environmental data and to read the marking data set containing the object data set describing the reference object from the data memory and to determine a position of the lane marking based on the relative position of the lane marking relative to the reference object.

[0035] The advantages of such a design for the driver assistance system and the procedure have already been mentioned. Both the driver assistance system and the procedure for operating it can be further developed as explained in this description, and reference is made to that description in this respect.

[0036] Furthermore, the invention relates to a computer program product comprising commands that cause the driver assistance system to execute the described method as described herein. For the advantages and possible advantageous further developments, reference is made to the description in its entirety.

[0037] The features and combinations of features described in the description, in particular those described in the following figure description and / or shown in the figures, are not only usable in the combinations specified, but also in other combinations or individually, without departing from the scope of the invention, in particular the scope of the claims. Thus, embodiments that are not explicitly shown or explained in the description and / or the figures, but which emerge from or can be derived from the explained embodiments, particularly within the scope of the claims, are also to be considered as encompassed by the invention.

[0038] The invention is explained in more detail below with reference to the exemplary embodiments shown in the drawing, without limiting the invention. The drawing shows: Fig. 1 a schematic representation of a traffic situation for a motor vehicle, wherein environmental data is recorded by means of an environment detection device, on the basis of which a road marking of the roadway is determined, and Fig. 2. A schematic representation of the traffic situation in abstract form.

[0039] The Fig. Figure 1 shows a schematic representation of a traffic situation 1, which illustrates a procedure for operating a driver assistance system for a motor vehicle. In traffic situation 1, the environment 2 of the motor vehicle is detected by an environment detection device. Image data from an image sensor or camera is shown. The environment 2 includes a roadway 3, which is being traveled on by the motor vehicle and has several lanes 4. Several lane markings 5, 6, and 7 are applied to the roadway 3, with lane markings 5 ​​and 6 being lane boundaries and lane marking 7 being a guide line. Additionally, several reference objects 8 and 9 are present in the environment 2, with reference object 8 being traffic signs and reference object 9 being a building.

[0040] In the first part of the procedure used to operate the driver assistance system, one or more of the lane markings 5, 6, and 7 are detected based on the environmental data. Additionally, at least one of the reference objects 8 and 9 is detected. Subsequently, a relative position of the lane marking 5, 6, or 7 relative to the detected reference object 8 and / or 9 is determined. This relative position is stored in a marking data record that describes the lane marking 5, 6, or 7. Furthermore, an object data record describing the reference object 8 or 9 is assigned to the marking data record, and the marking data record is stored in a data memory.

[0041] The Fig.Figure 2 shows a schematic representation of a virtual representation of the previously described traffic situation, derived from the marking data set. The road markings 5, 6, and 7, as well as the reference objects 8 and 9, can be derived from the marking data set. In a second part of the procedure, one or more of the reference objects 8 and 9 are to be detected using the surrounding data. Subsequently, the marking data set containing the object data set describing the reference object is read from the data storage. Based on the read marking data set, a conclusion is then drawn as to whether the road markings 5, 6, or 7 stored in the marking data set are present.

[0042] Even if one or more of the lane markings 5, 6, and 7 cannot be detected by the environmental sensing device, for example, due to environmental conditions, the position of the lane markings 5, 6, and 7 can be determined based on the positions of the reference objects 8 and 9. For this purpose, the position of the reference object 8 or 9 is first determined, and then the position of the lane marking is determined based on the relative position contained in the marking data set. Based on the lane marking determined in this way, or rather its position, assisted driving and / or at least partially autonomous driving of the vehicle is then carried out. REFERENCE MARK LIST: 1. Traffic situation 2 Environment 3 lanes 4 lanes 5. Road markings 6. Road markings 7 Road markings 8 Reference object 9 Reference object QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 11,232,600 B2

[0002] US 11,248,925 B2

[0004] DE 10 2009 046 699 B4

[0004] US 10,633,027 B2

[0004]

Claims

[1] Method for operating a driver assistance device for a motor vehicle, wherein an environment detection device of the motor vehicle is used to at least temporarily capture environment data describing the environment (2) of the motor vehicle, characterized by , that - based on the environmental data, at least temporarily a lane marking (5, 6, 7) of a lane (3) traveled on by the motor vehicle and a reference object (8, 9) are detected, and a relative position of the lane marking (5, 6, 7) relative to the reference object (8, 9) is determined, wherein the relative position together with an object data set describing the reference object (8, 9) is stored in a data storage as part of a marking data set describing the lane marking (5, 6, 7); and / or that - the reference object (8, 9) is detected using the environment data and the marking data set containing the object data set describing the reference object (8, 9) is read from the data storage and a position of the road marking (5, 6, 7) is determined based on the relative position of the road marking (5, 6, 7) relative to the reference object (8, 9). [2] Method according to claim 1, characterized by , that the detection of the road marking (5, 6, 7) and the reference object (8, 9) is carried out using the environment data by means of a control unit of the driver assistance system or by means of a central computing unit. [3] Method according to any one of the preceding claims, characterized by , that the environment detection device has an image sensor and the detection of the road marking (5, 6, 7) and the reference object (8, 9) is carried out using image data acquired by means of the image sensor. [4] Method according to any one of the preceding claims, characterized by , that when detecting the road marking (5, 6, 7) and the reference object (8, 9), the position of the road marking (5, 6, 7) and a position of the reference object (8, 9) are determined and the relative position is determined from these. [5] Method according to any one of the preceding claims, characterized by , that the environmental sensing device has a reflection sensor and the position of the reference object (8, 9) is determined using the reflection sensor. [6] Method according to any one of the preceding claims, characterized by , that the environment detection device has a position determination device and a position of the motor vehicle determined by means of the position determination device is used in determining the position of the road marking (5, 6, 7) and the position of the reference object (8, 9). [7] Method according to any one of the preceding claims, characterized by , that at least one of the following objects is used as a reference object (8, 9): traffic signs, signaling system, street lighting system, buildings and other road markings. [8] Method according to any one of the preceding claims, characterized by , that the road marking (5, 6, 7) is used at least temporarily for carrying out assisted driving and / or at least partially autonomous driving of the motor vehicle. [9] Driver assistance device for a motor vehicle, in particular for carrying out the method according to one or more of the preceding claims, wherein the driver assistance device is designed and configured to at least temporarily capture environmental data describing an environment (2) of the motor vehicle by means of an environment detection device of the motor vehicle, characterized bythat the driver assistance system is further designed and configured to - to detect, at least temporarily, a lane marking (5, 6, 7) of a lane (3) traveled on by the motor vehicle, as well as a reference object (8, 9), and to determine a relative position of the lane marking (5, 6, 7) relative to the reference object (8, 9), whereby the relative position, together with an object data set describing the reference object (8, 9), is stored in a data storage device as part of a marking data set describing the lane marking (5, 6, 7); and / or - to use the environment data to detect the reference object (8, 9) and to read the marking data set containing the object data set describing the reference object (8, 9) from the data storage and to determine a position of the road marking (5, 6, 7) relative to the reference object (8, 9) based on the relative position of the road marking (5, 6, 7). [10] Computer program product comprising commands that cause the driver assistance device according to claim 9 to execute the method according to one or more of claims 1 to 8.

Images