Assistance system for supporting a driver of a vehicle during a lateral guidance of the vehicle while taking into account a dazzling of the driver, vehicle, and method

Abstract

The invention relates to an assistance system (2) for supporting a driver of a vehicle (1) during a lateral guidance of the vehicle (1). The assistance system (2) is designed to: carry out an automatic lateral guidance of the vehicle (1) on a lane, provide a hands-off operation, in which the driver can permanently remove their hands from the steering wheel of the vehicle (1), during the automatic lateral guidance, receive fatigue data, said fatigue data describing the eyelids of the driver, determine the degree of fatigue of the driver using the fatigue data, deactivate the hands-off operation if the degree of fatigue exceeds a specified fatigue threshold, estimate a current dazzling of the driver, and additionally deactivate the hands-off operation on the basis of the estimated dazzling.

Description

[0001] Assistance system for supporting a driver of a vehicle during lateral guidance of the vehicle taking into account glare of the driver, vehicle and method

[0002] The present invention relates to an assistance system for assisting a driver of a vehicle during lateral guidance of the vehicle. Furthermore, the present invention relates to a vehicle having such an assistance system. Furthermore, the present invention relates to a method for assisting a driver of a vehicle during lateral guidance of the vehicle.

[0003] Lane guidance assistants for vehicles are well known in the art. Such lane guidance assistants, also known as steering and lane guidance assistants or active lane keeping assistants, serve primarily to keep the vehicle within the lane or its own lane. Such a lane guidance assistant uses the surrounding data from at least one of the vehicle's surrounding sensor to detect the lane boundaries or the road markings. The lane guidance assistant can perform automated steering interventions to keep the vehicle within the lane.

[0004] In addition, assistance systems that enable so-called hands-off operation are known from the state of the art. Such assistance systems allow the driver to permanently remove their hands from the steering wheel during lateral guidance. To maintain this function, the driver must be attentive, observe the traffic situation, and be ready to intervene at any time. Driver attention is monitored using a driver monitoring device or an interior camera. In the event of an error in the automated lateral guidance, it is assumed that the driver can intervene immediately and steer the vehicle back into the lane.

[0005] In addition, such hands-off systems monitor driver fatigue. This determines fatigue based on the movement and / or position of the driver's eyelids. For example, the degree to which the driver's eyes are open or closed is determined. Driver fatigue can therefore be determined from the degree of eye opening. If it is detected that the driver is fatigued, the hands-off mode is immediately deactivated, and the function is only available to the driver again when the driver is recognized as awake. Only then is the function offered to the driver again, and the driver can use it again.

[0006] When driving into the sun when the sun is low in the sky, the driver is blinded by the sun. This causes the driver to close their eyes or squint. When evaluating driver fatigue, it can then be concluded that the driver has closed their eyes and is tired. This often leads to unjustified detections of driver fatigue in such cases. The hands-off mode is then deactivated and the driver can only use it again once they are recognized as awake. However, this can take some time when the sun is low in the sky. This means that the function is not available to the driver for an extended period of time as long as they are driving into the low sun.

[0007] The object of the present invention is to show a solution how a driver can be supported more reliably in the automated lateral guidance of the vehicle.

[0008] This object is achieved according to the invention by an assistance system, a vehicle, and a method having the features according to the independent claims. Advantageous developments of the present invention are specified in the dependent claims.

[0009] An assistance system according to the invention serves to assist a driver of a vehicle during lateral guidance of the vehicle. The assistance system is configured to perform automated lateral guidance of the vehicle on a roadway. Furthermore, the assistance system is configured to provide a hands-off mode during automated lateral guidance, in which the driver can permanently remove their hands from the steering wheel of the vehicle. Furthermore, the assistance system is configured to receive fatigue data, wherein the fatigue data describes the driver's eyelids. Furthermore, the assistance system is configured to determine a degree of fatigue of the driver based on the fatigue data. Furthermore, the assistance system is configured to deactivate the hands-off mode if the degree of fatigue exceeds a predetermined fatigue threshold.In addition, the assistance system is configured to estimate the current glare of the driver and to deactivate the hands-off mode based on the estimated glare. The assistance system is intended to support the driver in lateral guidance of the vehicle. In particular, the assistance system is intended to provide the functionality of a lane guidance assistant or the functionality of automated lane guidance. The driver can be supported in steering tasks. In particular, the assistance system can intervene in the steering in such a way that the vehicle is kept within a lane of the roadway or the current lane.

[0010] The assistance system can have a computing device, which can be formed by at least one processor, by at least one electronic control unit or the like. The computing device of the assistance system can receive the environmental data from at least one environmental sensor of the assistance system. This environmental data can describe the surroundings of the vehicle or an environment of the vehicle. The environmental sensor can be a camera, for example. In this case, the environmental data can be image data. The respective lanes of the road on which the vehicle is currently located can be recognized in this environmental data. In addition, lane markings of the respective lanes can be recognized on the basis of the environmental data. The lane markings can in particular be road markings, for example solid, dashed or broken lines.Lane markings can also be structural barriers, such as a wall, a guardrail or the like.

[0011] The computing device or the assistance system can also receive satellite-based positioning data, which can be provided, for example, with a corresponding receiver for a satellite-based positioning system. The receiver can, for example, receive the positioning data from a global navigation satellite system. In particular, a so-called differential global positioning system is used, which can increase the accuracy of positioning by transmitting correction data.

[0012] Furthermore, the computing device or the assistance system can receive digital map data describing the vehicle's surroundings. In particular, high-resolution map data or HD maps can be used as the digital map data to determine the vehicle's current position. This high-resolution map data can be stored in a memory or storage device of the assistance system and / or received from an external device.

[0013] It is also intended that hands-off operation will be enabled during automated lateral guidance of the vehicle using the assistance system. In this hands-off operation, the driver can permanently remove their hands from the steering wheel during automated lateral guidance. In particular, it is intended that driver fatigue will be monitored to maintain hands-off operation. For this purpose, fatigue data from a driver monitoring device or an interior camera can be used. This fatigue data describes the driver's eyelids. In particular, the fatigue data can describe a movement and / or position of the driver's eyelids. The fatigue data can also describe the degree of eye or eyelid opening of the driver.

[0014] Based on the fatigue data, the driver's degree of fatigue can be determined. This makes it possible to determine whether the driver is awake and able to monitor the traffic and / or the hands-off operation, or whether the driver is tired. If the degree of fatigue exceeds the predetermined fatigue threshold, it can be assumed that the driver is tired and cannot adequately monitor the traffic and the hands-off operation. The fatigue threshold can be defined in advance and can, for example, describe the degree of eye opening, a maximum eye closure time, or the like. If the degree of fatigue exceeds the predetermined fatigue threshold, the hands-off operation can be deactivated. The driver can be prompted to place their hands back on the steering wheel.It may also be provided that activation of the hands-off mode is only possible again if it is detected that the degree of fatigue falls below the fatigue threshold.

[0015] It can also be provided that the driver's attention is monitored during hands-off operation. For this purpose, data from the driver monitoring device or the interior camera can be used, which describes the driver's current line of sight and / or head position. Based on the data from the driver monitoring device, it can be checked whether the driver is attentive or is directing their attention to the traffic situation. If this is the case, hands-off operation can generally be maintained. As explained in more detail below, the assistance system or the computing device can also be used to estimate the driver's current glare. It can therefore be determined whether and to what extent the driver is dazzled by external influences. For example, the driver can be dazzled by the sun and / or by a light source in the vehicle's vicinity.

[0016] According to the present invention, the current glare of the driver is taken into account when deactivating the hands-off mode due to driver fatigue. If the driver is currently dazzled, this can in particular result in the driver at least partially closing their eyes and even squinting. This can lead to the erroneous assumption that the driver's degree of fatigue exceeds the fatigue threshold. In this case, the hands-off mode would be deactivated immediately, even though the driver is not actually tired but merely dazzled. If the estimated driver glare is taken into account when deactivating the hands-off function due to driver fatigue, the function can be prevented from being deactivated incorrectly or prematurely. Overall, the driver support for lateral control of the vehicle can thus be improved.

[0017] In one embodiment, the assistance system is configured to adapt a time period between the detection of exceeding the fatigue threshold and the deactivation depending on the estimated glare. It can be provided that the hands-off mode is deactivated immediately if it is detected that the degree of fatigue exceeds the fatigue threshold and if it is detected that the driver is not currently being dazzled. However, if it is detected that the driver is currently being dazzled, the time period or a so-called debounce time between the detection of exceeding the fatigue threshold and the deactivation of the hands-off mode can be adapted. The time period can be between a few seconds and a few minutes, for example. The time period can be increased as glare increases. In this way, the time of deactivation can be adapted to the driver's current level of glare.

[0018] In a further embodiment, the assistance system is configured to determine the degree of fatigue based on the estimated glare. The estimated glare of the driver can therefore also be directly incorporated into the determination of the driver's fatigue or the degree of fatigue. For example, the driver's eyelid position or the degree to which the eyes are closed can be estimated based on the glare. This can be done individually for the driver. An appropriate neural network and / or a machine learning method can be used for this purpose. Thus, the actual fatigue of the driver can be precisely determined taking into account the driver's current glare. In this way, the hands-off function can be prevented from being deactivated. It can also be ensured that the function can be activated by the driver.This would otherwise not be possible if the driver is assumed to be tired.

[0019] It may be provided that a calibration is performed when determining driver fatigue or the degree of fatigue. For example, the detection of fatigue can be calibrated at the start of a journey. This calibration can take into account the driver's current glare.

[0020] In a further embodiment, the assistance system is configured to determine an orientation of the vehicle to the sun and / or a current position of the sun and to estimate the glare based on the orientation and / or the position of the sun. As already explained, the assistance system can have a receiver for a global navigation satellite system, in particular for a differential global positioning system. Based on the position data, the current direction of travel of the vehicle and the current position of the vehicle or the geoposition of the vehicle can be determined. Based on the current date and time, the angle between the vehicle and the sun can then be determined. Furthermore, the position of the sun can be determined based on the date, time and geoposition of the vehicle. The position of the sun can describe the angle of the sun above the horizon.Based on the orientation of the vehicle to the sun and the position of the sun, the driver's current glare can then be estimated.

[0021] Current weather data can also be used to estimate driver glare. This weather data can, for example, describe whether there is cloud cover or something similar that could influence the current glare. Furthermore, based on the digital map data, it can be determined whether there is any shadowing from tall buildings or the like. It can also be provided that data from an optical sensor is used to determine or verify the plausibility of the driver's current glare. It can also be estimated whether the driver is currently being dazzled by light sources in the vehicle's surroundings. Such light sources can be light sources from buildings and / or headlights of other road users. Data from optical sensors, the vehicle's environmental sensors, sensors from an adaptive headlight system, or the like can be used for this purpose.

[0022] A vehicle according to the invention comprises an assistance system according to the invention. The vehicle is designed, in particular, as a passenger car.

[0023] A method according to the invention serves to assist a driver of a vehicle during lateral guidance of the vehicle. The method comprises performing automated lateral guidance of the vehicle on a roadway. Furthermore, the method comprises providing a hands-off mode, in which the driver can permanently remove their hands from a steering wheel of the vehicle. Furthermore, the method comprises receiving fatigue data, wherein the fatigue data describes the driver's eyelids, during automated lateral guidance. Furthermore, the method comprises determining a degree of fatigue of the driver based on the fatigue data. Furthermore, the method comprises deactivating the hands-off mode if the degree of fatigue exceeds a predetermined fatigue threshold.In addition, the method includes estimating the current glare of the driver and deactivating the hands-off mode additionally depending on the estimated glare.

[0024] A further aspect of the invention relates to a computer program comprising instructions which, when executed by a computing device of the assistance system, cause the computing device to execute a method according to the invention and the advantageous embodiments thereof. Furthermore, the invention relates to a computer-readable storage medium comprising instructions which, when executed by the computing device of the assistance system, cause the computing device to execute a method according to the invention and the advantageous embodiments thereof.

[0025] The preferred embodiments presented with reference to the assistance system according to the invention and their advantages apply accordingly to the vehicle according to the invention, to the method according to the invention, to the computer program according to the invention and to the computer-readable storage medium according to the invention. Further features of the invention emerge from the claims, the figures and the description of the figures. The features and combinations of features mentioned above in the description, as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the figures can be used not only in the respective combination specified, but also in other combinations or on their own, without departing from the scope of the invention.

[0026] The invention will now be explained in more detail using preferred embodiments and with reference to the accompanying drawings.

[0027] Fig. 1 is a schematic representation of a vehicle having an assistance system for assisting a driver of the vehicle during lateral guidance of the vehicle;

[0028] Fig. 2 is a schematic flow diagram of a method for assisting a driver in transverse guidance of the vehicle, taking into account dazzling of the driver according to a first variant; and

[0029] Fig. 3 is a schematic flow diagram of a method for assisting a driver in transverse guidance of the vehicle taking into account dazzling of the driver according to a second variant.

[0030] Fig. 1 shows a top view of a vehicle 1, which in this case is designed as a passenger car. The vehicle 1 comprises an assistance system 2, which is configured to assist a user or driver of the vehicle 1 in lateral guidance of the vehicle 1. In particular, the assistance system 2 serves to provide automated lateral guidance of the vehicle 1. Preferably, the assistance system 2 can assist the driver in lateral guidance of the vehicle 1 in such a way that the driver can permanently remove their hands from the steering wheel.

[0031] The assistance system 2 comprises a computing device 3, which can be formed, for example, by at least one electronic control unit of the vehicle 1. In addition, the assistance system 2 comprises at least one environmental sensor 4, which in this case is designed as a camera. This environmental sensor 4 can provide environmental data or image data that describe an environment 5 of the vehicle 1. In particular, the environmental data or image data can describe lane boundaries, in particular lane markings of a roadway. The environmental data can be transmitted from the environmental sensor 4 or the camera to the computing device 3. It can also be provided that the assistance system 2 has further environmental sensors, for example radar sensors, lidar sensors, or the like.

[0032] Furthermore, the assistance system 2 comprises a receiver 7 for a satellite-based positioning system. Satellite-based positioning data describing the position of the vehicle 1 can be received by the receiver 7. Furthermore, the assistance system 2 comprises a storage device 8 on which high-resolution map data, or so-called HD maps, are stored. Furthermore, the assistance system 2 comprises an output device 9, by means of which an output can be provided to the driver of the vehicle 1. This output can generally be provided visually, acoustically, and / or haptically.

[0033] In addition, the assistance system 2 comprises a driver monitoring device 6, by means of which the driver's attention can be monitored. In particular, the driver monitoring device 6 can determine data describing a direction of gaze and / or the orientation of the driver's head. Furthermore, fatigue data can be provided by the driver monitoring device 6 and transmitted to the computing device 3. The fatigue data describe an eyelid position or a degree of eye opening of the driver. Based on the fatigue data, the computing device 3 can determine whether the driver is fatigued. In particular, it can be checked whether the degree of fatigue exceeds a predetermined fatigue threshold.

[0034] Furthermore, the computing device 3 is configured to control a steering system 10 of the vehicle 1, shown only schematically here. By controlling the steering system 10, the driver of the vehicle 1 can be assisted with lateral guidance. In this case, steerable wheels 11 of the vehicle 1 can be moved by controlling the steering system 10. Preferably, it is also provided that a drive motor and / or a braking system of the vehicle 1 can also be controlled by the computing device 3 in order to also intervene in the longitudinal guidance of the vehicle 1.

[0035] It is also provided that during the automated lateral guidance of the vehicle 1, a hands-off mode is enabled, allowing the driver to permanently remove their hands from the steering wheel. During hands-off mode, the driver's attention and fatigue are monitored, particularly based on data from the driver monitoring device 6. If it is detected that the driver is tired, the hands-off mode is deactivated.

[0036] Fig. 2 shows a schematic flow diagram of a method for assisting a driver during lateral guidance of the vehicle 1 according to a first variant. In a step S1, the current orientation of the vehicle 1 is determined based on the position data received by the receiver. In a step S2, the current position or geoposition of the vehicle 1 is determined based on the position data. In a step S3, the current date and time are determined. For this purpose, data are received from a clock 12 of the vehicle 1.

[0037] In a step S4, an angle between the vehicle 1 and the sun is determined based on the orientation of the vehicle 1, the geoposition of the vehicle 1, and the date and time. Furthermore, in a step S5, the position of the sun or the angle of the sun above the horizon is determined based on the geoposition of the vehicle 1 and the date and time. To determine the angle between the vehicle 1 and the sun, as well as to determine the position of the sun, appropriate data or calculation rules can be used, which are stored, for example, on the storage device 8.

[0038] In a step S6, the driver's degree of fatigue is then determined based on the fatigue data from the driver monitoring device 6. If it is detected that the degree of fatigue exceeds the predetermined fatigue threshold, the hands-off mode is deactivated in a step S7.

[0039] In step S8, a time period between the detection of driver fatigue or the exceeding of the fatigue threshold and the deactivation of hands-off mode is determined. This time period or debounce time is determined as a function of the relative position of vehicle 1 to the sun and the position of the sun. If, for example, it is detected that vehicle 1 is moving directly towards the sun at an angle of 0° and the sun is at an angle of less than 20° above the horizon, then fatigue is debounced for, for example, 2 minutes before the function is deactivated. If, on the other hand, it is detected that vehicle 1 is moving towards the sun at an angle of 45° and the sun is at an angle of less than 20° above the horizon, then fatigue is debounced for, for example, 1 minute before the function is deactivated.If, on the other hand, it is detected that vehicle 1 is moving away from the sun at an angle of 180°, then fatigue is immediately recognized and the hands-off mode is deactivated. In this case, the angle of the sun above the horizon is irrelevant because vehicle 1 is moving in exactly the opposite direction. Fig. 3 shows a schematic flow diagram of a method for assisting a driver during lateral guidance of vehicle 1 according to a second variant. In comparison to the method according to Fig. 2, the angle between vehicle 1 and the sun as well as the angle of the sun above the horizon are directly included in the calculation of fatigue or the degree of fatigue according to step S6' of the method. A neural network, for example, can be used for this purpose.

Claims

Claims 1. Assistance system (2) for supporting a driver of a vehicle (1) during lateral guidance of the vehicle (1), wherein the assistance system (2) is designed to: - to carry out automated lateral guidance of the vehicle (1) on a roadway, - to provide a hands-off mode in automated lateral guidance, in which the driver can permanently remove his hands from the steering wheel of the vehicle (1), - to receive fatigue data, where the fatigue data describes the driver’s eyelids, - to determine the degree of driver fatigue based on the fatigue data, and - to deactivate the hands-off mode if the degree of fatigue exceeds a predetermined fatigue threshold, characterized in that Assistance system (2) is further configured to - to estimate the current glare of the driver and - to deactivate the hands-off mode additionally depending on the estimated glare.

2. Assistance system (2) according to claim 1, characterized in that the assistance system (2) is designed to adapt a time period between the detection of the exceeding of the fatigue limit and the deactivation depending on the estimated glare.

3. Assistance system (2) according to claim 1 or 2, characterized in that the assistance system (2) is designed to determine the degree of fatigue as a function of the estimated glare.

4. Assistance system (2) according to one of the preceding claims, characterized in that the assistance system (2) is designed to orient the vehicle (1) the sun and / or the current position of the sun and to estimate the glare based on the orientation and / or position of the sun.

5. Vehicle (1), in particular a passenger car, comprising an assistance system (2) according to one of the preceding claims.

6. A method for assisting a driver of a vehicle (1) during transverse guidance of the vehicle (1), comprising the steps: - carrying out an automated lateral guidance of the vehicle (1) on a roadway, - Providing a hands-off operation in which the driver can permanently remove his hands from a steering wheel of the vehicle (1), - Receiving fatigue data, where the fatigue data describes the driver's eyelids, during automated lateral guidance, - Determining a degree of driver fatigue based on the fatigue data and - Deactivation of the hands-off mode if the degree of fatigue exceeds a predetermined fatigue threshold, characterized by - Estimate the current glare of the driver and - Deactivation of the hands-off mode additionally depending on the estimated glare.

Images