Method for operating an assistance system for a vehicle and corresponding assistance system

Abstract

Method for operating an assistance system for a vehicle (1), comprising an environment sensor for determining road data representative of a road course (2) lying in the direction of travel of the vehicle (1), and a sensor system for determining vehicle data representative of a trajectory (3) of the vehicle (1) in the direction of travel, wherein the vehicle data includes at least one of the following: steering angle, yaw rate, wheel angle or wheel speeds, and in the method - it is checked whether road data is available that meets a specified accuracy standard, - if road data is available that meets the specified accuracy, an attention area (5) is determined based on the road data which is representative of a desired gaze direction of the driver, and - otherwise the attention area (5) is determined depending on the vehicle data.

Description

[0001] The invention relates to a method for operating an assistance system for a vehicle as well as a corresponding assistance system, computer program and computer program product.

[0002] Modern driver assistance systems for monitoring a driver's attention can, for example, include a camera sensor that detects the driver's head pose, gaze direction, and / or position, perhaps based on facial features. The assessment of the driver's attention can then be based on a detected head movement within a fixed vehicle model. Generally, a driver is considered attentive if their head is looking out of the vehicle's interior. This can correspond to a head orientation towards an outer, central area of ​​interest (AOI). However, a problem arises from the fact that, due to the high dynamism of road traffic, the driver's attention is often misjudged.

[0003] DE 10 2016 208 405 A1 relates to a method for operating a driver assistance system for a motor vehicle, wherein a driver's gaze direction is monitored and compared with at least one target value, wherein the at least one target value is determined depending on the course of a journey, and an attention value for the driver is determined depending on the comparison.

[0004] The object underlying the invention is to create a method for operating an assistance system for a vehicle and a corresponding assistance system that contributes to a precise detection of the driver's attention.

[0005] The problem is solved by the independent patent claims. Advantageous embodiments are characterized in the dependent claims.

[0006] According to a first aspect, the invention relates to a method for operating a vehicle assistance system. The assistance system comprises an environmental sensor for determining road data representative of a road course in the direction of travel of the vehicle and a sensor system for determining vehicle data representative of a trajectory of the vehicle in the direction of travel. The vehicle data consists of at least one of the following: steering angle, yaw rate, wheel angle, or wheel speeds, which can be detected by appropriate sensors.

[0007] The procedure checks whether road data is available that meets a specified accuracy standard.

[0008] If road data is available that meets the specified accuracy, an attention area is determined based on the road data, which is representative of a desired viewing direction of the driver; otherwise, the attention area is determined based on the vehicle data.

[0009] This advantageously allows the assistance system to adapt to the dynamics of road traffic. In particular, it contributes to a more precise detection of the driver's attention. Improved attention detection during cornering can increase both driving comfort and safety, as only those who are following or anticipating the road's course with their minds are recognized as attentive. Specifically, this prevents certain additional assistance functions from deactivating when the driver is correctly identified as attentive. For example, from a legal perspective, it may be necessary for the continued operation of autonomous systems that the driver is recognized as attentive. An adaptation like the one described above thus increases the duration of fully autonomous journeys and enables the correct adjustment of other support systems, even on winding stretches of road.

[0010] The environmental sensor is, for example, a camera that detects road boundaries or lane markings. The environmental sensor may also be equipped with a radar or lidar unit, either alternatively or additionally. Specifically, the environmental sensor is designed to detect the curvature of the road in relation to the current distance from the vehicle.

[0011] Particularly in the case of sensor malfunctions, the environmental sensor may fail to collect any road data or obtain insufficiently accurate data. In this case, the vehicle data serves as a fallback for determining the area of ​​attention.

[0012] The attention zone refers specifically to a circular or elliptical cone whose apex is located at the driver's head position and whose base defines a field of vision within which the driver's intended gaze direction lies. Depending on the determined road curvature, the cone can be horizontally pivoted so that a dynamic road profile is taken into account within the attention zone.

[0013] In an advantageous embodiment according to the first aspect, the assistance system also includes a navigation device for providing road data.

[0014] The procedure first checks whether the road data provided by the navigation device meets the specified accuracy requirements.

[0015] If road data provided by the navigation device is available that meets the specified accuracy, the attention zone is determined based on the road data provided by the navigation device; otherwise, it is checked whether road data provided by the environment sensor is available that meets the specified accuracy.

[0016] If road data provided by the environmental sensor is sufficient for the specified accuracy, the attention zone is determined based on the road data; otherwise, the attention zone is determined based on the vehicle data.

[0017] This advantageously enables a precise and computationally efficient estimation of the driver's attention. In particular, it creates an additional fallback level that increases the reliability of the assistance system.

[0018] The road data can be stored in the navigation system or provided via the cloud. Specifically, the navigation system includes a positioning unit that enables the provision of road data, for example, segment by segment, based on the vehicle's current position and, optionally, its direction of travel and / or speed. The road data can include, in particular, information about road curvature relative to the current distance to the vehicle.

[0019] In a further advantageous embodiment according to the first aspect, the assistance system also includes a gaze sensor device for determining a direction parameter that is representative of the driver's actual gaze direction.

[0020] In this procedure, an attention value is determined based on the direction parameter and the attention range, which is representative of the estimated attention of the driver.

[0021] The gaze sensor device may, in particular, include a camera positioned facing the driver and configured to detect the driver's head pose, position, or movement, and / or actual direction of gaze. Anatomical features such as the position of the nose (tips), eyes, and / or pupils, the angle of the eyes, and / or the corners of the mouth, as well as the relationships between these features, may be used to determine the head pose, direction of gaze, and / or head position. The sensor unit may also include an optional lighting device.

[0022] For example, a driver is considered attentive if their head or gaze is directed towards the determined area of ​​attention. The determined directional indicator can be used, for instance, to monitor the driver's attention and issue warnings if necessary.

[0023] In a further advantageous embodiment according to the first aspect, a warning or assistance function is controlled depending on the attention indicator.

[0024] In a further advantageous embodiment according to the first aspect, the direction parameter is representative of a driver's head pose.

[0025] According to a second aspect, the invention relates to a vehicle assistance system. The assistance system comprises an environmental sensor for determining road data representative of the road's course in the vehicle's direction of travel. Furthermore, the assistance system comprises a sensor system for determining vehicle data representative of the vehicle's trajectory in the direction of travel. In addition, the assistance system comprises a control unit configured to carry out the method according to the first aspect.

[0026] In an advantageous embodiment according to the second aspect, the assistance system also includes a navigation device for providing road data.

[0027] In a further advantageous embodiment according to the second aspect, the assistance system also includes a gaze sensor device for determining a direction parameter that is representative of the actual gaze direction of a driver of the vehicle.

[0028] According to a third aspect, the invention relates to a computer program for operating an assistance system. The computer program is configured to perform a method according to the first aspect when executed on a data processing device.

[0029] According to a fourth aspect, the invention relates to a computer program product comprising executable program code. When executed by a data processing device, the program code performs the method according to the first aspect.

[0030] Exemplary embodiments of the invention are explained in more detail below with reference to the schematic drawings.

[0031] They show: Fig. 1 an exemplary flowchart of a method for operating an assistance system according to the invention; Fig. 2a, Fig. 2b a first embodiment for determining an attention area; Fig. 3a, Fig. 3b a second embodiment for determining an attention area; and Fig. 4a, Fig. 4b a third embodiment for determining an attention area.

[0032] Elements of the same construction or function are provided with the same reference symbols across all figures.

[0033] The (in)attention of a motor vehicle driver can be assessed based on detected head movements within a fixed vehicle model. The head movement or position is detected by a sensor system, such as a camera, inside the vehicle. Generally, the driver is considered attentive if their head is facing outwards from the vehicle interior. This corresponds to a head orientation towards an outer, central area of ​​interest (AOI).

[0034] On a straight road, the driver's level of attention can be determined by assuming that the driving context observed by the driver – especially the road itself – is generally directly in front of them. However, this is not the case when driving through a curve or on a generally winding road. In such situations, the driver's level of attention is not adequately assessed.

[0035] The following proposes an assistance system for a vehicle 1 and a method for operating it, which contributes to a precise determination of the driver's attention, especially in a dynamic driving context.

[0036] The assistance system includes a navigation device for providing road data from a local data storage device, which is representative of a road course 2 lying in the direction of travel of the vehicle 1. A location unit assigned to the navigation device determines a current position, direction of travel and speed of the vehicle, so that the corresponding required road data can be provided by the navigation device section by section.

[0037] Furthermore, the assistance system includes a camera as an environmental sensor for determining road data. Specifically, the environmental sensor is designed to detect the road curvature in relation to the current distance to vehicle 1. The road data determined by the environmental sensor may differ from the data provided by the navigation system. Such a difference can occur, in particular, if the vehicle's position cannot be determined with sufficient accuracy, or if vehicle 1 is traveling through unfamiliar terrain for which no road data is available. Limitations may also arise if the environmental sensor cannot determine any road data, for example, due to limited visibility or obscured or ambiguous lane markings.

[0038] The assistance system further comprises a yaw rate sensor as a sensor system for determining the trajectory 3 of the vehicle 1 in the direction of travel as vehicle data, a camera for determining the driver's current viewing direction as a directional parameter, and a control unit with a data and program memory in which a program for operating the assistance system is stored, which is based on the flowchart of the Fig. 1 is explained in more detail below.

[0039] The program starts in step S1, in which, for example, variables are initialized. Furthermore, at least the navigation system is activated to provide road data for the next upcoming section of road 2. The program then continues in step S3.

[0040] In step S3, the program checks whether the navigation system was able to provide road data. If not, the program continues in step S5. Otherwise, the data provided by the navigation system is checked for accuracy. This check can take into account factors such as the reception quality of GPS signals, which allows conclusions to be drawn about how accurately vehicle 1 could be located. If the road data provided by the navigation system meets a predefined accuracy requirement, the program continues in step S11a. Otherwise, the program continues in step S5. For example, the road data meets the predefined accuracy requirement if vehicle 1 could be located with a deviation of less than 1 meter.

[0041] In step S5, the environmental sensor is activated to determine the road data for the next preceding section of road 2. Alternatively, the environmental sensor may have already collected such data, which is then simply read out in step S5. For example, a central environmental model is used, which has accumulated and processed environmental sensor data. The program then continues in step S7.

[0042] In step S7, the program checks whether road data could be determined by the environmental sensor. If not, for example due to sensor errors, the program continues in step S9. Otherwise, the data determined by the environmental sensor is checked for accuracy. If the road data determined by the environmental sensor meets a predefined accuracy standard, the program continues in step S11a. Otherwise, the program continues in step S9. Camera data can be verified on several levels: on the image sensor itself, on the associated control unit, and finally, within the environmental model. When querying data that depends on these sensors, specific characteristics can be checked that provide information about the quality of the resulting data (e.g., the road model).

[0043] In step S9, the sensor system is controlled to determine the vehicle data. Depending on the vehicle data, a trajectory 3 of the vehicle 1 in the direction of travel can be calculated in order to estimate the next upcoming section of the road 2. The program then continues in step S11b.

[0044] In step S11a, an attention area 5 is determined based on the road data, which is representative of the driver's intended gaze direction. Here, the attention area 5, e.g., the middle outer AOI, is moved along with the actual road course 2. For example, an angle α is determined by which the attention area 5 is swiveled relative to a "straight ahead" direction of travel of the vehicle (e.g., with a wheel steering angle of 0°) (see...). Fig. 2a, Fig. 2b, Fig. 3a, Fig. 3b). The program then continues in step S13.

[0045] In step S11b, the attention zone 5 is determined based on the vehicle data. Here, the road course 2 is estimated based on the determined trajectory 3 of the vehicle, and the attention zone 5 is adjusted accordingly. As an example, the angle α is again determined by which the attention zone 5 is swiveled relative to the vehicle's "straight ahead" direction of travel (see...). Fig. 4a, Fig. 4b). The program then continues at step S13.

[0046] In step S13, the gaze sensor is activated to record the driver's current gaze direction as a directional parameter. Based on this directional parameter and attention range 5, an attention parameter is then calculated, which is representative of the driver's estimated attention level. Depending on this attention parameter, a warning or assistance function can then be activated. The program then terminates.

[0047] In summary, the route of road 2 is determined or estimated as follows: - Accurate map information is used where available. If these are not available, an ego track determined by the environment model software is used. If this is also unavailable, the vehicle's trajectory is estimated based on its current ego curvature and used as the assumed road course.

[0048] In the Fig. 2a, Fig. Figure 2b shows a first embodiment for determining the attention zone 5 depending on the road data. The vehicle 1 initially drives on a straight section of road 10 ( Fig. 2a) with a lane marking 11, which separates the lane in which the vehicle 1 is traveling from an adjacent lane. At a predetermined distance d in the direction of travel of the vehicle, for example 50 m, the road segment 10 is still straight. The attention zone 5, which indicates the intended direction of the driver's gaze, is therefore chosen such that its horizontal center lies on a point (marked by a circle with x) located on the center of the lane 12 at the predetermined distance. Depending on the speed, the attention zone 5 can be chosen to be narrower (see hatched area) or wider (unhatched circular segment surrounding the hatched area).

[0049] If vehicle 1 now enters a curved section of road 10 ( Fig. 2b), the attention area 5 is pivoted by the angle α in this direction. The new horizontal center of the attention area 5 is chosen to be an intersection point (marked by a circle with x) of the lane center with a tangent 4 of the circle, which is perpendicular to the original, straight direction of travel.

[0050] In the Fig. 3a, Fig. Figure 3b shows a second embodiment for determining the attention zone 5 depending on the road data. The vehicle 1 again initially drives on a straight section of road 10 ( Fig. 3a).

[0051] If vehicle 1 now enters a curved section of road 10 ( Fig. 3b), the attention area 5 is swivelled by the angle α in this direction. However, in contrast to the first embodiment, the angle α is determined depending on a tangent 4 of the road course 2 at the predetermined distance d.

[0052] For example, depending on the driver's observed behavior, particularly based on their head and / or eye movements, a choice can be made between determining the attention area 5 according to the first and second embodiments. Thus, if the head movement is low or the eye movement is high, the attention area 5 is determined according to the first embodiment, while if the head movement is high or the eye movement is low, the attention area 5 is determined according to the second embodiment.

[0053] In the Fig. 4a, Fig. Figure 4b shows a third embodiment for determining the attention area 5, wherein, in contrast to the first and second embodiments, the attention area 5 is determined depending on the vehicle data. The vehicle 1 initially travels straight ahead; its projected trajectory 3 is therefore straight ( Fig. 4a) The attention area 5 is therefore chosen such that its horizontal center lies on a point (marked by a circle with x) located on trajectory 3 at the specified distance d. Depending on the speed, the attention area 5 can again be chosen to be narrower (see hatched area) or wider (unhatched circular segment surrounding the hatched area).

[0054] If vehicle 1 now steers, its projected trajectory 3 is also curved ( Fig.4b). The attention area 5 is pivoted by the angle α in this direction. The new horizontal center of the attention area 5 is chosen as the intersection point (marked by a circle with x) of the tangent 4 of the circle, which is perpendicular to the original, straight direction of travel, with the projected trajectory 3 of the vehicle 1. The trajectory 3 can be described by a circular segment of radius r, where the radius r corresponds to the reciprocal of the yaw rate.

[0055] In all embodiments, the positioning of the outer AOI boundaries (each represented as small circles) can be adapted to the road's curvature 2 to reflect the driver's natural visual behavior when cornering. The boundaries, measured at the specified distance d, can be rotated by the angle α as shown.

[0056] In summary, the process reads in a curve profile or the vehicle's curvature; the angle α is calculated, for example, depending on the vehicle's speed at a predefined distance d from the driver's standard position. The angle α can be calculated from the deviation of the vehicle's direction from a straight line through the intersection point of a circle of radius d whose center lies on the ego lane. This calculation is performed in each cycle. The angle α is limited to a range between, for example, -45° and +45°. A positive value corresponds to a counterclockwise rotation. The outer AOI boundaries are rotated by the angle α, centered on the driver's standard position. The newly calculated AOI boundaries are then used in the subsequent normalization. Reference symbol list: 1 vehicle 2. Road route 3 Trajectory 4 Tangent 5 Attention area 10 road section 11 Road marking 12 Lane Center d specified distance r radius S1... S13 Program steps

Claims

Method for operating an assistance system for a vehicle (1), comprising an environment sensor for determining road data representative of a road course (2) lying in the direction of travel of the vehicle (1), and a sensor system for determining vehicle data representative of a trajectory (3) of the vehicle (1) in the direction of travel, wherein the vehicle data includes at least one of the following: steering angle, yaw rate, wheel angle or wheel speeds, and in the method: - it is checked whether road data is available that meets a specified accuracy, - if road data is available that meets the specified accuracy, an attention area (5) is determined depending on the road data which is representative of a desired gaze direction of the driver, and - otherwise the attention area (5) is determined depending on the vehicle data. The method according to claim 1, further comprising a navigation device for providing the road data, wherein the method: - checks whether road data provided by the navigation device is available that meets the specified accuracy, - if road data provided by the navigation device is available that meets the specified accuracy, the attention area (5) is determined depending on the road data provided by the navigation device, and - otherwise checks whether road data provided by the environment sensor is available that meets the specified accuracy, - if road data provided by the environment sensor is available that meets the specified accuracy, the attention area (5) is determined depending on the road data, and - otherwise the attention area (5) is determined depending on the vehicle data. Method according to one of the preceding claims, further comprising a gaze sensor device for determining a direction parameter that is representative of an actual gaze direction of the driver, and in the method - depending on the direction parameter and the attention area (5) - an attention parameter is determined that is representative of an estimated attention of the driver. Method according to claim 3, wherein a warning or assistance function is controlled depending on the attention indicator. Method according to claim 3 or 4, wherein the direction parameter is representative of a driver's head pose. Assistance system for a vehicle (1), comprising: - an environment sensor for determining road data that is representative of the road course (2) in the direction of travel of the vehicle (1), - a sensor system for determining vehicle data that is representative of a trajectory (3) of the vehicle (1) in the direction of travel, wherein the vehicle data includes at least one of the steering angle, yaw rate, wheel angle or wheel speeds, and - a control unit that is configured to carry out the method according to one of claims 1 to 5. Assistance system according to claim 6, further comprising a navigation device for providing road data. Assistance system according to claim 6 or 7, further comprising a gaze sensor device for determining a direction parameter that is representative of an actual gaze direction of a driver of the vehicle. Computer program for operating an assistance system, wherein the computer program is configured to perform a method according to one of claims 1 to 5 when executed on a data processing device. A computer program product comprising executable program code, wherein the program code, when executed by a data processing device, performs the method according to any one of claims 1 to 5.

Images