Computer-implemented method for assisting a driver of a vehicle, system, vehicle and computer program product

Abstract

A computer-implemented method for assisting the driver of a vehicle, comprising: (100) acquiring sensor data by means of at least one vehicle sensor, wherein the sensor data represent the environment and / or distances between the vehicle and objects in the environment of the vehicle; (200) determining distance data between the vehicle and at least one object in the environment of the vehicle based on the acquired sensor data; and (600) displaying at least one visual cue on a display of the vehicle depending on the determined distance data, wherein the visual cue represents a distance between the vehicle and an object in the environment, wherein the visual cue comprises patterns and / or shades of gray and / or high-contrast colors which are easily distinguishable for persons with color vision deficiencies;and / or generation (700) of at least one acoustic and / or haptic signal depending on the determined distance data for the driver.;

Description

[0001] The present invention relates to a computer-implemented method for assisting the driver of a vehicle. The present invention also relates to a system configured to carry out the computer-implemented method for assisting the driver. Furthermore, the invention relates to a vehicle with the system and a computer program product comprising commands which, when the program is executed by a computer, cause the computer to perform the steps of the method according to the invention. State of the art

[0002] The display of a distance to an object in the vicinity of a vehicle by driver assistance systems is known, particularly for parking assistance systems, where the distances are color-coded; in particular, an impending collision or a distance below a certain threshold is indicated in red. Different markings may be provided for different distance ranges. Additionally, acoustic warnings of impending collisions are generally known.

[0003] The object of the present invention is to improve a method for assisting a driver of a vehicle. Disclosure of the invention

[0004] The above problem is solved according to the invention in accordance with independent claims 1 and 9 to 11.

[0005] The present invention relates to a computer-implemented method for assisting a vehicle driver, in particular a parking assistance method, which supports the driver when parking or maneuvering the vehicle out of a parking space. The method comprises the acquisition of sensor data by means of at least one vehicle sensor, wherein the sensor data represent the environment and / or distances between the vehicle and objects in the vehicle's environment. The vehicle sensor comprises a lidar sensor, an ultrasonic sensor, a radar sensor, and / or a camera sensor. Subsequently, distance data between the vehicle and objects in the vehicle's environment are determined based on the acquired sensor data.

[0006] In a further step of the invention, at least one visual indicator is displayed on a vehicle display depending on the determined distance data. The visual indicator represents a distance between the vehicle and an object in the environment and includes patterns and / or shades of gray and / or high-contrast colors, which are particularly easy to distinguish for people with color vision deficiencies. An advantage of this display is the increased user-friendliness and accessibility for a wider user group, including people with visual impairments.

[0007] Alternatively or additionally, at least an acoustic and / or haptic signal can be generated for the driver based on the determined distance data, which is particularly easy to perceive for people with color vision deficiencies. This offers the advantage that the driver is informed about the distances even without visual cues or in addition to the visual cue, thus increasing safety and reaction time.

[0008] In a preferred embodiment, various patterns and / or shades of gray and / or high-contrast colors serve as visual cues on the display, representing different distances or distance ranges between the vehicle and an object in the vehicle's vicinity. Specifically, an increasing fill level of the visual cue represents a decreasing distance between the object and the vehicle, and thus an increasing risk of collision. For example, dotted patterns represent a safe distance, striped patterns a distance without collision and less than or equal to a threshold value, and solid patterns an imminent collision with an object in the vehicle's vicinity. Optionally, the fill level is continuously and constantly adjusted to the detected distance to the object as the vehicle approaches it.This enables the driver to intuitively and quickly recognize the dangerous situation despite the driver's color vision deficiency.

[0009] In a further embodiment, the acoustic signal comprises at least one tone and / or a voice message, where the tone and / or voice message represents the distance between the vehicle and an object in the environment, the type of object, and / or the progress of a parking maneuver. This offers the advantage that the driver receives detailed and specific information without having to take their eyes off the road.

[0010] Preferably, the haptic signal is generated at the steering wheel and / or the driver's seat, particularly by means of at least one vibration actuator or vibration actuator assembly. The vibration frequency and / or intensity represents information about the distance between the vehicle and an object in the environment, the object type, and / or the progress of the parking maneuver. This enables discreet yet effective communication of important information to the driver. The at least one vibration actuator is, for example, positioned at the steering wheel and / or in the driver's seat, such that a vibration of the actuator is easily perceptible to the driver.

[0011] In a further advantageous embodiment, the visual cue, the acoustic signal, and / or the haptic signal are additionally generated depending on the position of the nearest object relative to the vehicle. For example, the steering wheel comprises at least four vibration actuators for this purpose, with the vibration as a haptic signal always being generated at the point on the steering wheel in the direction of which the nearest object to the vehicle is positioned. The generation of the haptic signal can also be dependent on the current steering angle.For example, a haptic signal is generated as a vibration at the top of the steering wheel for an object positioned in front of the vehicle at a short distance, and / or as a vibration at the bottom of the steering wheel for an object positioned behind the vehicle at a short distance, and / or as a vibration on the right side of the steering wheel for an object positioned on the right side of the vehicle at a short distance, and / or as a vibration on the left side of the steering wheel for an object positioned on the left side of the vehicle at a short distance, where a short distance is, in particular, a distance between the vehicle and the object of less than one meter. Alternatively, analogous to the above descriptions, the vehicle includes at least four loudspeakers, with the acoustic signal always being generated as a haptic signal by the loudspeaker in the direction towards which the object closest to the vehicle is positioned.This design makes the direction of an impending collision or the location of the nearest object particularly easy to perceive, even for people with visual impairments.

[0012] In an additional configuration, vehicle operating parameters are recorded, including odometry data such as steering angle and vehicle speed. Based on these parameters, a predicted trajectory is generated. A visual cue is then displayed on the vehicle's screen, and / or an audible and / or haptic signal is generated, depending on the predicted trajectory. This provides the advantage of proactive driver assistance, enhancing safety and precision when parking.For example, in this version no visual indication is given if, based on the determined predicted trajectory, there is no risk of collision with a nearby object in the environment, because, for example, the vehicle will safely pass the object by making a sharp steering maneuver.

[0013] In a further embodiment, an environmental model is generated based on the acquired sensor data and / or the determined distance data, whereby the environmental model is specifically a surround view or bird's eye view (virtual bird's-eye view). The generated environmental model is displayed, and the visual cue, as well as optionally the calculated predicted trajectory, are shown as an overlay within the environmental model. This offers the advantage of a comprehensive and clear representation of the surrounding situation, which facilitates the driver's orientation and decision-making.

[0014] A further aspect of the invention comprises the detection of driver input representing at least reduced color vision, and / or the performance of a color vision test of the driver, in particular by displaying at least one test pattern and detecting driver input based on the displayed test pattern, as well as determining a test result based on the driver's input. The visual cue and / or the acoustic and / or haptic signal are displayed or generated depending on the detected input and / or the determined test result. This offers the advantage of individually adapting the assistance functions to the driver's needs, which increases user-friendliness and safety.

[0015] The invention also includes a system for assisting the driver of a vehicle, comprising at least one vehicle sensor, which includes at least one lidar sensor, one ultrasonic sensor, one radar sensor, and / or one camera sensor. The vehicle sensor is configured to acquire sensor data representing the environment and / or distances between the vehicle and objects in the vehicle's vicinity. The system includes a display device comprising a screen for showing a visual cue and / or a loudspeaker for generating acoustic signals and / or an actuator for generating haptic signals. The system also includes a processing unit.The computing unit is configured to execute the method according to the invention, wherein the computing unit determines distance data between the vehicle and objects in the vicinity of the vehicle based on the acquired sensor data and generates control signals for displaying a visual indication and / or for generating at least one acoustic and / or haptic signal by means of the display device, each depending on the determined distance data.

[0016] The invention also relates to a vehicle which includes a system according to the invention for supporting a driver. The vehicle offers the advantage of improved parking assistance and increased safety for the driver and the surroundings.

[0017] Finally, the invention comprises a computer program which includes instructions that, when the program is executed by a computer, cause it to perform the steps of the method according to the invention.

[0018] Further advantages will become apparent from the following description of the exemplary embodiment with reference to the figure. Fig. 1: Flowchart of the procedure Example of implementation

[0019] In Fig.Figure 1 is an exemplary embodiment of the computer-implemented method for assisting a driver, schematically represented as a block diagram. The method improves driver safety and parking comfort, particularly when parking and / or exiting parking spaces for drivers with color vision deficiencies. The method may initially include an optional step 50 in which input from the driver regarding their color vision deficiency is recorded and / or a test is performed, the test determining the driver's color vision as the test result. The method is preferably continued or executed depending on the recorded input and / or the test performed or the test result obtained. In a step 100, sensor data is acquired using at least one vehicle sensor. A lidar sensor, an ultrasonic sensor, a radar sensor, and / or a camera sensor can be used as the sensor.In other words, the vehicle sensor preferably comprises a lidar sensor, an ultrasonic sensor, a radar sensor, and / or a camera sensor. The acquired sensor data represents information about the vehicle's surroundings, in particular distances between the vehicle and objects in the environment, especially in the immediate vicinity of one, two, three, four, or five meters or less. The advantageously diverse sensor array enables robust and comprehensive environmental sensing, even under challenging conditions. The combination of different sensor types and subsequent sensor data fusion (not shown) increases the reliability and accuracy of the distance measurement. Based on the acquired sensor data, 200 distance measurements between the vehicle and objects in the environment are calculated or determined in the following step.This step 200 can, for example, include signal processing, sensor data fusion, and / or object detection and localization, particularly using a neural network. In other words, step 200, used to determine distance data, can also employ neural networks, such as feature generators or transformer models, including, for example, self- and / or cross-attention mechanisms, with potentially multiple specific detection heads for object detection and / or mapping, or two- or three-dimensional map creation. Calculating precise distance data enables an accurate assessment of the situation and / or the determination of a convenient and fast trajectory for parking or maneuvering, for example, in a parking situation.In an optional step 300, further vehicle operating parameters, in particular odometry data such as the current steering angle and the vehicle's current speed, can be acquired and used for the optional calculation 400 of a predicted trajectory. Optionally, in step 500, an environment model is created and displayed based on the sensor data and / or distance data, in particular by means of a display or other display device. For example, a surround view or a virtual bird's-eye view is displayed as the environment model. Displaying the environment as a surround view or from a virtual bird's-eye view provides the driver with a comprehensive overview of the parking situation and facilitates orientation.Depending on the calculated distance data, at least one visual cue is determined and displayed in the next step (600) via the display or the vehicle's indicator device. The visual cue is optionally displayed based on the driver's input or the test result from step 50, thus adapting the visual cue to the driver's color vision deficiency. The visual cue can optionally be displayed as an overlay on the environmental model. The visual cue is also optionally displayed based on the calculated predicted trajectory. Considering the vehicle's movement and the predicted trajectory for the visual cue enables proactive driver assistance, further improving precision and safety when parking.The displayed visual cue represents at least the distance to at least one object in the vehicle's vicinity and includes patterns, shades of gray, and / or high-contrast colors that are easily distinguishable even for people with color vision deficiencies. For example, different pattern fill levels can represent different distances: dotted for a safe distance, striped for a short distance, and solid for an imminent collision. The fill level is continuously and steadily adjusted based on the determined distance data; in particular, the fill level of the visual cue increases with increasing distance. The visual representation of distances, which includes patterns, shades of gray, and / or high-contrast colors, advantageously enables an intuitive and rapid assessment of the surroundings, especially for drivers with impaired color vision.The use of patterns and grayscale improves the ability to distinguish distances for people, especially drivers with visual impairments. The trajectory can then also be displayed, for example, as an overlay in Surround View or Birds Eye View. In addition to or as an alternative to the visual cue, an acoustic and / or haptic signal can be generated in step 700. The acoustic signal can consist of tones or voice information indicating the distance, object type, or progress of the parking maneuver. The haptic signal, such as vibrations in the steering wheel or seat (e.g., in the backrest or seat cushion), can also convey information about the distance, object type, or parking progress.For example, in step 700, a haptic signal is generated in the seat and / or steering wheel, whereby the frequency and / or intensity of the haptic signal changes based on distance data or the distance to the nearest object. In particular, the frequency and / or intensity of the haptic signal increases as the distance decreases. The haptic signal can be generated depending on the location or position(s) of the nearest object(s) and the distance to the object(s). Acoustic and haptic signals provide additional warnings and information without requiring the driver to take their eyes off the road. This increases safety and comfort when parking, especially in complex situations.The visual cue and the acoustic / haptic signals can also be adjusted depending on the predicted trajectory.

[0020] This embodiment illustrates the functionality of the method according to the invention and the advantages of the individual steps. The optional extension, for example to include color vision deficiency detection, trajectory calculation, and / or environmental modeling, allows for flexible adaptation. The method can be implemented in a control unit or a central processing unit of a vehicle and contributes significantly to improving driver assistance for drivers with color vision deficiency and thus to road safety.

Claims

[1] Computer-implemented method for assisting the driver of a vehicle, comprising: ◯ Acquisition (100) of sensor data by means of at least one vehicle sensor, wherein the sensor data represent the environment and / or distances between the vehicle and objects in the environment of the vehicle; ◯ Determination (200) of distance data between the vehicle and at least one object in the vicinity of the vehicle based on the acquired sensor data; and ◯ Display (600) at least one visual cue on a display of the vehicle depending on the determined distance data, wherein the visual cue represents a distance between the vehicle and an object in the environment, wherein the visual cue comprises patterns and / or shades of gray and / or high-contrast colors which are easily distinguishable for persons with color vision deficiencies; and / or ◯ Generation (700) of at least one acoustic and / or haptic signal depending on the determined distance data for the driver. [2] Method according to claim 1, wherein different patterns and / or shades of gray and / or high-contrast colors represent different distances as visual cues on the display, wherein in particular an increasing fill level of the visual cue represents a decreasing distance between an object and the vehicle, wherein the fill level of the visual cue is in particular continuously and steadily increased depending on the distance to the object. [3] Method according to any of the preceding claims, wherein the acoustic signal comprises at least one tone and / or speech information, wherein the tone and / or speech information represents at least the distance between the vehicle and an object in the environment, the object type and / or the progress of a parking maneuver. [4] Method according to one of the preceding claims, wherein the haptic signal is generated in particular at the handlebar and / or at the seat of the driver, in particular by means of a vibration actuator, wherein in particular a vibration frequency and / or an intensity of the vibration of the vibration actuator represents at least one piece of information about the distance between the vehicle and an object in the environment, the object type and / or the progress of the parking maneuver. [5] Method according to one of the preceding claims, wherein the visual cue, acoustic signal and / or haptic signal is additionally generated depending on the position of the nearest object to the vehicle. [6] A method according to any of the preceding claims, wherein the following steps are performed: ◯ Acquisition (300) of operating parameters of the vehicle, wherein the operating parameters include the steering angle and the speed of the vehicle; and ◯ Determination (400) of a predicted trajectory of the vehicle based on the recorded operating parameters; wherein ◯ the display (600) of the visual cue on a display of the vehicle is additionally carried out depending on the determined predicted trajectory, and / or ◯ the generation (700) of at least one acoustic and / or haptic signal additionally depending on the determined predicted trajectory. [7] A method according to any of the preceding claims, wherein the following steps are performed: ◯ Determination and display (500) of an environment model based on the acquired sensor data and / or the determined distance data, wherein ◯ the display (600) of the visual cue as an overlay and optionally of the determined predicted trajectory as an overlay in the environment model is performed. [8] A method according to any of the preceding claims, wherein the following steps are performed: ◯ Recording (50) an input from the driver representing color vision deficiency and / or performing a test, wherein test patterns are displayed to the driver via the display and the driver's color vision deficiency is determined as a test result; wherein ◯ the display (600) of the visual cue and / or the generation of the acoustic and / or haptic signal depending on the detected and / or determined color vision deficiency of the driver. [9] System for assisting the driver of a vehicle, comprising: ◯ at least one vehicle sensor comprising at least one lidar sensor, one ultrasonic sensor, one radar sensor and / or one camera sensor, which are configured to capture sensor data representing the environment and / or distances between the vehicle and objects in the vehicle's vicinity; ◯ a display device comprising at least one display for showing a visual indication and / or a loudspeaker for generating acoustic signals and / or an actuator for generating haptic signals; and ◯ a computing unit which is configured to execute a method according to one of the preceding claims, wherein the computing unit determines distance data between the vehicle and at least one object in the vicinity of the vehicle based on the acquired sensor data and generates control signals for displaying a visual indication and / or for generating at least one acoustic and / or haptic signal by means of the display device, each depending on the determined distance data. [10] Vehicle comprising a system according to claim 9. [11] Computer program comprising instructions which, when the program is executed by a computer, cause it to perform the steps of the method according to any one of claims 1 to 8.

Images