Methods for operating a driver assistance system and driver assistance system
The driver assistance system uses a large language model to analyze and explain driving deviations, enhancing the driver's skills by comparing their behavior with expert data, thus optimizing driving techniques on specific routes.
Patent Information
- Application Number
- DE102023132218
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-11-20
- Publication Date
- 2024-11-07
- Estimated Expiration
- 2043-11-20
AI Technical Summary
Existing driver assistance systems lack the ability to interactively explain driving instructions and optimize driving behavior for designated routes, particularly in scenarios like classic racing courses, without providing clear explanations or options for the driver.
A driver assistance system utilizing a large language model trained with expert driving data, including audio-visual recordings and vehicle parameters, to identify deviations and provide acoustic explanations to enhance the driver's understanding and optimize their driving techniques.
Enables interactive learning by highlighting deviations and providing explanations, thereby improving the driver's driving skills and knowledge through comparative analysis with expert techniques.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] The present invention relates to a method for operating a driver assistance system. Furthermore, the driver assistance system is presented.
[0002] For designated routes, such as classic race tracks, a wealth of expert knowledge is available, along with a large amount of data on vehicle parameters, which can suggest optimal driving behavior with regard to lane, speed, gear shifting, etc. However, such driving instructions, or in the case of autonomous driving, driving executions, are carried out without explanation or explanation options for the driver.
[0003] The publication DE 10 2005 055 744 A1 discloses a driver assistance system that records traffic and vehicle data and calculates the optimal trajectory, particularly for upcoming curves that the driver may not be able to fully visualize. Based on this trajectory, instructions for driving behavior are generated, which can be transmitted to the driver via various channels: visually, for example, via a display or a head-up display, acoustically through spoken language, or haptically, for example, through vibration of the seat or steering wheel. The driver thus receives information regarding speed, steering, braking, and, if necessary, gear shifting, in order to better adapt driving dynamics to the traffic situation and road layout.
[0004] The publication DE 10 2010 003 985 A1 describes a driver assistance system that uses sensors to collect data on the current driving situation and driving dynamics. This data is used to generate driving recommendations for the driver, which can then be provided visually, acoustically, or haptically, and even enable direct changes to the vehicle settings. Furthermore, information from various vehicle systems is collected centrally to provide driving recommendations based on the driver's optimization criteria. Furthermore, the system is designed to automatically recognize the driver and adapt the driving recommendations to their abilities and driving experience.
[0005] The publication DE 10 2018 214 665 A1 proposes comparing a planned journey with the actual course and intervening if necessary to reduce deviations. Furthermore, the extent to which the driver accepts past deviations should be examined based on driver behavior in order to adapt the planned course to the expected actual course and / or select an assistance profile depending on the result. The system provides the driver with appropriate notifications on a display, via loudspeaker, or by vibrating the steering wheel.
[0006] Against this background, it is an object of the present invention to present a method for assisting a car journey that enables interaction with a driver for route explanation and trip optimization. Furthermore, a driver assistance system that enables such interaction is to be presented.
[0007] To solve the above-mentioned problem, a method for operating a driver assistance system is proposed, in which the driver assistance system comprises a large language model and in which a training data set is provided for a given vehicle and a given route. The training data set comprises: • Recorded expert driving behavior of the respective specified vehicle during a drive on the specified route carried out by experts; • expert knowledge of a driving technique provided by experts during a drive on the specified route carried out by experts, recorded in acoustic and visual form; • expert knowledge of driving techniques provided by experts during a general driving route carried out by experts, recorded in acoustic and visual form, e.g. with explanations of cornering, acceleration, braking; • Recorded camera data from the respective inspection carried out by experts; • Measured vehicle parameters for individual route sections from the respective inspections carried out by experts; • Technical information on driving dynamics for the specific vehicle provided by experts.
[0008] The driver assistance system is trained using the training data set. When driving on the specified route, the driver is shown any deviations between the current driving behavior and the expert driving behavior. In response to the driver's acoustic questions, the large language model provides information from the training data acoustically.
[0009] The method according to the invention is advantageously capable of detecting deviations between lanes selected by experts and a currently selected lane or trajectory and of suggesting corrective action. Furthermore, these corrective action suggestions can be made audibly to the driver using the large language model. This enables the user, i.e. the driver, to optimize their driving technique. Furthermore, the driver is enabled to receive additional information about the route being traveled, thus gaining a deeper understanding of specific driving situations and appropriate measures for appropriate driving behavior. For example, if the driver were to ask, "Why do I have to accelerate immediately after a bend?", the trained driver assistance system would respond with a "Kamm circle" response, along with an accompanying explanation.Furthermore, by means of the method according to the invention, currently measured driving dynamics can be compared with the driving dynamics considered optimal by experts, i.e. by comparing actual parameters and target parameters, and from this conclusions can be drawn about the driver's own driving behavior, which is advantageous for the driver. It is therefore the task of the large language model as a form of artificial intelligence to recognize differences between driving behaviors, draw conclusions from them, and report these back to the driver in natural language. By embedding ideal cases of professional drivers or experts, the large language model is thus able to identify deviations between the driver's own driving behavior and ideal behavior on the specified route and to justify them using natural language and trained specialist knowledge. This also advantageously increases the learning effect and knowledge of specific driving situations and necessary measures.
[0010] It is conceivable that the large language model could generate specific answers that were not previously part of the training data. To achieve this, the large language model learns contextual relationships during a training process that can be reproduced in the same way but not word for word. For the same questions in different situations, the driver assistance system can thus provide answers that are only relevant to that situation and / or repeat answers in different words, e.g., if the driver does not immediately understand the answer, or adapt the scope of the answer to the driver and the situation.
[0011] In one embodiment of the method according to the invention, at least one predetermined route is selected from the following list: Hockenheimring race track, Nürburgring race track, Lausitzring race track, Sachsenring race track, Deutsche Alpenstraße panoramic road, Circuit de Spa-Francorchamps, Oschersleben race track, Circuit des 24 Heures (Le Mans), Silverstone Circuit, Autodromo Nazionale di Monza. In a further embodiment of the method according to the invention, the vehicle is controlled by the driver assistance system on the predetermined route according to the expert driving behavior provided in the training data set.
[0012] In yet another embodiment of the method according to the invention, the driver is supported by the driver assistance system in his driving behavior in selected route sections.
[0013] In a still further embodiment of the method according to the invention, the support by the driver assistance system is limited to at least one of the following areas: steering, braking behavior, acceleration behavior, gearshift behavior.
[0014] Furthermore, a driver assistance system is claimed, in which the driver assistance system comprises a computer processor and a storage medium, a large language model stored on the storage medium and executable on the computer processor, means for voice input and output, and means for providing driver assistance on a respective specified vehicle. A training data set is provided for the respective specified vehicle and a respective specified route. The training data set comprises: • Recorded expert driving behavior of the respective specified vehicle during a drive on the specified route carried out by experts; • expert knowledge of a driving technique provided by experts during a drive on the specified route carried out by experts, recorded in acoustic and visual form; • expert driving technique knowledge provided by experts during a general driving route driven by experts, recorded in audible and visual form; • Recorded camera data from the respective inspection carried out by experts; • Measured vehicle parameters for individual route sections from the respective inspections carried out by experts; • Technical information on driving dynamics for the specific vehicle provided by experts.
[0015] The driver assistance system is trained using the training data set. The driver assistance system is configured to indicate deviations between current driving behavior and expert driving behavior to the driver during driving on the specified route and to provide information from the training data acoustically in response to driver questions using the large language model.
[0016] In one embodiment of the driver assistance system according to the invention, at least one predetermined route is selected from the following list: Hockenheimring race track, Nürburgring race track, Lausitzring race track, Sachsenring race track, Deutsche Alpenstraße panoramic road, Circuit de Spa-Francorchamps, Oschersleben race track, Circuit des 24 Heures (Le Mans), Silverstone Circuit, Autodromo Nazionale di Monza.
[0017] In a further embodiment of the driver assistance system according to the invention, it is configured to control the vehicle on the predetermined route according to the expert driving behavior provided in the training data set.
[0018] In yet another embodiment of the driver assistance system according to the invention, it is configured to support the driver in his driving behavior in selected sections of the route.
[0019] In a further further embodiment of the driver assistance system according to the invention, the support by the driver assistance system is limited to at least one of the following areas: steering, braking behavior, acceleration behavior, gearshift behavior.
[0020] Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawings.
[0021] It is understood that the features mentioned above and those to be explained below can be used not only in the combination specified in each case, but also in other combinations or on their own, without departing from the scope of the present invention.
[0022] Fig. 1 shows schematically an embodiment of the driver assistance system according to the invention.
[0023] In Fig.1 schematically shows an embodiment of the driver assistance system according to the invention. A trajectory comparison 10 for a predetermined route, which, as shown here, forms a circuit, shows a different expert route 12 and driver route 13. For a single route point, here "0," where "0" or "1" can be, for example, a parameter setting for an extended rear wing, a respective trajectory 17 driven by an expert 15 and a respective trajectory 18 driven by the driver 16 can be displayed. Trajectories 11 that can be assigned by matching or a similarity metric, as well as driver behavior 14 that deviates from them, are fed 51 to a large language model 50, referred to in English as a “Large Language Model” and based as a neural network in the field of machine learning or artificial intelligence (AI). Any driver behavior 14 that deviates from this is also fed to the large language model 50.The trajectory comparison 10 receives as input 21 measurements by vehicle sensors information or data 20 on the current position or trajectory, further sensor data and driving dynamics. For comparison with this data 20, data 30 obtained from expert driving behavior or from recorded camera and vehicle data during the driving of predetermined and / or general routes by experts is stored in a storage medium and is also fed 31 to the trajectory comparison 10. In addition, for training the driver assistance system or the large language model 50 comprised thereof, specialist knowledge 40 provided by experts on driving dynamics behavior related to a respective predetermined route is input 54. In a question / answer dialog 56, 65 between driver 60 and the large language model 50, information from the training data is provided acoustically to a driver 60. List of reference symbols 10 Trajectory comparison 11 Assignable trajectories expert / driver behavior 12 Expert driving on the route 13 Driver driving on route 14 Deviating driver behavior 15 Representation of the target trajectory at a specific route point 16 Representation of the actual trajectory at a specific route point 17 Expert trajectory 18 Driver trajectory 20 vehicle data regarding trajectory, sensor data, driving dynamics 21 Vehicle input for trajectory comparison 30 expert data regarding trajectory, sensor data, driving dynamics 31 Expert input on trajectory comparison 40 Expert knowledge 50 Large Language Model 51 Input from trajectory comparison 54 Input from knowledge database 56 Language model dialog response to driver 60 Vehicle control (ADAS, driver) 65 Language model dialog question from driver
Claims
[1] Method for operating a driver assistance system, in which the driver assistance system comprises a large language model (50) in which a set of training data is provided for a respective predetermined vehicle and a respective predetermined route, the set of training data comprising: • Recorded expert driving behavior of the respective specified vehicle during a drive on the specified route carried out by experts; • expert knowledge (40) on a driving technique provided by experts during a drive (12) of the specified route carried out by experts, recorded in acoustic and visual form; • expert knowledge (40) on driving technique during a general route driving test carried out by experts, recorded in acoustic and visual form; • Recorded camera data from the respective inspection carried out by experts; • Measured vehicle parameters (30) for individual route sections from the respective inspection carried out by experts; • specialist information on driving dynamics for the respective specified vehicle provided by experts; in which the driver assistance system is trained with the amount of training data, in which, when driving on the specified route, a driver (60) is shown deviations between current driving behavior and the expert driving behavior, in which, in response to questions (65) from the driver (60), information from the training data is provided acoustically by the large language model (50). [2] Method according to claim 1, in which at least one predetermined route is selected from the following list: Hockenheimring race track, Nürburgring race track, Lausitzring race track, Sachsenring race track, Deutsche Alpenstraße panoramic road, Circuit de Spa-Francorchamps, Oschersleben race track, Circuit des 24 Heures (Le Mans), Silverstone Circuit, Autodromo Nazionale di Monza. [3] Method according to one of the preceding claims, in which the vehicle is controlled by the driver assistance system on the predetermined route in accordance with the expert driving behavior provided in the training data set. [4] Method according to one of the preceding claims, in which the driver (60) is supported by the driver assistance system in his driving behavior in selected route sections. [5] Method according to claim 4, wherein the support by the driver assistance system is limited to at least one of the following areas: steering, braking behavior, acceleration behavior, gearshift behavior. [6] Driver assistance system, in which the driver assistance system comprises a computer processor and a storage medium, a large language model (50) stored on the storage medium and executable on the computer processor, means for voice input and voice output, and means for performing driver assistance on a respective predetermined vehicle, wherein a set of training data is provided for the respective predetermined vehicle and a respective predetermined route, wherein the set of training data comprises: • Recorded expert driving behavior of the respective specified vehicle during a drive on the specified route carried out by experts; • expert knowledge (40) on a driving technique provided by experts during a drive (12) of the specified route carried out by experts, recorded in acoustic and visual form; • expert knowledge (40) on driving technique during a general route driving test carried out by experts, recorded in acoustic and visual form; • Recorded camera data from the respective inspection carried out by experts; • Measured vehicle parameters (30) for individual route sections from the respective inspection carried out by experts; • specialist information on driving dynamics for the respective specified vehicle provided by experts; wherein the driver assistance system is trained with the amount of training data, wherein the driver assistance system is configured to indicate to a driver (60) deviations between a current driving behavior and the expert driving behavior during driving on the specified route and to provide information from the training data acoustically in response to questions (65) from the driver (60) using the large language model (50). [7] Driver assistance system according to claim 6, wherein at least one predetermined route is selected from the following list: Hockenheimring race track, Nürburgring race track, Lausitzring race track, Sachsenring race track, Deutsche Alpenstraße panoramic road, Circuit de Spa-Francorchamps, Oschersleben race track, Circuit des 24 Heures (Le Mans), Silverstone Circuit, Autodromo Nazionale di Monza. [8] Driver assistance system according to one of claims 6 or 7, which is configured to control the vehicle on the predetermined route according to the expert driving behavior provided in the training data set. [9] Driver assistance system according to one of claims 6 to 8, which is configured to support the driver in his driving behavior in selected route sections. [10] Driver assistance system according to claim 9, wherein the support by the driver assistance system is limited to at least one of the following areas: steering, braking behavior, acceleration behavior, gearshift behavior.
Citation Information
Patent Citations
Driver assisting system for motor vehicle, has comparison module generating information for guiding vehicle based on dynamic characteristics measured by dynamic module, where information is displayed by display device
DE102005055744A1
Procedure for operating a driver assistance system of a motor vehicle and motor vehicle
DE102010003985A1
Mentoring device for assisting a user in handling a predetermined object, motor vehicle and procedure
DE102018214665A1
Cited By
Method and device for data processing in vehicles on recurring routes
DE102025101800A1
