Method and system for an individual vehicle AI agent

Abstract

A method for providing AI-supported vehicle operation, wherein a vehicle comprises a computing unit on which an AI agent for processing input signals is installed, the AI ​​agent being configured to control at least one technical device depending on the input signals, several interior sensors (13, 14) for recording a predetermined set of actions, and a driver display (11) wherein the interior sensors (13, 14) are directed towards a driver's seat (18), wherein the interior sensors (13, 14) continuously measure each action of a respective driver, and wherein, in a first step before driving operation, the AI ​​agent displays at least one changeable control parameter for at least one technical device of the vehicle on the driver display (11).The respective driver executes a specific action from the predefined set of action processes, and the AI ​​agent is trained to assign the measured action as an input signal to a change in at least one control parameter of at least one technical device, in a second step parameters describing the AI ​​agent are stored on a storage medium and the storage medium is individually assigned to the respective driver, and in a third step parameters describing the AI ​​agent are loaded from the storage medium for a driving operation performed by the respective driver, and when a respective action of the respective driver is measured, at least one control parameter is changed by the AI ​​agent, whereby the AI ​​agent is animated as an avatar (12) on the driver display (11).and in which a change of the respective control parameter is animated as a change to a control of the at least one technical device carried out with the help of the animation of the avatar (12).

Description

The present invention relates to a method for an individual vehicle AI agent, which enables a user-tailored control panel. Furthermore, a system with which the method can be implemented is presented. With advancing digitalization, new operating and interaction concepts for designing human-machine interfaces in technical devices are increasingly appearing in everyday life. Furthermore, machine learning methods, or artificial intelligence (AI), have developed ever greater capabilities to cater to the individual preferences of a user of a given technical device, thus facilitating its operation. To enable rapid intervention in critical situations, optimal adaptation of operating elements to the individual user is particularly important. German patent application DE 10 2022 201 859 A1 describes a method for user-specific operation of a vehicle system using a support module with artificial intelligence. During vehicle operation, the module first performs an interaction process in which it issues suggestions for controlling the vehicle system and records user reactions to these suggestions. These interactions serve to train the artificial intelligence. Subsequently, the module checks the confidence level of the artificial intelligence by evaluating the agreement between the user reactions and the suggested actions. If an agreement criterion is met, the module increases the confidence level. In a second interaction process, the module then issues further suggestions based on the increased confidence level.User interactions can be captured via a microphone or touchscreen, for example, and the suggested actions can be displayed visually or audibly. Furthermore, the artificial intelligence can be assigned to a user profile to enable use across multiple vehicles. German patent application DE 10 2010 036 666 A1 discloses a system for intelligent music selection in a vehicle that learns user preferences according to different driving conditions. The system consists of an artificial intelligence module for music and a context-aware music player. The module learns user preferences for music selection based on various driving conditions and selects music according to the current driving situation based on these learned preferences. The vehicle can also feature an emotional guidance system with a natural language interface and a visually represented avatar that interacts with the user and conveys emotions. Document WO 2008 / 134 625 A1 discusses an Emotional Support System (ESS) for a vehicle that uses an avatar or animated image to convey information to the driver. Sensors in the vehicle capture driver input and convert it into digital information, which is processed by a computer. The avatar can change its appearance and audio output based on the captured data, which reflects the driver's personal attributes. For example, the avatar can react to the driver's emotional state and adjust its interactions accordingly. Furthermore, the ES can respond to driver commands and modify vehicle parameters, such as engine tuning for improved performance. The ES can also learn to anticipate the driver's requests, commands, and preferences based on the history of interactions between the driver and the ES. German patent DE 10 2020 003 718 A1 relates to a method for assigning input signals to vehicle functions. An action by a vehicle occupant is detected, and when converted into an input signal, this action triggers the execution of a vehicle function. Depending on the emotional reaction of the vehicle occupant to this execution, detected by further sensors, a probability distribution regarding the assignment is modified. DE 10 2020 214 206 A1 describes a method for increasing the safety of a vehicle occupant, in which, in a dangerous situation, a trigger signal is provided to an optical display device for optical output as a virtual object. The virtual object, in particular represented as a hologram, is intended to cause a startle response. German patent DE 10 2006 049 965 A1 discloses a device for providing information and / or assistance to a motor vehicle user, comprising an output unit, a data acquisition unit for vehicle components and / or driving situations, an evaluation unit, and an explanation unit for information selection or assistance. A presentation method for the information is selectable, with the information being provided automatically depending on a driving situation or at a selected time. Against this background, it is an object of the present invention to present a method for providing a user-tailored control panel. Furthermore, a system with which the method can be carried out is to be presented. To solve the aforementioned problem, a method for providing AI-supported vehicle operation is proposed. In this method, the vehicle comprises a computing unit on which an AI agent for processing input signals is installed. The AI ​​agent is configured to control at least one technical device based on the input signals. The vehicle also includes several interior sensors for recording a predefined set of actions and a driver display. The interior sensors are directed towards the driver's seat and continuously measure each action performed by the driver.In a first step, before a driving operation, the AI ​​agent displays at least one modifiable control parameter for at least one of the vehicle's technical devices on the driver display. The driver then performs one action from a predefined set of actions, and the AI ​​agent is trained to recognize and interpret this action as an input signal for changing the control parameter of at least one of the technical devices. In a second step, parameters describing the AI ​​agent are stored on a storage medium, which is then individually assigned to each driver. In a third step, during a driving operation performed by the driver, these parameters are loaded from the storage medium.The AI ​​agent changes at least one control parameter when measuring a specific action of the respective driver. The inventive method provides the driver with a personal AI agent or avatar to operate the vehicle. Since the AI ​​agent is ultimately defined by a large but finite number of storable parameters, it can, for example, be stored on a USB drive or in the cloud to be made available to the driver in any vehicle whose system allows the use of such an AI agent. This would advantageously eliminate the need to train the driver on a specific vehicle type; instead, it would only require the provision of a suitable interface between the AI ​​agent and the vehicle (type). It is also conceivable that the AI ​​agent is networked with other AI agents in other vehicles via the cloud or a wireless network and exchanges information. Through interaction with the AI ​​agent, the driver could then also indirectly contact another vehicle. The AI ​​agent can automatically adjust the operation or settings of individual functions, such as interior temperature, at the start of the journey based on the stored parameters, so that, for example, a previously set comfort temperature is maintained or the volume of radio or navigation announcements is reduced when the interior sensors detect several vehicle occupants talking. Last but not least, the AI ​​agent, as a centralized control unit or human-machine interface, offers the possibility of eliminating the need for many individual switches, buttons or rotary controls in vehicle operation, thus saving on manufacturing costs. Repeated execution of the inventive method advantageously increases the possibilities of having the learning AI agent perform an ever-increasing number of vehicle operating functions. It is also conceivable to allow the AI ​​agent to interact with automated driving systems. In one embodiment of the method according to the invention, the predetermined set of actions is formed by at least one action of the driver from the following list: speech, touch, gesture, facial expression. "Speech" refers, for example, to voice commands, which—especially due to recent developments in the field of "large speech models"—can also be formulated as complete sentences or questions. "Touch" could be implemented by buttons located on the driver display or the steering wheel, or more generally by a touchscreen functionality of the driver display. A palm moved upwards or downwards could, in the case of a "gesture," increase or decrease a control parameter. And, for example, a smiling face, in the case of a "facial expression," represents confirmation of the execution of a setting, while a frustrated expression would result in the rejection of the last applied setting.The diverse applications of the AI ​​agent for vehicle operation advantageously increase comfort for the driver, as he can, for example, change the same control parameter through different actions, e.g., through voice commands (loud / quiet) and gestures (hand up / down) and touch (fingertip up / down on display). According to the invention, the AI ​​agent is animated as an avatar on the driver display. A change to the respective control parameter is animated as a change to the control of the at least one technical device, carried out with the aid of the avatar's animation. The animation of the avatar on the driver display is shown, for example, throughout the entire driving operation. In a further embodiment of the method according to the invention, an animation of the AI ​​agent is displayed on a screen that is currently within the driver's field of vision. This can be achieved, for example, by projecting the animation of the AI ​​agent onto a windshield, if, for example, interior sensors directed at the driver's eyes detect that the driver is looking towards them. In a further embodiment of the method according to the invention, the AI ​​agent provides a technical description or operating instructions for the associated technical device in response to each change in the at least one control parameter. It is also conceivable that the AI ​​agent is queried about the vehicle's functions and its condition. Furthermore, a system is claimed which comprises a vehicle, a storage medium, a computing unit on which an AI agent for processing input signals is installed, wherein the AI ​​agent is configured to execute control of at least one technical device depending on the input signals, several interior sensors for recording a predetermined set of actions, and a driver display. At least one interior sensor is directed towards a driver's seat, wherein the interior sensors are designed to continuously measure each action of each driver, wherein the system is configured to: • in a first step train the AI ​​agent by: • displaying at least one modifiable control parameter for at least one technical device of the vehicle on the driver display before driving operation.and • a specific action performed by the respective driver from the specified set of actions and measured by the interior sensors is assigned as an input signal to a change in at least one control parameter of at least one technical device, • in a second step, to store parameters describing the AI ​​agent on a storage medium and to assign the storage medium individually to the respective driver, and • in a third step, to load parameters describing the AI ​​agent from the storage medium for a driving operation performed by the respective driver and to change at least one control parameter by the AI ​​agent when measuring a specific action of the respective driver. The parameters describing the trained or learned AI agent can, for example, be combined on a memory stick as an additional function for a general vehicle key of a fleet, so that the respective driver has his individually learned AI agent available on all vehicles of the fleet, provided that a corresponding read function is available in each ignition lock. In one embodiment of the system according to the invention, the specified set of action processes is formed with at least one action from the following list: speech, touch, gesture, facial expression. According to the invention, the AI ​​agent is animated as an avatar on the driver display. A change to the respective control parameter is animated as a change to the control of the at least one technical device, carried out with the aid of the avatar's animation. In a further embodiment of the system according to the invention, it is configured to display an animation of the AI ​​agent on a display that is currently in the field of vision of the respective driver. In a further embodiment of the system according to the invention, it is configured to provide a technical description of the associated technical device via the AI ​​agent for each change of the at least one control parameter. Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawing. It is understood that the features mentioned above and those to be explained below can be used not only in the combinations specified, but also in other combinations or on their own, without leaving the scope of the present invention. Fig. 1 schematically shows a vehicle interior with an animated AI agent in an embodiment of the method according to the invention. Figure 1 schematically shows a vehicle interior 10 with an AI agent 12 animated as an avatar in an embodiment of the method according to the invention. The avatar of the AI ​​agent 12 is displayed on a driver display 13 arranged between the driver's seat 18 and the passenger seat 19. Interaction occurs, for example, via gestures or facial expressions of a driver seated in the driver's seat 18, who is observed by a camera arranged on the driver display 11 as a first interior sensor 13 and a camera arranged on the steering wheel 15 as a second interior sensor 14. The avatar of the AI ​​agent 12 can also be projected onto a windshield 17 or a head-up display 16, so that the driver can continue to monitor traffic ahead. An example of an animation of the avatar of the AI ​​agent 12 is shown in the enlarged display 20 of the driver display. Reference symbol list 10 Vehicle interior diagram 11 Driver display 12 AI agent avatar 13 Camera as first interior sensor 14 Camera as second interior sensor 15 Steering wheel 16 Head-up display 17 Windscreen 18 Driver's seat 19 Passenger seat 20 Display on driver screen

Claims

A method for providing AI-supported vehicle operation, wherein a vehicle comprises a computing unit on which an AI agent for processing input signals is installed, the AI ​​agent being configured to control at least one technical device depending on the input signals, several interior sensors (13, 14) for recording a predetermined set of actions, and a driver display (11) wherein the interior sensors (13, 14) are directed towards a driver's seat (18), wherein the interior sensors (13, 14) continuously measure each action of a respective driver, and wherein, in a first step before driving operation, the AI ​​agent displays at least one changeable control parameter for at least one technical device of the vehicle on the driver display (11).The respective driver executes a specific action from the predefined set of action processes, and the AI ​​agent is trained to assign the measured action as an input signal to a change in at least one control parameter of at least one technical device, in a second step parameters describing the AI ​​agent are stored on a storage medium and the storage medium is individually assigned to the respective driver, and in a third step parameters describing the AI ​​agent are loaded from the storage medium for a driving operation performed by the respective driver, and when a respective action of the respective driver is measured, at least one control parameter is changed by the AI ​​agent, whereby the AI ​​agent is animated as an avatar (12) on the driver display (11).and in which a change of the respective control parameter is animated as a change to a control of the at least one technical device carried out with the help of the animation of the avatar (12). Method according to claim 1, wherein the specified set of action processes is formed with at least one action from the following list: speech, touch, gesture, facial expression. Method according to one of the preceding claims, wherein an animation of the AI ​​agent is displayed on a display (11, 16) that is currently in the field of vision of the respective driver. Method according to one of the preceding claims, wherein the AI ​​agent provides a technical description of the associated technical device for each change of the at least one control parameter. A system comprising a vehicle, a computing unit on which a computer agent for processing input signals is installed, wherein the computer agent is configured to control at least one technical device depending on the input signals, several interior sensors (13, 14) for recording a predetermined set of actions, and a driver display (11), wherein at least one interior sensor (13, 14) is directed towards a driver's seat (18), wherein the interior sensors (13, 14) are designed to continuously measure each action of each driver, wherein the system is configured to: • in a first step train the computer agent by displaying at least one modifiable control parameter for at least one technical device of the vehicle on the driver display (11) before driving operation.and • a respective action performed by the respective driver from the specified set of action processes and measured by the interior sensors (13, 14) is assigned as an input signal to a change of at least one control parameter of at least one technical device, • in a second step to store parameters describing the AI ​​agent on a storage medium and to assign the storage medium individually to the respective driver, and • in a third step to load parameters describing the AI ​​agent from the storage medium for a driving operation performed by the respective driver and to change the at least one control parameter by the AI ​​agent when measuring a respective action of the respective driver, wherein the AI ​​agent is animated as an avatar (12) on the driver display (11),and in which a change of the respective control parameter is animated as a change to a control of the at least one technical device carried out with the help of the animation of the avatar (12). System according to claim 5, wherein the specified set of action processes is formed with at least one action from the following list: speech, touch, gesture, facial expression. System according to one of claims 5 to 6, which is configured to display an animation of the AI ​​agent on a display (11, 16) that is currently in the field of vision of the respective driver. System according to one of claims 5 to 7, which is configured to provide, via the AI ​​agent, a technical description of the associated technical device for each change of the at least one control parameter.

Images