Computer-implemented method for determining a competence level of a user, and vehicle

Abstract

The invention relates to a computer-implemented method for determining a competence level (1) of a user (2) using a vehicle function, in which method the behavior of the user (2) in the vehicle (3) is recorded, and a computing unit (4) automatically determines the competence level (1) on the basis of the observed behavior. The computer-implemented method according to the invention is characterized in that behavior information describing the user behavior is fed to a generative AI model which is executed by the computing unit (4) and which is trained to determine competence levels (1) for vehicle functions by processing corresponding behavior information, and in that the generative AI model determines the competence level (1).

Description

[0001] Mercedes-Benz Group AG

[0002] Computer-implemented method for determining the competence level of a user and vehicle

[0003] The invention relates to a computer-implemented method for determining a user's competence level in the use of a vehicle function according to the type defined in more detail in the preamble of claim 1, and to a vehicle for carrying out the method.

[0004] The user experience (UX) of devices, equipment, machines, applications, software, and the like depends on the prior knowledge and experience of the individual user. Experienced users possess a comparatively extensive understanding of the available functions and their scope, enabling them to use the device or application more effectively, quickly, and efficiently. Inexperienced users, on the other hand, are not fully aware of the available functionality, which can lead to some functions going unused or even resulting in incorrect operation.

[0005] For example, fast, efficient, and intuitive operation of programs or apps is possible when the user enters corresponding keyboard shortcuts or predefined gestures via a touch-sensitive display. This eliminates the need to navigate through extensive menus or submenus, which is time-consuming and places a high cognitive load on the user. A user's knowledge or skills in using a particular device or application are also referred to as a "mental model." Such mental models thus reflect the user's respective level of competence. The mental model describes the user's knowledge and perceptions of the application. To enable a satisfactory user experience, operating concepts should be aligned with the user's respective level of competence, i.e., their mental model.It is therefore particularly important not to overwhelm inexperienced users and to provide experienced users with more in-depth operating options. By providing targeted guidance, users can gradually learn the system's behavior and interaction, gaining a better understanding of how to operate each function. The appropriate design of operating concepts is especially relevant in the automotive environment, as a large number of complex functions are available for use in vehicles. These include the use of various human-machine interfaces such as physical buttons, knobs, and rotary controls; navigation through graphical user interfaces (GUIs), especially using touch-sensitive displays; interaction with voice control systems via voice commands; gesture control; and similar methods.Furthermore, information must be processed across various transmission channels, such as visual, acoustic, and haptic information. For example, visual information is displayed on a screen in the vehicle, acoustic information is emitted via speakers in the vehicle, and haptic information is delivered through vehicle components touched by the user. Additionally, the functions operable within the vehicle relate to diverse areas, such as the vehicle's handling characteristics, particularly longitudinal and lateral acceleration; driver assistance functions, such as adaptive cruise control, high-beam assist, and the like; entertainment functions, such as the playback of media content via the infotainment system; and comfort functions, such as climate control or route guidance via the navigation system.Operating these functions is further complicated by the fact that the driver must simultaneously focus on the driving process. The driver must not be excessively distracted, as this could increase the risk of accidents.

[0006] Several methods exist for determining a user's competence level when using a function or operating a specific device or application. For example, competence level can be assessed using questionnaires. Alternatively, the user's behavior while using the function can be recorded, allowing conclusions to be drawn about their competence level based on how they utilize the function. For instance, if a user utilizes many different sub-functions, this indicates that they are experienced in using the function.

[0007] German patent DE 102006 012 172 A1 discloses a computer-implemented method for the automated comparison of at least one competency topology of a vacant and / or filled position with the competency topology of one or more candidates. The patent discloses the computer-assisted determination of an applicant's competency level for a job opening. The assessment of the competency level is based on the completion of questionnaires. The manner in which the individual completes the questionnaire can be used as an additional parameter for evaluating the competency level. In particular, the order and / or duration of the answers to the respective questions are considered as criteria.

[0008] Furthermore, US 8,731,736 B2 discloses a system and procedure for reducing the risk of losing one's driving skills. The document is based on the premise that a driver gradually loses their driving skills if they do not operate a vehicle or use automated driving functions for an extended period. To assess whether a driver has experienced changes in their driving skills, these changes must first be determined. For this purpose, characteristics are defined that allow conclusions to be drawn about the driver's ability to control the vehicle. Subsequently, the user's behavior in controlling the vehicle is recorded, and the driver's competence level is automatically determined. To evaluate the driver's driving skills, the parameters measured in the vehicle are compared with corresponding parameters of a control subject.The comparison person can be an average driver or even past driving parameters of the driver themselves. Depending on the driver's assessed driving skills or changes in those skills, warnings can be issued or automated vehicle functions can be activated. Furthermore, JP H11-250 395 A discloses an assistance device for a vehicle. This device determines a user's competence level in interacting with vehicle functions based on the user's reaction time. The competence level can be updated by continuously monitoring user behavior.

[0009] Furthermore, DE 102018209 717 A1 discloses a method for controlling the operation of a medical device, a medical device, a computer program, and an electronically readable data carrier. The medical device has an input and an output device for interacting with a user. The output device can display a user interface that can be modified using artificial intelligence to adapt to user behavior.

[0010] The present invention is based on the objective of providing an improved computer-implemented method for determining the competence level of a user in the use of a vehicle function, which makes it possible to enable improved user interaction and increase road safety.

[0011] According to the invention, this problem is solved by a computer-implemented method for determining a user's competence level in using a vehicle function with the features of claim 1. Advantageous embodiments and further developments, as well as a vehicle for carrying out the method, are described in the dependent claims.

[0012] A generic computer-implemented method for determining a user's competence level in using a vehicle function, wherein the user's behavior in the vehicle is recorded and a computing unit automatically determines the competence level depending on the observed behavior, is further developed according to the invention in that behavioral information describing the user's behavior is fed to a generative AI model executed by the computing unit, which is trained to determine competence levels for vehicle functions by processing corresponding behavioral information, and the generative AI model determines the competence level, wherein instructions for using the vehicle function adapted to the user's competence level are displayed in the vehicle.and wherein the generative AI model, by processing behavioral information, identifies a user need to increase their competence level and only displays the instructions in the vehicle if the existence of such a need is recognized. User behavior in the vehicle can be captured in a variety of ways, which will be discussed in more detail later. The aforementioned behavioral information can be generated from this. The method according to the invention is based on the analysis of this behavioral information by a suitably trained generative AI model. User behavior can be described in a variety of ways, so that so-called multimodal generative AI models are preferably used. Various known multimodal generative AI models are generally suitable for this purpose; reference is made to: A survey of multimodal deep generative models, Masahiro Suzuki et al.,Department of Technology Management for Innovation, The University of Tokyo, Tokyo, Japan, arXiv:2207.02127v1 [cs.LG] 5 Jul 2022, https: / / arxiv.org / abs / 2207.02127v1.,

[0013] Thanks to appropriate pre-training, AI models are reliably able to calculate appropriate target variables from a wide variety of input data.

[0014] The computing unit can be integrated into the vehicle or implemented externally. For example, it could be a central on-board computer, the control unit of a vehicle subsystem, a telematics unit, or similar device. It could also be a server or server network, such as a high-performance computing cluster. This allows the vehicle to communicate with the server via the internet. For this purpose, the vehicle can be connected to the internet via the aforementioned telematics unit, for example, using a mobile network. In this way, behavioral information can be generated in the vehicle, transmitted to the server via the internet, and evaluated there by the generative AI model. The competence level determined by the AI ​​model is then transmitted back to the vehicle.Using an external computing unit in the vehicle has the advantage of allowing the use of high-performance hardware components, which in turn enables the use of complex AI models while simultaneously providing results relatively quickly. Conversely, using an internal computing unit in the vehicle has the advantage that no communication with external components is necessary to perform the procedure. This allows the competency level to be determined even when the vehicle is, for example, in an area with no mobile network coverage. Furthermore, this reduces the risk of data leakage during a cyberattack.

[0015] The competence level represents or reflects the user's mental model. In its simplest form, the competence level can be described as a number within a defined range. For example, the competence level could range from 0 to 10 or 0 to 100. Thus, the competence level can be described as a percentage. For instance, 0% represents an inexperienced user, and 100% represents an experienced user who knows all functionalities. However, the competence level, or rather the mental model, can also be described in more detail, which will be discussed later.

[0016] As previously described, the vehicle displays instructions for using its functions, tailored to the user's skill level. This gradually trains the user to improve their competence in using the vehicle's functions. The instructions are adapted to the user's current skill level, enabling a particularly targeted increase in competence. In this way, the user learns the relevant vehicle functions step by step. This process avoids overwhelming or under-challenging the user.

[0017] Furthermore, additional parameters can be taken into account to adapt the instructions, particularly parameters describing external conditions. These include, for example, information such as whether the vehicle is currently moving, the prevailing weather conditions, the current traffic volume, and similar factors. For instance, if it is detected that the user is engaged in driving, corresponding instructions can be delivered via a purely acoustic transmission channel instead of a visual one.

[0018] The generative AI model processes behavioral information to determine a user's need to improve their skill level and only displays instructions in the vehicle when such a need is detected. Often, users ignore or actively hide these instructions. For example, it's common practice to display a tutorial when launching an application or app for the first time. This increases the time required for the user to access the desired function, negatively impacting the user experience. By only displaying instructions when the user's willingness to improve their skill level is recognized, such negative user experiences can be avoided.

[0019] The generative AI model, by processing behavioral information, is capable of recognizing relevant situations. For example, it can detect that a user is actively accessing and reading instructions or descriptions for specific vehicle functions via a graphical user interface. A tutorial or instructional video can then be automatically displayed on a screen. Alternatively, training on how to use the vehicle function could be conducted within the vehicle itself.

[0020] For example, the various functions can be described to the user visually and / or audibly, after which the user, especially in a quiet situation, operates the respective vehicle function. Because the user actually operates the respective vehicle function, they can remember the operating procedure better than if they were only informed about it in writing.

[0021] Various methods are possible for the AI ​​model to recognize such a need. For example, the user might make a comment like, "Oh man, how does this work?" while operating the device, whereupon appropriate instructions are provided.

[0022] An advantageous further development of the method according to the invention provides that, to capture user behavior, the user is detected by sensors in the vehicle, the activation of an actuator in the vehicle by the user is tracked, and / or the user's interaction with a user interface is tracked. Various sensors can be present in the vehicle that allow for the corresponding detection of user behavior. For example, the user can be detected visually and / or the vehicle interior can be scanned to determine the position and orientation of the user's limbs. Cameras are particularly useful for visually detecting the user. Depth information can also be determined using radar sensors, ultrasonic sensors, radio antennas, and the like.For example, multiple radio antennas, such as Wi-Fi antennas, can be present in or on the vehicle. By analyzing the radio signal strength measured by each antenna, movements inside the vehicle can be detected. This allows for such precise motion detection that even the breathing rate and heartbeat of vehicle occupants can be recorded. Infrared cameras can also be used to measure the skin temperature of the vehicle occupants. Visually tracking users allows their gaze direction to be monitored, emotional states to be recognized (such as an uncertain or questioning look), pupil size and changes in pupil size to be observed, and so on. For example, the pupil may dilate or constrict when the user is mentally focused on a specific task or is startled.By evaluating the direction of gaze, the AI ​​model can estimate which activity the respective user is currently engaged in within the vehicle. Looking at a display device showing a graphical user interface can indicate that the user is reading information from the display. Looking through the windshield, on the other hand, can indicate that the user is concentrating on driving. Furthermore, if a user is dissatisfied with a user interface concept, they may become emotionally agitated. For example, the user might gesticulate angrily, shout, or use profanity. Vital data can also be recorded, allowing for the detection of emotional changes, such as an increase in heart rate.This can be achieved using sensors both within the vehicle, such as camera-based detectors for recognizing changes in facial color or pressure sensors in the steering wheel, and wearables worn by the user, such as a smartwatch. By capturing depth information within the vehicle interior, this behavior can be recognized and used to assess the user's competence level. Specifically, this involves establishing a connection between the user's behavior and a control input received via a corresponding control channel for operating a vehicle function. Furthermore, it is important to consider that the recognized behavior, and thus the derived competence level, must be viewed relative to the system's functionality, as a poorly designed user experience (UX) can negatively impact the competence level of all users.

[0023] Furthermore, the acoustic environment inside the vehicle can be recorded using acoustic sensors such as microphones. This allows for the detection of acoustic reactions such as shouting or the use of expletives. By employing established methods of computational linguistics, the semantic content of the user's speech can also be analyzed. This makes it possible to identify positive, neutral, and negative user experiences when operating the corresponding vehicle function based on the spoken words. For example, an inexperienced user might utter a phrase like "Aha" or "Finally!" when navigating a vehicle function through a submenu, whereas an experienced user would remain silent.

[0024] Furthermore, it's possible to track how the user interacts with the vehicle's actuators. This includes, for example, how the user operates the steering wheel, accelerator pedal, brake pedal, and so on. For instance, an inexperienced user might not be aware that fully depressing the accelerator pedal in an automatic transmission initiates a so-called "kickdown." An inexperienced user can be inferred if they abruptly release their foot from the accelerator pedal after executing a kickdown.

[0025] It is also possible to track how the user interacts with these user interfaces. This includes, for example, operating vehicle functions via the graphical user interface, as well as using manual switches, buttons, rotary push-buttons, and the like. The frequency and duration of such function usage can indicate the user's competence level. It is also possible to determine which interaction channel each user prefers for interacting with the vehicle. For instance, various vehicle functions can be controlled simultaneously via multiple channels, such as voice commands, physical buttons, and graphical controls within a graphical user interface.

[0026] According to a further advantageous embodiment of the method according to the invention, the implementation of the vehicle function by the vehicle is designed depending on the competence level. Thus, the vehicle function is adapted to the specific situation, taking into account the competence level within the vehicle. This allows for a significant improvement in the user experience. The changes particularly affect the range of functions, the output of information, and / or the interaction possibilities. For example, the graphical user interface can be adapted. Graphical controls can be rearranged, provided with more or fewer explanatory texts, explanatory animations can be displayed, the level of detail for operation can be adjusted, individual controls can be moved between different submenus, in particular, moved to a main menu, and so on.Furthermore, the available range of functions can be increased or decreased. The degree of automation used to control individual functions or sub-functions can be adjusted. This allows, in particular, an experienced user to increase the manual functionality and, for a less experienced user, to automate more sub-functions.

[0027] The vehicle's functions are thus designed to be simpler for users with a lower skill level and more complex for users with a higher skill level. This ensures that the user always feels addressed on an equal footing, preventing beginners from being overwhelmed and professionals from feeling incompetent by the vehicle.

[0028] According to a further advantageous embodiment of the method according to the invention, the competence level is described by a multidimensional vector space, wherein the generative AI model determines the values ​​of the vector's components. The individual components of the vector represent various user skills. For example, a first component could describe the user's knowledge of the available subfunctions of a function. A second component of the vector could describe the user's knowledge of which user interface can be used to operate which subfunction. A third component of the vector could describe the user's execution speed of operations via corresponding user channels, and so on. This allows for a particularly differentiated specification of the competence level. In turn, corresponding behavioral recommendations can be issued in a more targeted manner.The user can be assisted in using vehicle functions in a more targeted manner. For example, if it is recognized that the user can operate the respective function relatively quickly via a first control channel in one situation, but requires a longer time in a second situation via the same control channel, the user can be made aware that the respective function can be controlled via a second control channel, which enables faster operation in the second situation. A further advantageous embodiment of the method according to the invention provides that the competence level for an individual user is determined multiple times over time. This makes it possible to record changes in the competence level and thereby recognize learning progress or a decline in the user's learning.This can be used to make learning measures for the user even more targeted. For example, if a particular user learns more slowly than another user, corresponding instructions for using vehicle functions can be simplified or repeated more frequently to ensure continuous learning progress.

[0029] According to a further advantageous embodiment of the method according to the invention, it is further provided that the method according to the invention is executed in the vehicle after the implementation of a new or updated vehicle function. For example, vehicle functions can be modified by installing different hardware components and / or installing modified program code sections. A software update can thus be introduced into the vehicle, particularly wirelessly. This is also referred to as an over-the-air (OTA) update. This can be used to change the functionality or behavior of vehicle functions. By executing the method according to the invention, it can be tested to what extent the user is familiar with the modified or newly introduced function, which in turn allows targeted information to be output to the user regarding the respective modified or new vehicle function.

[0030] In a vehicle comprising sensing means for recording user behavior and a processing unit for processing information generated by the sensing means, the sensing means and the processing unit are configured, according to the invention, to execute a method described above. The vehicle can be any road vehicle such as a car, truck, van, bus, or the like. However, it could also be a rail vehicle, watercraft, aircraft, or spacecraft. Suitable sensing means include the aforementioned sensors, actuators, and user interfaces. The information generated by the sensing means is then the aforementioned behavioral information. The processing unit serves to process the behavioral information using the generative AI model.In this case, the computing unit could also be a communication module through which said communication with a server executing the AI ​​model is possible. The computing unit has at least read access to a computer-readable storage medium containing machine-interpretable instructions which, when executed by a processor of the computing unit, cause it to provide the corresponding method according to the invention.

[0031] Further advantageous embodiments of the inventive method for determining the competence level and the vehicle also result from the exemplary embodiments, which are described in more detail below with reference to the figures.

[0032] This shows:

[0033] Fig. 1 shows a schematic representation of the graphical user interface displayed on a display device in a vehicle;

[0034] Fig. 2 is a schematic representation of a vehicle according to the invention; and Fig. 3 is a representation of a competence level determined by the vehicle of a

[0035] User's.

[0036] As in other technical fields, users can be more or less familiar with the operation of vehicle functions. Some users have very little information about the individual functions in the vehicle, or even no knowledge of them at all. This goes so far that some users even shy away from operating vehicle functions for fear of unintentionally changing an otherwise functioning system. Other users, however, are very knowledgeable and enjoy delving deeply into the underlying functionalities.

[0037] This will be illustrated using the graphical user interface shown in Figure 1, which is an example of a touch-sensitive display device in a vehicle. Modern vehicles have a wide range of functionalities, and various pieces of information can be displayed on a corresponding display device. Furthermore, settings can be changed, and vehicle functions can be operated both via touch input and using physical controls. The example shown in Figure 1 is intentionally overloaded. This illustrates the cognitive overload experienced by someone unfamiliar with vehicle functions.

[0038] To enable both technically skilled and technically unskilled individuals to have a positive user experience with the respective vehicle functions, a computer-implemented method according to the invention is carried out in a vehicle 3 according to the invention, as shown in Figure 2, to determine the competence level 1 of a user 2 in using a vehicle function. The aim is to automatically determine the competence level 1 of the user 2, which in turn allows the provision of the respective vehicle function to be designed depending on the competence level 1. For example, the menu navigation of a submenu on the graphical user interface can be designed to be more complex for a skilled user and simpler for an inexperienced user.

[0039] Vehicle 3 includes suitable data acquisition devices to record the user behavior of user 2. These include, for example, sensors such as an interior camera 7 or a microphone 8. Furthermore, user 2's interaction with vehicle 3 can be recorded via an actuator, such as the accelerator pedal 9, or the aforementioned graphical user interface displayed on a screen 10. The corresponding data, in the form of behavioral information, is fed to a processing unit 4. For example, the processing unit 4 can be integrated into vehicle 3. A generative AI model is executed on the processing unit 4, which is capable of determining the aforementioned competence level 1 by processing the behavioral information. Instead of an internal vehicle processing unit 4, an external vehicle processing unit 4, which can also be referred to as a cloud server, could also be used.The vehicle 3 can communicate wirelessly with such an external computing unit 4, for example via a mobile internet connection. For this purpose, the vehicle 3 can be equipped with a telecommunications unit 11.

[0040] Figure 3 shows examples of different implementations of competence level 1. As shown in the left half of Figure 3, competence level 1 is represented by a number within a defined range, for example, between 0 and 100. Competence level 1 depends on the parameters P, t, and A. Here, the parameter P represents the individual user 2, and t represents the time at which the competence level is reached.

[0041] 1 was determined, and A for the underlying vehicle function.

[0042] It is also possible that competence level 1 is described by a multidimensional vector space, with Figure 3 showing a corresponding vector 6 in the right half. The individual components 5 of vector 6 represent individual aspects of user 2's ability in the context of operating vehicle functions. For example, one of the components 5 could represent user 2's general knowledge of which functions are usable and to what extent. Another component 5 of vector 6 could describe user 2's reaction speed during operation. As an example, the respective components 5 also have values ​​in the range of 0 to 100.

[0043] The advantages of the inventive method and the inventive vehicle become clear with reference to the following application examples. For example, a user could

[0044] User 2 has purchased vehicle 3 as a new car. Vehicle 3 is an electric vehicle. User 2 now wants to perform a charging process for the first time. To do this, User 2 could interact with vehicle 3 via the display device 10 and / or issue a voice command such as: "How can I charge now?". Based on this voice command and the seemingly aimless operation of the display device 10, the generative AI model concludes that User 2 is inexperienced with the charging process. As a response, for example, the full functionality of a graphical user interface displayed on the display device 10 can be withheld to avoid overwhelming or confusing User 2. Only after several successful interactions and the corresponding recognition of an increased competence level 1 are further sub-functions unlocked.As part of this, information can also be displayed upon request from user 2. The level of automation of vehicle functions can also be automatically adjusted. For example, if user 2 performs charging manually and it is observed that charging is always at full power, fully charging the traction battery of vehicle 3, then the respective charging processes can be automated through controlled charging. For instance, the charging power can be automatically regulated to extend the lifespan of the traction battery. In another application example, a curious user 2 explores a corresponding menu item in the graphical user interface related to intelligent charging. The distinctive interaction pattern reveals user 2's readiness to learn through the generative AI model. User 2 is then proactively supported in improving and increasing their competence level 1.For this purpose, information relevant to charging can be output to user 2 via various transmission methods. For example, a corresponding tutorial can be displayed on the display device 10. If, however, the interaction takes place while driving, an acoustic information output could be used instead of a visual one.

[0045] Using the method according to the invention, a vehicle 3 according to the invention is able to provide interaction options adapted to the respective competence level 1 of the users 2. This improves the personal user experience and also increases road safety. Users 2 can thus operate the respective vehicle function more effectively. This ensures that the desired effect is achieved more easily and quickly. Furthermore, frustration, which can negatively impact driving, is avoided. Potentially dangerous traffic situations can thus be avoided or at least the risk of such situations arising reduced. The competence level 1 of the respective users 2 can also be improved over time. This results in users 2 becoming increasingly familiar with the respective vehicle functions.

Claims

Mercedes-Benz Group AG Patent claims 1. A computer-implemented method for determining a user's (2) competence level (1) in using a vehicle function, wherein the user's (2) behavior in the vehicle (3) is recorded and a computing unit (4) automatically determines the competence level (1) depending on the observed behavior, characterized in that behavioral information describing the user's behavior is fed to a generative AI model executed by the computing unit (4), which is trained to determine competence levels (1) for vehicle functions by processing corresponding behavioral information, and the generative AI model determines the competence level (1), wherein instructions for using the vehicle function adapted to the user's (2) competence level (1) are displayed in the vehicle (3).and wherein the generative AI model, by processing the behavioral information, determines a user's need (2) to increase their competence level (1) and only outputs the instructions in the vehicle (3) if the existence of a respective need on the part of the user (2) is recognized.

2. Method according to claim 1, characterized in that, in order to capture the user behavior, the user (2) in the vehicle (3) is sensorially detected, an actuation of an actuator of the vehicle (3) by the user (2) is tracked and / or an interaction of the user (2) with an operating interface is tracked.

3. Method according to claim 1 or 2, characterized in that the implementation of the vehicle function by the vehicle (3) is designed depending on the competence level (1).

4. Method according to claim 3, characterized in that the vehicle function is designed to be simpler for a comparatively low level of competence (1) and to be more complex for a comparatively high level of competence (1).

5. Method according to one of claims 1 to 4, characterized in that the competence level (1) is described by a multidimensional vector space, wherein the generative AI model determines the characteristics of the components (5) of the vector (6).

6. Method according to one of claims 1 to 5, characterized in that the competence level (1) for an individual user (2) is determined multiple times over time.

7. Method according to any one of claims 1 to 6, characterized by an embodiment according to the implementation of a new or updated vehicle function in the vehicle (3).

8. Vehicle (3), comprising recording means for recording user behavior and a computing unit (4) for processing information generated by the recording means, characterized in that the recording means and the computing unit (4) are configured to perform a method according to one of claims 1 to 7.

Images