Computer-implemented method for determining the competence level of a user and vehicle

Abstract

The invention relates to a computer-implemented method for determining a user's (2) competence level (1) in using a vehicle function, wherein the user's (2) behavior in the vehicle (3) is recorded and a computing unit (4) automatically determines the competence level (1) based on the observed behavior. The computer-implemented method according to the invention is characterized in that behavioral information describing the user's behavior is fed to a generative AI model executed by the computing unit (4), which is trained to determine competence levels (1) for vehicle functions by processing corresponding behavioral information, and the generative AI model determines the competence level (1).

Description

[0001] The invention relates to a computer-implemented method for determining a user's competence level in the use of a vehicle function according to the type defined in more detail in the preamble of claim 1, and to a vehicle for carrying out the method.

[0002] The user experience (UX) of devices, equipment, machines, applications, software, and the like depends on the prior knowledge and experience of the individual user. Experienced users possess a comparatively extensive understanding of the available functions and their scope, enabling them to use the device or application more effectively, quickly, and efficiently. Inexperienced users, on the other hand, are not fully aware of the available functionality, which can lead to some functions going unused or even resulting in incorrect operation.

[0003] For example, programs or apps can be operated quickly, efficiently, and intuitively if the user enters corresponding keyboard shortcuts or predefined gestures via a touch-sensitive display. This eliminates the need to navigate through extensive menus or submenus, which is time-consuming and places a high cognitive load on the user. A user's knowledge or skills in using a particular device or application are also referred to as a "mental model." Such mental models thus reflect the user's respective level of competence. The mental model describes the user's knowledge and understanding of the application.

[0004] To ensure a satisfying user experience, operating concepts should be tailored to the user's respective skill level, i.e., their mental model. It is particularly important not to overwhelm inexperienced users, while providing more advanced operating options for experienced users. By providing targeted guidance, users can gradually learn the system's behavior and interactions, gaining a deeper understanding of how to operate each function. This approach to designing operating concepts is especially relevant in the automotive sector, given the large number of complex functions available for use in vehicles.This includes the use of various human-machine interfaces such as physical buttons, knobs, and rotary controls; navigation through graphical user interfaces (GUIs), particularly using touch-sensitive displays; interaction with voice dialog systems via voice commands; and the control of functions via gestures, among other things. Furthermore, information must be processed on various transmission channels, such as visual, acoustic, and haptic information. For example, visual information is displayed on a screen in the vehicle, acoustic information is output via speakers in the vehicle, and haptic information is output via vehicle components touched by the user.Furthermore, the functions operable within the vehicle relate to various areas, such as the vehicle's handling characteristics, particularly longitudinal and lateral acceleration; driver assistance functions, such as adaptive cruise control, high-beam assist, and the like; entertainment functions, such as media playback via the infotainment system; and comfort functions, such as climate control or route guidance via the navigation system. Operating these functions is further complicated by the fact that the driver must simultaneously focus on the road. The driver must not be excessively distracted, as this could increase the risk of accidents.

[0005] Several methods exist for determining a user's competence level when using a function or operating a specific device or application. For example, competence level can be assessed using questionnaires. Alternatively, the user's behavior while using the function can be recorded, allowing conclusions to be drawn about their competence level based on how they utilize the function. For instance, if a user utilizes many different sub-functions, this indicates that they are experienced in using the function.

[0006] German patent DE 10 2006 012 172 A1 discloses a computer-implemented method for the automated comparison of at least one competency topology of a vacant and / or filled position with the competency topology of one or more candidates. The patent discloses the computer-assisted determination of an applicant's competency level for a job opening. The assessment of the competency level is based on the completion of questionnaires. The manner in which the individual completes the questionnaire can be used as an additional parameter for evaluating the competency level. In particular, the order and / or duration of the answers to the respective questions are considered as criteria.

[0007] Furthermore, US 8,731,736 B2 discloses a system and procedure for reducing the risk of losing one's driving skills. The document is based on the premise that a driver gradually loses their driving skills if they do not operate a vehicle or use automated driving functions for an extended period. To assess whether a driver has experienced changes in their driving skills, these changes must first be determined. For this purpose, characteristics are defined that allow conclusions to be drawn about the driver's ability to control the vehicle. Subsequently, the user's behavior in controlling the vehicle is recorded, and the driver's competence level is automatically determined. To evaluate the driver's driving skills, the parameters measured in the vehicle are compared with corresponding parameters of a control subject.The comparison person can be an average driver or even past driving parameters of the driver themselves. Depending on the driver's driving skills or changes in those skills, warnings can be issued or automated vehicle functions can be activated.

[0008] The present invention is based on the objective of providing an improved computer-implemented method for determining the competence level of a user in the use of a vehicle function, which makes it possible to enable improved user interaction and increase road safety.

[0009] According to the invention, this problem is solved by a computer-implemented method for determining a user's competence level in using a vehicle function with the features of claim 1. Advantageous embodiments and further developments, as well as a vehicle for carrying out the method, are described in the dependent claims.

[0010] A generic computer-implemented method for determining a user's competence level in using a vehicle function, wherein the user's behavior in the vehicle is recorded and a computing unit automatically determines the competence level based on the observed behavior, is further developed according to the invention by feeding behavioral information describing the user's behavior into a generative AI model executed by the computing unit. This model is trained to determine competence levels for vehicle functions by processing corresponding behavioral information, and the generative AI model then determines the competence level. User behavior in the vehicle can be recorded in a variety of ways, which will be discussed in more detail later. The aforementioned behavioral information can be generated from this data.The method according to the invention is based on the analysis of this behavioral information by a suitably trained generative AI model. User behavior can be described in a variety of ways, so that so-called multimodal generative AI models are preferably used. Several well-known multimodal generative AI models are generally suitable for this purpose; reference is made to: A survey of multimodal deep generative models, Masahiro Suzuki et al., Department of Technology Management for Innovation, The University of Tokyo, Tokyo, Japan, arXiv:2207.02127v1 [cs.LG] 5 Jul 2022, https: / / arxiv.org / abs / 2207.02127v1.

[0011] Thanks to appropriate pre-training, AI models are reliably able to calculate appropriate target values ​​from a wide variety of input data.

[0012] The computing unit can be integrated into the vehicle or implemented externally. For example, it could be a central on-board computer, the control unit of a vehicle subsystem, a telematics unit, or similar device. It could also be a server or server network, such as a high-performance computing cluster. This allows the vehicle to communicate with the server via the internet. For this purpose, the vehicle can be connected to the internet via the aforementioned telematics unit, for example, using a mobile network. In this way, behavioral information can be generated in the vehicle, transmitted to the server via the internet, and evaluated there by the generative AI model. The competence level determined by the AI ​​model is then transmitted back to the vehicle.Using an external computing unit in the vehicle has the advantage of allowing the use of high-performance hardware components, which in turn enables the use of complex AI models while simultaneously providing results relatively quickly. Conversely, using an internal computing unit in the vehicle has the advantage that no communication with external components is necessary to perform the procedure. This allows the competence level to be determined even if the vehicle is, for example, in an area with no mobile network coverage. Furthermore, this reduces the risk of data leakage during a cyberattack.

[0013] The competence level represents or reflects the user's mental model. In its simplest form, the competence level can be described as a number within a defined range. For example, the competence level can range from 0 to 10 or 0 to 100. Thus, the competence level can be described as a percentage. For instance, 0% represents an inexperienced user, and 100% represents an experienced user who knows all functionalities. However, the competence level, or rather the mental model, can also be described in more detail, which will be discussed later.

[0014] An advantageous further development of the method according to the invention provides that, to capture user behavior, the user is detected by sensors in the vehicle, the activation of an actuator in the vehicle by the user is tracked, and / or the user's interaction with a user interface is tracked. Various sensors can be present in the vehicle that allow for the corresponding detection of user behavior. For example, the user can be detected visually and / or the vehicle interior can be scanned to determine the position and orientation of the user's limbs. Cameras are particularly useful for visually detecting the user. Depth information can also be determined using radar sensors, ultrasonic sensors, radio antennas, and the like.For example, multiple radio antennas, such as Wi-Fi antennas, can be present in or on the vehicle. By analyzing the radio signal strength measured by each antenna, movements inside the vehicle can be detected. This allows for such precise motion detection that even the breathing rate and heartbeat of vehicle occupants can be recorded. Infrared cameras can also be used to measure the skin temperature of the vehicle occupants. Visually tracking users allows their gaze direction to be monitored, emotional states to be recognized (such as an uncertain or questioning look), pupil size and changes in pupil size to be observed, and so on. For example, the pupil may dilate or constrict when the user is mentally focused on a specific task or is startled.By evaluating the direction of gaze, the AI ​​model can estimate what activity the user is currently engaged in while in the vehicle. Looking at a display showing a graphical user interface can indicate that the user is reading information from it. Looking through the windshield, on the other hand, can indicate that the user is concentrating on driving. If a user is dissatisfied with a user interface, they may also become emotionally agitated. For example, they might gesticulate angrily, shout, or use profanity. Vital data can also be recorded, allowing for the detection of emotional changes, such as an increase in heart rate.This can be achieved using sensors both within the vehicle, such as camera-based detectors for recognizing changes in facial color or pressure sensors in the steering wheel, and wearables worn by the user, such as a smartwatch. By capturing depth information within the vehicle interior, this behavior can be recognized and used to assess the user's competence level. Specifically, this involves establishing a connection between the user's behavior and a control input received via a corresponding control channel for operating a vehicle function. Furthermore, it is important to consider that the recognized behavior, and thus the derived competence level, must be viewed relative to the system's functionality, as a poorly designed user experience (UX) can negatively impact the competence level of all users.

[0015] Furthermore, the acoustic environment inside the vehicle can be recorded using acoustic sensors such as microphones. This allows for the detection of acoustic reactions such as shouting or the use of expletives. By employing established methods of computational linguistics, the semantic content of the user's speech can also be analyzed. This makes it possible to identify positive, neutral, and negative user experiences when operating the corresponding vehicle function based on the spoken words. For example, an inexperienced user might utter a phrase like "Aha" or "Finally!" when navigating a vehicle function through a submenu, whereas an experienced user would remain silent.

[0016] Furthermore, it's possible to track how the user interacts with the vehicle's actuators. This includes, for example, how the user operates the steering wheel, accelerator pedal, brake pedal, and so on. For instance, an inexperienced user might not be aware that fully depressing the accelerator pedal in an automatic transmission initiates a so-called "kickdown." An inexperienced user can be inferred if they abruptly release their foot from the accelerator pedal after executing a kickdown.

[0017] It is also possible to track how the user interacts with these user interfaces. This includes, for example, operating vehicle functions via the graphical user interface, as well as using manual switches, buttons, rotary push-buttons, and the like. The frequency and duration of such function usage can indicate the user's competence level. It is also possible to determine which interaction channel each user prefers for interacting with the vehicle. For instance, various vehicle functions can be controlled simultaneously via multiple channels, such as voice commands, physical buttons, and graphical controls within a graphical user interface.

[0018] According to a further advantageous embodiment of the method according to the invention, the implementation of the vehicle function by the vehicle is designed depending on the competence level. Thus, the vehicle function is adapted to the specific situation, taking into account the competence level within the vehicle. This allows for a significant improvement in the user experience. The changes particularly affect the range of functions, the output of information, and / or the interaction possibilities. For example, the graphical user interface can be adapted. Graphical controls can be rearranged, provided with more or fewer explanatory texts, explanatory animations can be displayed, the level of detail for operation can be adjusted, individual controls can be moved between different submenus, in particular, moved to a main menu, and so on.Furthermore, the available range of functions can be increased or decreased. The degree of automation used to control individual functions or sub-functions can be adjusted. This allows, in particular, an experienced user to increase the manual functionality and, for a less experienced user, to automate more sub-functions.

[0019] The vehicle's functions are thus designed to be simpler for users with a lower skill level and more complex for users with a higher skill level. This ensures that the user always feels addressed on an equal footing, preventing beginners from being overwhelmed and professionals from feeling incompetent by the vehicle.

[0020] According to a further advantageous embodiment of the method according to the invention, instructions for using the vehicle's functions are displayed in the vehicle, adapted to the user's skill level. This allows the user to gradually improve their competence in using the vehicle's functions. Taking the current skill level into account, the manner of the instructions is adapted, enabling a particularly targeted increase in the skill level. In this way, the user can gradually learn the relevant vehicle functions. Over- or under-challenging the user is avoided.

[0021] Furthermore, additional parameters can be taken into account to adapt the instructions, particularly parameters describing external conditions. These include, for example, information such as whether the vehicle is currently moving, the prevailing weather conditions, the current traffic volume, and similar factors. For instance, if it is detected that the user is engaged in driving, corresponding instructions can be delivered via a purely acoustic transmission channel instead of a visual one.

[0022] The generative AI model, by processing behavioral information, preferably identifies a user's need to increase their competence level and only displays instructions in the vehicle when such a need is detected. Often, users ignore or actively hide these instructions. For example, it's common practice to display a tutorial when launching an application or app for the first time. This increases the time required for the user to access the desired function, negatively impacting the user experience. By only displaying instructions when the user's willingness to improve their competence level is also recognized, such negative user experiences can be avoided.

[0023] The generative AI model, by processing behavioral information, is capable of recognizing relevant situations. For example, it can detect that a user is actively accessing and reading instructions or descriptions for specific vehicle functions via a graphical user interface. A tutorial or instructional video can then be automatically displayed on a screen. Alternatively, training on how to use the vehicle function could be conducted within the vehicle itself. For instance, the various functions could be described to the user visually and / or audibly, after which the user, particularly in a calm environment, would operate the respective vehicle function.Because the user actually operates the respective vehicle function, he can remember the respective operating method better than if the user were only informed about the corresponding operating method in writing.

[0024] Various methods are possible for the AI ​​model to recognize such a need. For example, the user might make a comment like, "Oh man, how does this work?" while operating the device, whereupon appropriate instructions are provided.

[0025] According to a further advantageous embodiment of the method according to the invention, the competence level is described by a multidimensional vector space, wherein the generative AI model determines the characteristics of the vector's components. The individual components of the vector represent various user skills. For example, a first component could describe the user's knowledge of the available sub-functions of a function. A second component of the vector could describe the user's knowledge of which user interface can be used to operate which sub-function. A third component of the vector could describe the user's execution speed of operations via corresponding user channels, and so on. This allows for a particularly differentiated specification of the competence level. In turn, corresponding behavioral recommendations can be issued in a more targeted manner.The user can be assisted in using vehicle functions in a more targeted manner. For example, if it is recognized that the user can operate the respective function relatively quickly via a first control channel in one situation, but requires a longer time in a second situation via the same control channel, the user can be made aware that the respective function can be controlled via a second control channel, which enables faster operation for the user in the second situation.

[0026] A further advantageous embodiment of the method according to the invention provides that the competence level for an individual user is determined multiple times over time. This makes it possible to record changes in the competence level and thereby identify learning progress or a decline in the user's learning. This can be used to design corresponding learning measures for the user in a more targeted manner. For example, if a particular user learns more slowly than another user, corresponding instructions for using vehicle functions can be designed more simply or repeated more frequently to ensure continuous learning progress.

[0027] According to a further advantageous embodiment of the method according to the invention, it is further provided that the method according to the invention is executed in the vehicle after the implementation of a new or updated vehicle function. For example, vehicle functions can be modified by installing different hardware components and / or installing modified program code sections. A software update can thus be introduced into the vehicle, particularly wirelessly. This is also referred to as an over-the-air (OTA) update. This can be used to change the functionality or behavior of vehicle functions. By executing the method according to the invention, it can be tested to what extent the user is familiar with the modified or newly introduced function, which in turn allows targeted information to be output to the user regarding the respective modified or new vehicle function.

[0028] In a vehicle comprising sensing means for recording user behavior and a processing unit for processing information generated by the sensing means, the sensing means and the processing unit are configured, according to the invention, to execute a method described above. The vehicle can be any road vehicle such as a car, truck, van, bus, or the like. However, it could also be a rail vehicle, watercraft, aircraft, or spacecraft. Suitable sensing means include the aforementioned sensors, actuators, and user interfaces. The information generated by the sensing means is then the aforementioned behavioral information. The processing unit serves to process the behavioral information using the generative AI model.In this case, the computing unit could also be a communication module through which said communication with a server executing the AI ​​model is possible. The computing unit has at least read access to a computer-readable storage medium containing machine-interpretable instructions which, when executed by a processor of the computing unit, cause it to provide the corresponding method according to the invention.

[0029] Further advantageous embodiments of the inventive method for determining the competence level and the vehicle also result from the exemplary embodiments, which are described in more detail below with reference to the figures.

[0030] This shows: Fig. 1 a schematic representation of the graphical user interface displayed on a display device in a vehicle; Fig. 2 a schematic representation of a vehicle according to the invention; and Fig. 3. A representation of a user's competence level as determined by the vehicle.

[0031] As in other technical fields, users can be more or less familiar with the operation of vehicle functions. Some users have very little information about the individual functions in the vehicle, or even no knowledge of them at all. This goes so far that some users even shy away from operating vehicle functions for fear of unintentionally changing an otherwise functioning system. Other users, however, are very knowledgeable and enjoy delving deeply into the underlying functionalities.

[0032] This will be demonstrated using the in Fig. 1. An example graphical user interface is shown on a touch-sensitive display device in the vehicle. Modern vehicles have a wide range of functionalities, and various information can be displayed on a corresponding display device. Settings can also be changed, and vehicle functions can be operated both via touch input and using physical controls. The in Fig. The illustration shown is intentionally overloaded. This demonstrates the cognitive overload experienced by someone unfamiliar with vehicle functions.

[0033] To enable both technically skilled and technically unskilled individuals to have a positive user experience with the respective vehicle functions, a [missing word - likely "user experience"] is implemented in [missing word - likely "in a way"]. Fig. 2 shown in the inventive vehicle 3 a computer-implemented method according to the inventive for determining a in Fig. The three demonstrated competency levels 1 of user 2 in the use of a vehicle function are implemented. The goal is to automatically determine the competency level 1 of user 2, which in turn allows the provision of the respective vehicle function to be designed depending on the competency level 1. For example, the menu navigation of a submenu on the graphical user interface can be made more complex for an experienced user and simpler for an inexperienced user.

[0034] Vehicle 3 includes suitable data acquisition devices to record the user behavior of user 2. These include, for example, sensors such as an interior camera 7 or a microphone 8. Furthermore, user 2's interaction with vehicle 3 can be recorded via an actuator, such as the accelerator pedal 9, or the aforementioned graphical user interface displayed on a screen 10. The corresponding data, in the form of behavioral information, is fed to a processing unit 4. For example, the processing unit 4 can be integrated into vehicle 3. A generative AI model is executed on the processing unit 4, which is capable of determining the aforementioned competence level 1 by processing the behavioral information. Instead of an internal vehicle processing unit 4, an external vehicle processing unit 4, which can also be referred to as a cloud server, could also be used.The vehicle 3 can communicate wirelessly with such an external computing unit 4, for example via a mobile internet connection. For this purpose, the vehicle 3 can be equipped with a telecommunications unit 11.

[0035] Fig. Figure 3 shows examples of different implementations of competence level 1. As in Fig. As shown in Figure 3 on the left half, competence level 1 is represented by a number within a defined range, for example, between 0 and 100. Competence level 1 depends on the parameters P, t, and A. Here, parameter P represents the individual user 2, t the time at which competence level 1 was determined, and A the underlying vehicle function.

[0036] It is also possible that competence level 1 is described by a multidimensional vector space, where Fig.Figure 3 in the right half shows a corresponding vector 6. The individual components 5 of vector 6 represent different aspects of user 2's ability in the context of operating vehicle functions. For example, one component 5 could represent user 2's general knowledge of which functions are usable and to what extent. Another component 5 of vector 6 could describe user 2's reaction speed during operation. As an example, each component 5 also has a value in the range of 0 to 100.

[0037] The advantages of the inventive method and the inventive vehicle become clear with the following application examples. For instance, a user 2 might have purchased the vehicle 3 as a new car. The vehicle 3 is an electric vehicle. The user 2 now wants to perform a charging process for the first time. To do this, the user 2 could interact with the vehicle 3 via the display device 10 and / or issue a voice command such as: "How can I charge now?". Based on this voice command and the seemingly aimless operation of the display device 10, the generative AI model concludes that the user 2 is inexperienced with the charging process. As a response, for example, the full functionality of a graphical user interface displayed on the display device 10 can be withheld to avoid overwhelming or confusing the user 2.Only after several successful interactions and the corresponding recognition of an increased competence level 1 are further sub-functions unlocked. Sub-functions can also be displayed upon request from the user. The degree of automation of vehicle functions can also be automatically adjusted. For example, if user 2 manually performs charging processes and it is observed that charging is always at full power, fully charging the vehicle's traction battery 3, then the respective charging processes can be automated through controlled charging. For instance, the charging power can be automatically regulated to increase the lifespan of the traction battery.

[0038] In another application example, a curious user 2 explores a relevant menu item in the graphical user interface on the topic of intelligent charging. The distinctive interaction pattern allows the generative AI model to recognize user 2's readiness to learn. User 2 is then proactively supported in improving and increasing their competence level 1. To this end, relevant information about charging can be delivered to user 2 via various transmission channels. For example, a corresponding tutorial can be displayed on the display device 10. If, however, the interaction takes place while driving, an auditory information output could be used instead of a visual one.

[0039] Using the method according to the invention, a vehicle 3 according to the invention is able to provide interaction options adapted to the respective competence level 1 of the users 2. This improves the personal user experience and also increases road safety. Users 2 can thus operate the respective vehicle function more effectively. This ensures that the desired effect is achieved more easily and quickly. Furthermore, frustration, which can negatively impact driving, is avoided. Potentially dangerous traffic situations can thus be avoided or at least the risk of such situations arising reduced. The competence level 1 of the respective users 2 can also be improved over time. This results in users 2 becoming increasingly familiar with the respective vehicle functions. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2006 012 172 A1

[0006] US 8,731,736 B2

[0007] Cited non-patent literature

[0000] A survey of multimodal deep generative models, Masahiro Suzuki et al., Department of Technology Management for Innovation, The University of Tokyo, Tokyo, Japan, arXiv:2207.02127v1 [cs.LG] 5 Jul 2022, https: / / arxiv.org / abs / 2207.02127v1

[0010]

Claims

[1] Computer-implemented method for determining a user's (2) competence level in the use of a vehicle function, wherein the user's (2) behavior in the vehicle (3) is recorded and a computing unit (4) automatically determines the competence level (1) depending on the observed behavior, characterized by , that user behavior-descriptive behavioral information is fed to a generative AI model executed by the computing unit (4), which is trained to determine competence level (1) for vehicle functions by processing corresponding behavioral information, and the generative AI model determines the competence level (1). [2] Method according to claim 1, characterized by, that in order to capture user behavior the user (2) in the vehicle (3) is sensorially detected, an actuation of an actuator of the vehicle (3) by the user (2) is tracked and / or an interaction of the user (2) with an operating interface is tracked. [3] Method according to claim 1 or 2, characterized by , that the implementation of the vehicle function by the vehicle (3) is designed depending on the competence level (1). [4] Method according to claim 3, characterized by , that the vehicle function is designed to be simpler for a comparatively low level of competence (1) and more complex for a comparatively high level of competence (1). [5] Method according to any one of claims 1 to 4, characterized by , that the vehicle (3) provides instructions for using the vehicle function adapted to the competence level (1) of the user (2). [6] Method according to claim 5, characterized by, that the generative AI model, by processing the behavioral information, determines a need of the user (2) to increase his / her competence level (1) and only outputs the instructions in the vehicle (3) if the existence of a respective need on the part of the user (2) is recognized. [7] Method according to any one of claims 1 to 6, characterized by , that the competence level (1) is described by a multidimensional vector space, where the generative Kl model determines the form of the components (5) of the vector (6). [8] Method according to any one of claims 1 to 7, characterized by , that the competence level (1) for an individual user (2) is determined multiple times over time. [9] Method according to any one of claims 1 to 8, characterized by an execution following the implementation of a new or updated vehicle function in the vehicle (3). [10] Vehicle (3) comprising a data acquisition device for recording user behavior and a computing unit (4) for processing information generated by the data acquisition device, characterized by , that the recording means and the computing unit (4) are set up to carry out a method according to one of claims 1 to 9.

Images