Method for operating a digital assistant of a vehicle, computer-readable medium, system, and vehicle

Abstract

The invention relates to a method for operating a digital assistant of a vehicle, said method having the steps of: determining a trigger event of the digital assistant of the vehicle by means of the vehicle; determining at least one personal preference for a plurality of or each occupant of the vehicle on the basis of the trigger event of the digital assistant by means of the vehicle; transmitting the determined personal preferences of the plurality of occupants of the vehicle, as context information, from the vehicle to a first application interface of a large language model; transmitting a command line comprising the determined trigger event as input data from the vehicle to a second application interface of the large language model; receiving, by means of the vehicle, a response message of the large language model on the basis of the transmitted personal preferences of the plurality of occupants of the vehicle, as context information, and the transmitted command line, which is representative of the trigger event; and providing the response message of the large language model to the plurality of occupants of the vehicle via an output interface of the digital assistant of the vehicle and / or to a function of the vehicle via an application interface.

Description

[0001] Method for operating a digital assistant of a vehicle, computer-readable medium, system, vehicle

[0002] The invention relates to a method for operating a digital assistant of a vehicle. The invention further relates to a computer-readable medium for operating a digital assistant of a vehicle, a system for operating a digital assistant of a vehicle, and a vehicle comprising the system for operating a digital assistant of a vehicle.

[0003] Large language models (LLMs for short) are known from the prior art and can be provided with context information. The context information can usually be passed directly to the large language model in a request to the large language model. It is also known from the prior art to use a large language model for a vehicle's digital assistant. The vehicle's digital assistant can be adapted to a driver of the vehicle. Other vehicle occupants are not taken into account by the vehicle's digital assistant with regard to their individual preferences. This can lead to the outputs of the vehicle's digital assistant and / or the control of vehicle functions by the vehicle's digital assistant not being helpful for all vehicle occupants.

[0004] It is therefore an object of the invention to operate a digital assistant of a vehicle more efficiently. In particular, it is an object of the invention to efficiently adapt a response message of a large language model of a digital assistant of a vehicle to multiple occupants of a vehicle and / or to provide it to multiple occupants of a vehicle. In particular, a further object of the invention is to control the dialogue guidance of a large language model of a digital assistant of the vehicle depending on personal information of multiple occupants of a vehicle.

[0005] This object is achieved by the features of the independent claims. Advantageous embodiments and further developments of the invention emerge from the dependent claims. According to a first aspect, the invention is characterized by a method for operating a digital assistant of a vehicle. The method can be a computer-implemented method and / or a control unit-implemented method. The vehicle can be a motor vehicle. The method comprises determining a triggering event of the digital assistant of the vehicle by the vehicle. The triggering event can be triggered by a voice input and / or an operating interaction of a vehicle occupant. Additionally or alternatively, the triggering event can be triggered proactively by the digital assistant depending on one or more vehicle-specific conditions.

[0006] The method comprises determining at least one personal preference for each of the multiple occupants of the vehicle depending on the triggering event of the digital assistant by the vehicle. Preferably, the method can determine at least one personal preference for each detected occupant of the multiple occupants. A detected occupant is preferably a uniquely identified occupant of the vehicle. An occupant can be uniquely identified, for example, by an interior camera and / or by a mobile device coupled to the vehicle.

[0007] The method further comprises transmitting the determined personal preferences of the multiple occupants of the vehicle as context information from the vehicle to a first application interface of a large language model. The large language model can be a large language model of the vehicle's digital assistant. The method further comprises transmitting a command line prompt comprising the determined trigger event as input data from the vehicle to a second application interface of the large language model.Finally, the method comprises receiving a response message from the large language model depending on the transmitted personal preferences of the multiple vehicle occupants as context information and the transmitted command line representative of the triggering event by the vehicle, and providing the response message from the large language model to the multiple vehicle occupants via an output interface of the vehicle's digital assistant. The response message from the large language model can be a message of a dialogue between one or more vehicle occupants and the vehicle's digital assistant. Additionally or alternatively, the response message from the large language model can include an instruction to execute a function of the vehicle.The instruction of the response message can be generated by a large language model depending on the transmitted personal preferences of the vehicle occupants as context information. The vehicle's digital assistant can execute the vehicle's function depending on the instruction of the response message of the large language model. For example, if the vehicle's function is a setting function for a vehicle's seat heater, the instruction can include a configuration of the vehicle's seat heater, and the digital assistant can adjust the seat heater for each of the vehicle's occupants depending on the instruction of the response message. For example, if the vehicle's function is a route planning function of the vehicle, the instruction can include multi-stage route planning, and the vehicle's digital assistant can adapt the vehicle's route planning function to the multi-stage route planning of the instruction in the response message.Advantageously, the digital assistant can utilize different personal preferences of multiple vehicle occupants to provide a response message to the vehicle occupants. A dialogue between the digital assistant and the vehicle occupants can thus be more precisely tailored to the personal preferences of the multiple vehicle occupants regarding the trigger event.

[0008] According to an advantageous embodiment, the triggering event of the digital assistant can be determined depending on several vehicle-state-dependent triggering conditions. This allows a triggering event for a proactive dialogue between the vehicle's digital assistant and the vehicle's occupants to be controlled more efficiently.

[0009] According to a further advantageous embodiment, at least one vehicle-state-dependent triggering condition of the plurality of vehicle-state-dependent triggering conditions can be a vehicle-state-dependent triggering condition of a vehicle-external server, and / or at least one vehicle-state-dependent triggering condition of the plurality of vehicle-state-dependent triggering conditions can be a vehicle-state-dependent triggering condition of an in-vehicle component. This allows a triggering event for a proactive dialogue between the vehicle's digital assistant and the vehicle's occupants to be controlled more efficiently.

[0010] According to a further advantageous embodiment, the method can further comprise aggregating the determined personal preferences of the multiple occupants of the vehicle to form an aggregated personal preference of the multiple occupants, transmitting the aggregated personal preference of the multiple occupants of the vehicle as context information from the vehicle to the first application interface of a large language model, and receiving the response message of the large language model depending on the transmitted, aggregated, personal preference of the multiple occupants of the vehicle as context information and the transmitted command line representative of the triggering event by the vehicle. This allows the context information of the large language model to be adapted more efficiently. By aggregating, less data regarding the personal preferences of the vehicle occupants can be transmitted to the large language model.This can result in a response message using context information from the large language model being received more quickly and delivered to the vehicle's occupants by the digital assistant.

[0011] According to a further advantageous embodiment, aggregating the determined personal preferences of the multiple occupants of the vehicle into an aggregated personal preference of the multiple occupants can comprise combining identical personal preferences of the multiple occupants into a combined personal preference for all occupants of the vehicle, and / or resolving conflicting personal preferences of the multiple occupants into a uniform personal preference for all occupants using one or more predefined rules. By combining and / or resolving personal preferences, less data regarding the personal preferences of the vehicle occupants can be transmitted to the large language model.This can result in a response message using context information from the large language model being received more quickly and delivered to the vehicle's occupants by the digital assistant.

[0012] According to a further advantageous embodiment, the method can additionally transmit the determined personal preferences of the multiple vehicle occupants and / or the determined aggregated personal preferences of the multiple vehicle occupants as input data with the command line representative of the determined triggering event from the vehicle to a second application interface of a large language model. This allows the large language model to generate a more precise response message using the context information and transmit it to the vehicle. The vehicle's digital assistant can provide a response message that is more precisely based on the transmitted context information.

[0013] According to a further advantageous embodiment, the method can further include determining a situational context of the multiple occupants of the vehicle based on camera data from an interior camera, an occupant model, an environment model, driving information, and / or navigation information, and transmitting the determined situational context of the multiple occupants of the vehicle as context information from the vehicle to the first application interface of the large language model. By using additional context information, the large language model can take a vehicle-specific situation into account when generating the response message.

[0014] According to a further advantageous embodiment, the method may further comprise determining a priority of the response message of the large language model depending on the determined situational context and / or an aggregated, personal output preference of the multiple vehicle occupants, and providing the response message of the large language model to the multiple vehicle occupants via the output interface of the vehicle's digital assistant depending on the determined priority. This allows the vehicle's digital assistant to efficiently control the provision of the response message of the large language model.

[0015] According to a further advantageous embodiment, the first application interface can be a configuration interface of the large language model, and / or the second application interface can be a text-based speech input interface of the large language model. By combining text-based and data-based input interfaces of the language model, a response message for the triggering event can be efficiently generated, taking the context information into account. Converting the data-based context information into text-based context information can be avoided.

[0016] According to a further aspect, the invention is characterized by a computer-readable medium for operating a digital assistant of a vehicle, wherein the computer-readable medium comprises instructions that, when executed on a computer and / or a control unit, carry out the method described above. According to a further aspect, the invention is characterized by a system for operating a digital assistant of a vehicle, wherein the system is configured to carry out the method described above.

[0017] According to a further aspect, the invention is characterized by a vehicle comprising the above-described system for operating a digital assistant of a vehicle.

[0018] Further features of the invention emerge from the claims, the figures, and the description of the figures. All features and combinations of features mentioned above in the description, as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the figures, can be used not only in the respective specified combination, but also in other combinations or even on their own.

[0019] A preferred embodiment of the invention is described below with reference to the accompanying drawings. Further details, preferred embodiments, and developments of the invention will emerge from these. In detail, schematically

[0020] Fig. 1 shows an exemplary method for operating a digital assistant of a vehicle, and

[0021] Fig. 2 shows an application example of the method for operating a digital assistant of a vehicle.

[0022] In detail, Fig. 1 shows an exemplary method 100 for operating a digital assistant of a vehicle. When operating a digital assistant of a vehicle, it is often not sufficient to consider only the personal preferences of a driver of the vehicle, but also the personal preferences of all vehicle occupants. Furthermore, it is necessary to communicate the personal preferences of the vehicle occupants, in particular all vehicle occupants, to a large language model, which the vehicle's digital assistant can use to create and / or conduct dialogues, preferably proactive dialogues, with the vehicle occupants.

[0023] The method 100 may determine 102 a trigger event of the vehicle's digital assistant by the vehicle. The trigger event may be linked to a function of the vehicle that is performed by the vehicle's digital assistant. For example, the vehicle's function may be a dialog-based recommendation regarding a vehicle function and / or a configuration of a vehicle function. For example, the vehicle's function may provide information about a navigation route and / or adapt a navigation route.

[0024] The triggering event can depend on various triggering conditions. For example, a first triggering condition can be that route guidance is active in the vehicle. For example, a second triggering condition can be that a route guidance destination represents a new destination that has not yet been selected as a destination with the vehicle. The first triggering condition can be determined internally in the vehicle depending on a vehicle state. The second triggering condition can be determined externally in the vehicle, for example by an external service for storing the vehicle's historical destinations, and communicated to the vehicle by the external service upon request from the vehicle. If all triggering conditions of a triggering event are met, the digital assistant can execute a vehicle function linked to the triggering event.For example, the digital assistant can execute a function of the vehicle that provides information about a journey and / or a destination of the active route guidance using the large language model.

[0025] For example, the digital assistant can execute a vehicle function that adapts route guidance so that the personal preferences of all vehicle occupants are taken into account as destinations. A personal preference can be a vehicle occupant's home address.

[0026] The method 100 can determine 104 at least one personal preference for each occupant, in particular for each uniquely identified occupant, of the multiple occupants of the vehicle depending on the triggering event of the digital assistant by the vehicle. A personal preference can be representative of a personal characteristic and / or a personal behavior, and / or include personal data of an occupant, which is stored, for example, in a user profile of the occupant of the vehicle. Depending on the triggering event, various personal preferences of the respective occupant of the vehicle can be relevant. In other words, the triggering event can be used as a filter for determining at least one personal preference of an occupant. Preferably, only personal preferences of the occupants are determined that have a preferably high relevance to the triggering event.For example, a personal preference of an occupant of the vehicle may be determined that is required to execute a function of the vehicle associated with the trigger event.

[0027] Additionally, the method 100 can aggregate the determined personal preferences of the multiple occupants of the vehicle into an aggregated personal preference of the multiple occupants. For example, the method can combine identical personal preferences of the multiple occupants into a combined personal preference of all occupants of the vehicle. This can prevent personal preferences from being transmitted twice from the vehicle to the large language model. For example, the method can resolve conflicting personal preferences of the multiple occupants into a uniform personal preference for all occupants using one or more predefined rules.

[0028] The method 100 can transmit 106 the determined personal preferences of the multiple occupants of the vehicle as context information from the vehicle to a first application interface of a large language model. The first application interface of the large language model can be a configuration interface of the large language model that can receive the context information as data. The method 100 can transmit the determined personal preferences of the multiple occupants and / or the aggregated personal preferences of the multiple occupants of the vehicle as context information at the beginning of a dialogue of the vehicle's digital assistant. If the context information does not change during the dialogue, no further transmission of the context information during the dialogue is necessary.If the context information changes during the dialogue, the procedure can transmit a change in the context information to the large language model.

[0029] Furthermore, the method 100 can transmit 108 a command line comprising the determined triggering event as input data from the vehicle to a second application interface of the large language model. The second application interface can be a text-based voice input interface of the large language model. The method can receive 110 a response message from the large language model depending on the transmitted personal preferences of the multiple occupants of the vehicle as context information and the transmitted command line representative of the triggering event from the vehicle. Finally, the method 100 can provide 112 the response message from the large language model to the multiple occupants of the vehicle via an output interface of the vehicle's digital assistant and / or to a function of the vehicle via an application interface.

[0030] In detail, Fig. 2 shows an application example 200 of the method 100 for operating a digital assistant of a vehicle. The method 100 can determine a triggering event 202 and transmit a command line representative of the determined triggering event 202 as input data to a large language model 204. Furthermore, the method 100 can transmit one or more pieces of context information 206 to the large language model 204. The command line representative of the determined triggering event can, for example, request a recommendation regarding a function and / or a feature of a function of the vehicle from the large language model. The method 100 can determine one or more pieces of context information 206 and transmit them to the large language model.The method 100 can determine at least one personal preference 208, 210, 212, 214 for each of the multiple occupants of the vehicle depending on the triggering event of the digital assistant by the vehicle and summarize the determined personal preferences 208, 210, 212, 214 into an aggregated personal preference 216. Furthermore, the method 100 can determine a domain-specific vehicle state 218, a situation context 220, and / or an interaction history 224 as further context information. The method 100 can transmit the aggregated personal preference 216 and / or the further context information as context information 206 to the large language model 204. The large language model 204 can generate a response message using the command line representative of the triggering event 202 and the context information 206.For example, the response message may include information and / or recommendations regarding a destination, information and / or recommendations along a navigation route, routing information, and / or a summary of the trip, depending on the personal preferences of the vehicle's occupants. Furthermore, the routing information of the response message may be used to adapt a navigation route 226 to the personal preferences of the vehicle's occupants. The information and recommendations regarding the destination and along the route may be provided to the occupants by the vehicle's digital assistant. The response message io may additionally be used to update an interaction history 224 with the large language model.

[0031] Advantageously, the method can transmit personalized preferences, for example, personalized route preferences, as context information to the large language model. The context information can be transmitted from the vehicle as data via a first application interface of the large language model. The large language model can use the context information to adapt a dialogue with the vehicle's digital assistant to the multiple occupants. The digital assistant can thus act more intelligently on the multiple occupants and can provide helpful recommendations and / or information for all recognized occupants of the vehicle. Acceptance and convenience when using the digital assistant with multiple occupants of the vehicle can thus be efficiently increased.

[0032] List of reference symbols

[0033] 100 procedures

[0034] 102 Determining a trigger event

[0035] 104 Determine at least one personal preference

[0036] 106 Transmitting the specific personal preference

[0037] 108 Submitting a command line

[0038] 110 Receiving a reply message

[0039] 112 Providing the response message

[0040] 200 systems

[0041] 202 Trigger event

[0042] 204 large language model

[0043] 206 Context information

[0044] 208 personal preference

[0045] 210 personal preference

[0046] 212 personal preference

[0047] 214 personal preference

[0048] 216 aggregated personal preference

[0049] 218 vehicle-specific condition

[0050] 220 Situational context

[0051] 222 Interaction history

[0052] 224 Navigation route

Claims

Patent claims 1. A method for operating a digital assistant of a vehicle, the method comprising: Determining a vehicle digital assistant trigger event by the vehicle; Determining at least one personal preference for several or each occupant of the vehicle depending on the triggering event of the digital assistant by the vehicle; Transmitting the determined personal preferences of the plurality of occupants of the vehicle as context information from the vehicle to a first application interface of a large language model; Transmitting a command line comprising the determined trigger event as input data from the vehicle to a second application interface of the large language model; Receiving a response message from the large language model depending on the transmitted personal preferences of the multiple occupants of the vehicle as context information and the transmitted command line representative of the triggering event by the vehicle; and Providing the response message of the large language model via an output interface of the vehicle's digital assistant to the multiple occupants of the vehicle and / or via an application interface to a function of the vehicle.

2. The method according to claim 1, wherein the triggering event of the digital assistant is determined as a function of several vehicle state-dependent triggering conditions.

3. The method according to one of the preceding claims, wherein at least one vehicle-state-dependent triggering condition of the plurality of vehicle-state-dependent triggering conditions is a vehicle-state-dependent triggering condition of a vehicle-external server; and / or wherein at least one vehicle-state-dependent triggering condition of the plurality of vehicle-state-dependent triggering conditions is a vehicle-state-dependent triggering condition of a vehicle-internal component.

4. A method according to any one of the preceding claims, the method further comprising: Aggregating the determined personal preferences of the multiple occupants of the vehicle into an aggregated personal preference of the multiple occupants; Transmitting the aggregated personal preference of the multiple occupants of the vehicle as context information from the vehicle to the first application interface of a large language model; and Receiving the response message of the large language model depending on the transmitted, aggregated, and personal preference of the multiple occupants of the vehicle as context information and the transmitted command line representative of the triggering event by the vehicle.

5. The method of claim 4, wherein aggregating the determined personal preferences of the plurality of occupants of the vehicle to the aggregated personal preference of the plurality of occupants comprises: Combining similar personal preferences of multiple occupants into a single combined personal preference for all occupants of the vehicle; and / or resolving conflicting personal preferences of multiple occupants into a single unified personal preference for all occupants using one or more predefined rules.

6. The method according to any one of the preceding claims, wherein the method additionally transmits the determined personal preferences of the plurality of occupants of the vehicle and / or the determined aggregated personal preferences of the plurality of occupants of the vehicle as input data with the command line representative of the determined triggering event from the vehicle to a second application interface of a large language model.

7. A method according to any one of the preceding claims, the method further comprising: Determining a situational context of the multiple occupants of the vehicle depending on camera data from an interior camera, audio data from one or more microphones, an occupant model, an environment model, driving information, and / or navigation information; and Transmitting the determined situational context of the multiple occupants of the vehicle as context information from the vehicle to the first application interface of the large language model.

8. A method according to any one of the preceding claims, the method further comprising: Determining a priority of the response message of the large language model depending on the determined situational context and / or an aggregated, personal output preference of the multiple occupants of the vehicle; and Providing the response message of the large language model depending on the determined priority via the output interface of the vehicle's digital assistant to the multiple occupants of the vehicle.

9. The method according to any one of the preceding claims, wherein the first application interface is a configuration interface of the large language model; and / or wherein the second application interface is a text-based speech input interface of the large language model.

10. A computer-readable medium for operating a digital assistant of a vehicle, the computer-readable medium comprising instructions which, when executed on a computer and / or a control unit, carry out the method according to any one of claims 1 to 9.

11. A system for operating a digital assistant of a vehicle, wherein the system is designed to carry out the method according to one of claims 1 to 9.

12. A vehicle comprising the system for operating a digital assistant of a vehicle according to claim 11.

Images