Control of a motor vehicle function

Abstract

A method (200) for controlling a function of a motor vehicle (105) comprises steps of capturing (205) a voice input from a person (125); determining (210) a function associated with the voice input; and determining (215, 220) a preference of the person (125) regarding the function performed.

Description

[0001] The present invention relates to the control of a function of a motor vehicle. In particular, the invention relates to the control of the function by means of a voice assistant.

[0002] A motor vehicle includes a voice assistant that a person can use to control a function of the vehicle. The voice assistant can operate based on a Large Language Model (LLM) and enable a dialogue with the person regarding the performance of the function.

[0003] For some functions, such as a search for a destination, there is often no single best solution. The best destination for a given search query depends on the preferences of the person submitting the search. A voice assistant can support the storage of preferences or statistical analysis of searches to better assist in selecting the best solution.

[0004] One of the problems underlying the present invention is to provide an improved technology for controlling a function of a motor vehicle, in particular by means of a voice assistant. The invention solves this problem by means of the subject matter of the independent claims. Dependent claims describe preferred embodiments.

[0005] According to a first aspect of the present invention, a method for controlling a function of a motor vehicle comprises steps of acquiring a voice input from a person; determining a function associated with the voice input; and determining a preference of the person with regard to the function performed.

[0006] The preference can only be determined once the function has been successfully determined. This allows the preference to be determined with increased accuracy. Collected preferences can be used to improve the processing of subsequent speech inputs. In a particularly preferred embodiment, a function assigned to a subsequent speech input is determined or controlled with respect to the determined preference. According to this method, an improved picture of the person's preferences can be created across a number of speech inputs. This picture can be used to better evaluate a person's verbally formulated request. The function can thus be determined more accurately and executed more precisely.

[0007] Furthermore, an existing preference regarding a specific preference is updated. A person's habit or preference can be better defined by accumulating preferences. Should the person's preference change over time, this can be better reflected by updating a preference. This prevents, for example, the application of a preference that has become outdated or irrelevant.

[0008] The preference preferably relates to a manner, a boundary condition, or a means of the function to be performed. The manner might, for example, relate to the speed or intensity of the function. If a person requests that interior lighting be switched on, the interior lighting can be dimmed to full brightness at a speed that is comfortable for the person. A boundary condition might relate to a circumstance that accompanies the control of the function. For example, the function might be performed differently during the day and at night. A means might include a controllable device on board the motor vehicle by which the function is performed. For example, if an acoustic output is to be made, a loudspeaker or other playback device by which the output is made can be considered the means.

[0009] In another embodiment, the function relates to locating a Point of Interest (POI). A Point of Interest is a location to which a motor vehicle can be driven to perform a predetermined task. This location is typically associated with a geographical position. POIs can be grouped together. For example, gas stations where fuel is available for operating the motor vehicle can be grouped together. If a person is searching for a gas station, all POIs in this group can be considered as potential destinations. Using the technique described herein, a gas station can be selected based on the person's preference. For example, gas stations belonging to a particular chain can be preferred, while those belonging to another chain can be avoided.Additionally, another criterion can be considered, such as the distance between the current geographic position of the vehicle and the geographic position of a POI. Different criteria can be weighted or applied hierarchically.

[0010] In the weighted variant, criteria assigned to a POI can be given a predetermined weight to determine an overall suitability. The POI with the highest overall suitability can then be selected. In the hierarchical variant, POIs can first be sorted according to a primary criterion. POIs with the same or similar suitability in this respect can then be further differentiated according to a second criterion.

[0011] Preferably, the function is determined via voice input using an LLM-based voice assistant. In various embodiments, the LLM can be implemented locally in the vehicle or at a remote location. The LLM can allow for a certain degree of dialogue with the user to determine the function based on voice input. Both the outcome of the dialogue and, optionally, a statement made during the dialogue can be used to determine the preference. For example, a condition added by the user during the dialogue or a rejection of a suggestion can be factored into the preference determination.

[0012] The LLM typically operates by assigning a multidimensional vector, known as an embedding, to a speech input and comparing this embedding with one or more predetermined embeddings. In a preferred embodiment, an embedding is created for the preference, and a preference regarding a speech input is determined by identifying a preference embedding (i.e., an embedding of the preference) whose distance to the embedding of the speech input is as small as possible.

[0013] In one embodiment, the preferred embedding is determined whose distance to the speech input embedding is the smallest. In a variant, several preferred embeddings can be determined whose distances to the speech input embedding are as small as possible. Preferably, only those preferred embeddings are determined whose distances to the speech input embedding are below a predetermined threshold.

[0014] Various metrics are known for determining distance and can be used here. In one embodiment, a Euclidean distance is used. In a preferred embodiment, a cosine distance or cosine similarity is used to compare embeddings. The greater the cosine similarity between embeddings, the lower their cosine distance, and vice versa. Cosine similarity is generally a value between 0 and 1. Embeddings can be stored in a vector-oriented database, and searching for a stored embedding with the greatest possible similarity to a desired embedding in the database can be performed implicitly as part of a search query.

[0015] A preference category can be defined, and a specific preference can be assigned to that category. A category forms a thematic group of preferences and can be predefined. Optionally, a new category can also be created, either automatically or manually by a user or administrator. Categories can be merged, or a category can be split into several different categories. A category with only a few preferences assigned to it can be removed or merged with another category.

[0016] For example, one or more of the following categories may be predetermined: Airports Hair salons ATMs Hospitals Bakeries Hotels Banks Mini Service Bars, pubs Parking spaces BMW Service Parking garages books Park & ​​Ride Cafes Gas stations Car washes police Charging stations post Pharmacies Public restrooms Cinemas Rest areas Convenience stores Restaurants Basel stores, hardware stores Flower shops Drugstores Shopping centers Fast food Stadiums Furniture Supermarkets Garden center Tea houses Sports & Fitness

[0017] A category typically contains multiple preferences. Optionally, preferences can also be prioritized, for example, based on the emphasis or vehemence with which the person expressed the preference.

[0018] Category-based preference storage can be implemented securely with lower latency or higher functionality. Categories can be manually searched to identify and potentially edit artifacts such as LLM hallucinations. Data privacy can be simplified because only preferences within a selected category are stored, preventing the storage of data irrelevant to a closed list of intended use cases. It's worth noting that category-based preference storage can also be combined with embedding-based storage.

[0019] It remains preferred that the specified function be executed with respect to the specified preference. Execution can include initiating or controlling the function. The function can be executed in the context of the motor vehicle and may involve controlling the motor vehicle or one of its components. The preference can be taken into account once it has been initially determined. Subsequent voice inputs can be executed or controlled with respect to a saved or adjusted preference.

[0020] In a further preferred embodiment, the preference is assigned to the person. This allows the preferences of different people who can use the vehicle to be managed. The preferences of different users can be kept separate. For example, the person making a voice input can be determined based on optical person recognition, active authentication of the person to the vehicle, or voice recognition. A recognized preference that cannot be assigned to any person can instead be assigned to a virtual person who acts as a guest with respect to the vehicle.

[0021] A preference can be viewed and, preferably, edited by the user. A suitably privileged user can also view or edit preferences assigned to different people. For example, a preference from one person can be adopted by another. A guest's preference can be copied or moved to another person. A preference can also be modified, deleted, or created. Preferences can be edited in any way, such as via text input, voice input, or a graphical user interface.

[0022] According to a further aspect of the present invention, a device for controlling a function of a motor vehicle comprises an input device for acquiring a voice input; and a processing unit. The processing unit is configured to determine a function associated with the voice input. Furthermore, the processing unit is configured to determine a person's preference regarding the function performed. The preference is preferably also determined with respect to the voice input.

[0023] The device can operate partially or completely on board the motor vehicle.

[0024] The processing device is preferably configured to partially or completely execute a method described herein. For this purpose, the processing device may be electronic and may, for example, comprise an integrated circuit, a programmable logic device, or a programmable microcomputer. The method may be implemented as a configuration or as a computer program product with program code means for the processing device. The configuration or the computer program product may be stored on a computer-readable data carrier. Features or advantages of the method may be transferred to the device or vice versa.

[0025] According to yet another aspect of the present invention, a system comprises a device as described herein and an external device. The device and the external device are connected to each other via a wireless data connection. The external device is configured to store a person's preference. Optionally, the external device can perform one or more functions that can alternatively be carried out by the processing unit of the device. In particular, determining a function based on voice input can also be performed by the external device.

[0026] The external entity can be implemented as a server or as a service, particularly in a cloud. A preference can be synchronized between the vehicle and the external entity. Synchronization preferably occurs in both directions.

[0027] In another embodiment, a saved preference can be viewed or edited. A user interface for this purpose can be provided on board the vehicle. Alternatively, a user can also access saved preferences using another device, in particular a mobile device, a smartphone, a laptop computer, a tablet computer, or a desktop computer. The preferences are preferably stored in a format that can be read and understood by a human.

[0028] According to yet another aspect of the present invention, a motor vehicle comprises a device described herein. In one embodiment, the motor vehicle and the device form part of a system described herein.

[0029] The invention will now be described in more detail with reference to the attached drawings, in which: Fig. 1 a system; Fig.2. A flowchart of a procedure; and Fig. 3 A schematic representation of an exemplary determination of a stored preference is illustrated.

[0030] Fig. Figure 1 shows a system 100 with a motor vehicle 105, an external location 110 (also referred to herein as device 110) and a mobile device 115. A device 120 is attached on board the motor vehicle 105.

[0031] The device 120 can be used by a person 125 who is on board the motor vehicle 105. The device 120 is configured to capture a voice input from the person 125 and determine a function associated with that voice input. The function is preferably determined with respect to one or more preferences of the person 125 that have been previously defined. It is proposed to determine and manage preferences based on voice inputs and the resulting functions of the motor vehicle 105. The person 125 can manage these preferences by actively specifying, editing, or deleting an existing preference. The person 125 can do this using a user interface on board the motor vehicle 105. Alternatively, the person 125 can use a user interface on the mobile device 115. Instead of a mobile device 115, another device can also be used, in particular any type of computer.

[0032] The device 120 comprises a processing unit 130 and a speech interface 135. The speech interface 135 can include an input device, for example in the form of a microphone, and an output device, for example in the form of a loudspeaker. The processing unit 130 is configured to process speech input from person 125. For this purpose, a suitably trained Large Language Model 140 can be executed in the device 120. The LLM 140 preferably comprises a generic language model configured to process arbitrary speech input.

[0033] The LLM can be multimodal and operate directly on acoustic data. Alternatively, acoustic data can be converted into text data via speech-to-text, which the LLM 140 can then process. Output from the LLM 140 can be converted back into acoustic data via text-to-speech. In a further embodiment, the multimodality of the LLM can extend to one or more additional input channels. For example, gestures or facial expressions of person 120 can be taken into account based on optical scanning, in particular by means of a camera on board the vehicle 105.

[0034] Information from an optical channel can be processed directly by the LLM or first converted into text. This text can include a description of an image or a video stream. The focus can be on predetermined features while neglecting others. For example, it might be relevant that person 120 appears "angry" based on their facial expression, while the color of their clothing is irrelevant and requires no verbal description. Optionally, person 120 can also be identified based on such an image or video stream.

[0035] In another embodiment, the LLM 140 is executed by the external entity 110. For communication with the external entity 110, the device 120 preferably includes a wireless interface 145. The wireless interface 145 can, in particular, include mobile communication, WLAN, or Bluetooth.

[0036] Preferably, the device 120 comprises an interface 150 for connection to a device on board the motor vehicle 105, which can be controlled depending on the voice input of the person 125. The interface 150 can also be connected to several systems or subsystems on board the motor vehicle 105 and control them accordingly.

[0037] Device 120 can function as a voice assistant for person 125. It is proposed to determine person 125's preferences based on dialogue data and to use these preferences as the basis for determining and / or controlling a function associated with voice input. Preferences can be collected and processed together. This allows one preference to be compared with another. An existing preference can be updated, weakened, changed, strengthened, or deleted by a newly determined preference.

[0038] Fig.Figure 2 shows a flowchart of a method 200 for controlling a function of a motor vehicle 105. The method 200 can in particular be carried out by means of a device 120 on board a motor vehicle 105 or a system 100.

[0039] In step 205, a voice input from a person 125 can be captured. It is assumed below that only a single person 125 is to be processed. In another embodiment, different persons 125 can also be distinguished from one another. In this case, steps of the procedure 200 can be executed individually for different persons 125.

[0040] In step 210, an attempt can be made to determine a function of the motor vehicle 105 that corresponds to the recorded voice input. The determined function can be presented to person 125 for confirmation, for example, visually or audibly. Person 125 can evaluate the function determination positively or negatively, or further refine it by providing additional information or instructions. This can be done, for example, through further input in step 205. The further input can include confirmation, rejection, or modification. The input can be expressed, for example, verbally, manually (haptically), or by gesture or facial expression from person 120.

[0041] From the dialogue of steps 205 to 210, or from the initial and final information, a preference for person 125 can be determined in step 215. In parallel, a preference can be determined in step 220 that is associated with the speech input or its recognized content. An associated preference from step 220 and a specific preference from step 215 can be combined in step 225. In this process, the stored preference can be strengthened, modified, or weakened.

[0042] A preference can also be edited in step 230 by person 125 or another person 125 assigned to motor vehicle 105. Editing can include viewing, changing, adding, or deleting a preference. Preferences can be grouped into categories, and a preference can be assigned to, removed from, or moved from one category to another.

[0043] In step 235, in addition to a preference, further criteria relevant to determining or controlling the function can be defined. For example, if the function involves searching for a POI, then in step 235, POIs determined with respect to a preference can be sorted according to their geometric distances to the current geographic position of vehicle 105. The closer a POI is to the geographic position of vehicle 105, or the less effort it takes to bring vehicle 105 to a POI, the more likely that POI will be selected. In other contexts where the function does not involve determining a POI, a different metric can be applied as an additional criterion.

[0044] The function determined in step 210 can be executed in step 240 with respect to the assigned preference and the further criteria determined in step 235. The function is preferably controlled on board the motor vehicle 105. If the function does not involve the determination of a point of interest (POI), the control of a system or subsystem of the motor vehicle 105 may be part of the function. Such a system may, in particular, include an information system, an entertainment system, a navigation system, or a comfort system.

[0045] Fig. Figure 3 shows a schematic representation of an example of determining a stored preference. The procedure shown can be used to assign a previously known preference to a speech input using an LLM.

[0046] In step 305, a voice input from person 125 is recorded. This step can be compared to step 205 of procedure 200. Fig. 2 correspond. In step 310, the speech input can be converted into an embedding 315 using the LLM 140. The embedding 315 can be represented as a vector with a multitude of dimensions. Each dimension can be assigned a size, some of which are in Fig. 3 are presented in the form of purely illustrative values.

[0047] The specified embedding 315 can now be compared with previously known or stored embeddings 320. Similarity measures, such as cosine similarity, can be determined between the specified embedding 315 and the stored embeddings 320. A predetermined number of known embeddings 320, whose similarity to the specified embedding 315 exceeds a predetermined threshold, can then be selected. Fig.Figure 3 shows exemplary cosine similarities 325. The greater the cosine similarity 325, the more similar a known embedding 320 is to a given embedding 315. For example, the known embedding 320 with the greatest cosine similarity 325 to a given embedding 315 can be determined. This can then be identified and output as the most similar embedding 330.

[0048] The steps in Fig. Three can be used to determine a preference for a speech input that is stored as a known embedding 320. The content of the speech input can be interpreted with respect to the content of a known embedding 320. A function associated with the speech input can be determined, selected, or controlled with respect to the content of the most similar embedding 330. Reference sign 100 System 105 motor vehicle 110 external position 115 Mobile device 120 Device 125 people 130 processing facilities 135 Voice interface 140 Large Language Model (LLM) 145 wireless interface 150 interface 200 procedures 205 Capture speech input 210 Determine function 215 Determine preference from dialogue Determine 220 assigned preferences Update 225 preferences 230 Edit preference 235 further criteria 240 Execute function 305 Capture speech input 310 Determine embedding 315 Embedding 320 known embedding 325 cosine similarities 330 most similar embedding

Claims

[1] Method (200) for controlling a function of a motor vehicle (105), comprising the following steps: - Capturing (205) a person's voice input (125); - Determine (210) one of the functions assigned to speech input; and - Determine (215, 220) a preference of the person (125) regarding the function performed. [2] Method (200) according to claim 1, wherein a function that is associated with a subsequent speech input is determined (220) with respect to the specified preference. [3] Method (200) according to claim 1 or 2, wherein an existing preference with respect to the specified preference is updated (225). [4] Method (200) according to any of the preceding claims, wherein the preference relates to a manner, a boundary condition or a means of performing the function to be carried out. [5] Method (200) according to any of the preceding claims, wherein the function relates to locating a POI. [6] Method (200) according to one of the preceding claims, wherein the function with respect to speech input is determined by means of an LLM-based (140) voice assistant. [7] Method (200) according to claim 6, wherein an embedding (315) is created for the preference; and a preference (320) regarding a speech input is determined by determining a preference embedding (315) whose distance (325) to the embedding of the speech input is small. [8] Method (200) according to any of the preceding claims, wherein a preference category is determined and the specified preference is assigned to the category. [9] Method (200) according to any of the preceding claims, wherein the specified function is performed with respect to the specified preference (240). [10] Method (200) according to any of the preceding claims, wherein the preference is assigned to the person (125). [11] Device (120) for controlling a function of a motor vehicle (105), the device comprising the following elements: - an input device (135) for capturing speech input; and - a processing unit (130) for determining one of the functions associated with the speech input; - wherein the processing equipment (130) is set up to determine a preference of the person (125) regarding the function performed. [12] System (100) comprising a device (120) according to claim 11 and an external location (110) to the motor vehicle (105); wherein the device (120) and the external location (110) are connected to each other by means of a wireless data connection (145); wherein the external location (110) is configured to store a preference of the person (125). [13] Motor vehicle (105) comprising a device (120) according to claim 11.

Images