Computing system and method for generating user-specific automated vehicle actions using artificial intelligence

Abstract

Methods, computing systems, and techniques for automated vehicle action generation are presented. For example, a computing system may be configured to receive contextual data associated with a plurality of user interactions of a vehicle function of a vehicle. The contextual data may include data indicative of a plurality of user-selected settings for the vehicle function and data indicative of observed conditions associated with respective user-selected settings. The computing system may be configured to generate a user-activity cluster for the vehicle function based on the contextual data using a machine learning clustering model. The computing system may be configured to determine an automated vehicle action based on the user-activity cluster. The computing system may output command instructions for the vehicle to implement the automated vehicle action based on whether the vehicle detects one or more trigger conditions to automatically control the vehicle function according to an automation setting.

Description

Technical Field

[0001] The present disclosure generally relates to using artificial intelligence, including machine learning models, to generate vehicle actions that will be automatically performed by a vehicle and customized specifically for a user of the vehicle. Background Art

[0002] Vehicles, such as cars, have on-vehicle control systems that operate certain functions of the vehicle in response to inputs from an operator of the vehicle. This includes control functions, such as braking, accelerating, and steering, as well as comfort-related functions, such as air conditioning and seat position. The operator can physically manipulate devices or touch screen elements to control these functions as the operator deems appropriate. Summary of the Invention

[0003] Certain aspects and advantages of the present disclosure will be set forth in part in the following description, or may be learned from the description, or may be learned by practice of the specific embodiments.

[0004] One example aspect of the present disclosure relates to a computing system that includes control circuitry. The control circuitry may be configured to: receive context data associated with a plurality of user interactions with a vehicle function of a vehicle over a plurality of time instances. The context data may include data indicating settings of a plurality of user selections for the vehicle function and data indicating one or more observed conditions associated with the settings of the corresponding user selections. The control circuitry may be configured to: use a machine learning clustering model to generate user-activity clusters for the vehicle function based on the context data. The machine learning clustering model may be configured to identify the user-activity clusters based on at least a portion of the settings of the user selections and at least a portion of the one or more observed conditions associated with the settings of the corresponding user selections. The control circuitry may be configured to: determine an automated vehicle action based on the user-activity clusters. The automated vehicle action may indicate an automated setting for the vehicle function and one or more trigger conditions for automatically implementing the automated setting. The control circuitry may output a command instruction for the vehicle to implement the automated vehicle action based on whether the vehicle detects the one or more trigger conditions, thereby automatically controlling the vehicle function according to the automated setting.

[0005] In one embodiment, the machine learning clustering model may be an unsupervised learning model that is configured to use probabilistic clustering to process the context data to generate the user-activity clusters.

[0006] In one embodiment, the machine learning clustering model may be configured to determine the boundaries of the user-activity cluster for the vehicle function based on the one or more observed conditions.

[0007] In one embodiment, to determine the automated vehicle action, the control circuit may be configured to: process the user-activity cluster using a vehicle action model, where the vehicle action model is a rule-based model, and the rule-based model is configured to apply corresponding weights to each of the corresponding user-selected settings within the user-activity cluster. The control circuit may be configured to: determine the automated settings for the vehicle function based on the weights of the corresponding user-selected settings within the user-activity cluster.

[0008] In one embodiment, to receive the context data, the control circuit may be further configured to: receive data indicating the one or more observed conditions via multiple sensors or systems of the vehicle. The observed conditions may include at least one of the following: (i) the date and / or time of day when the corresponding user interaction with the vehicle function occurred; (ii) the location where the corresponding user interaction with the vehicle function occurred; (iii) the route on which the corresponding user interaction with the vehicle function occurred; or (iv) the temperature at the time when the corresponding user interaction with the vehicle function occurred.

[0009] In one embodiment, the control circuit is further configured to: generate, based on the automated vehicle action and using an action explanation model, content for presentation to the user via a user interface of a display device. The content indicates the vehicle function, the automated settings, and the one or more trigger conditions.

[0010] In one embodiment, the content may include a request for the user to approve the automated vehicle action.

[0011] In one embodiment, the control circuit may be further configured to: store the command instruction in an accessible memory on the vehicle for execution at a later time.

[0012] In one embodiment, the control circuit may be further configured to: detect the occurrence of the one or more trigger conditions, and based on the one or more trigger conditions, send a signal to implement the automated settings for the vehicle function.

[0013] In one embodiment, the control circuit may be further configured to: determine whether there is a conflict between the automated vehicle action and a pre-existing automated vehicle action.

[0014] In one embodiment, the control circuit may be further configured to: send a communication indicating the command instruction to a server system for storage in a manner associated with a user profile of a user.

[0015] In one embodiment, the plurality of user interactions may be associated with a first user. The command instruction for the automated vehicle action may be associated with a first user profile of the first user. The control circuit may be further configured to: receive data indicating a second user profile of a second user of the vehicle; and store a command instruction for a second automated vehicle action associated with the second user profile in an accessible memory of the vehicle. The command instruction for the second automated vehicle action may be based on at least one user interaction of the second user with a vehicle function of another vehicle.

[0016] In one embodiment, the vehicle may include a plurality of vehicle functions and a plurality of machine learning clustering models. The control circuit may be further configured to: select the machine learning clustering model from the plurality of machine learning clustering models based on the vehicle function.

[0017] In one embodiment, the vehicle function may include: (i) a window function; (ii) a seat function; or (iii) a temperature function.

[0018] In one embodiment, the seat function may include: a seat temperature function, a seat ventilation function, or a seat massage function.

[0019] In one embodiment, the settings selected by the plurality of users of the vehicle function may each include at least one of the following: (i) an on / off selection; (ii) an open / close selection; (iii) a temperature selection; or (iv) a massage level selection.

[0020] Another example aspect of the present disclosure relates to a computer-implemented method. The computer-implemented method may include: receiving context data associated with a plurality of user interactions with a vehicle function at a plurality of time instances. The context data may include data indicating settings of a plurality of user selections for the vehicle function and data indicating one or more observed conditions associated with the settings of the corresponding user selections. The computer-implemented method may include: using a machine learning clustering model to generate user-activity clusters for the vehicle function based on the context data. The machine learning clustering model may be configured to identify the user-activity clusters based on at least a portion of the settings of the user selections and at least a portion of the one or more observed conditions associated with the settings of the corresponding user selections. The computer-implemented method may include: determining an automated vehicle action based on the user-activity clusters. The automated vehicle action may indicate an automated setting for the vehicle function and one or more trigger conditions for automatically implementing the automated setting. The computer-implemented method may include: outputting command instructions for the vehicle to implement the automated vehicle action based on whether the vehicle detects the one or more trigger conditions, thereby automatically controlling the vehicle function according to the automated setting.

[0021] In one embodiment, the machine learning clustering model may include an unsupervised learning model configured to use probabilistic clustering to process the context data to generate the user-activity clusters; and the boundaries of the user-activity clusters for the vehicle function may be based on the one or more observed conditions.

[0022] In one embodiment, the computer-implemented method may further include: generating content for presentation to a user via a user interface of a display device based on the automated vehicle action and using an action explanation model. The content may indicate the vehicle function, the automated setting, and the one or more trigger conditions.

[0023] Another example aspect of the present disclosure relates to one or more non-transitory computer-readable media storing instructions executable by a control circuit to perform operations. The control circuit may receive context data associated with a plurality of user interactions with a vehicle function at a plurality of time instances. The context data may include data indicating settings for a plurality of user selections for the vehicle function and data indicating one or more observed conditions associated with the settings selected by the respective user. The control circuit may use a machine learning clustering model to generate user-activity clusters for the vehicle function based on the context data. The machine learning clustering model may be configured to identify the user-activity clusters based on at least a portion of the settings selected by the user and at least a portion of the one or more observed conditions associated with the settings selected by the respective user. The control circuit may determine an automated vehicle action based on the user-activity clusters. The automated vehicle action may indicate an automated setting for the vehicle function and one or more trigger conditions for automatically implementing the automated setting. The control circuit may output command instructions for the vehicle to implement the automated vehicle action based on whether the vehicle detects the one or more trigger conditions, thereby automatically controlling the vehicle function according to the automated setting.

[0024] Other example aspects of the present disclosure relate to other systems, methods, vehicles, devices, tangible non-transitory computer-readable media, and apparatuses for improving the operation of a vehicle and the computational efficiency associated with the vehicle.

[0025] These and other features, aspects, and advantages of the various embodiments will become better understood with reference to the following description and the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and together with the description serve to explain the related principles. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] A detailed discussion of embodiments involving those of ordinary skill in the art is set forth in the specification, which refers to the accompanying drawings, in which:

[0027] Figure 1 An example computing ecosystem in accordance with one embodiment of the present disclosure is illustrated.

[0028] Figure 2 A diagram illustrating an example computing architecture in accordance with one embodiment of the present disclosure is shown.

[0029] Figure 3 A diagram illustrating an example data processing pipeline in accordance with one embodiment of the present disclosure is shown.

[0030] Figure 4A diagram illustrating an example data structure including data associated with user interactions according to one embodiment of the present disclosure.

[0031] Figures 5A to 5D An example user-activity cluster according to one embodiment of the present disclosure is illustrated.

[0032] Figure 6 A diagram illustrating an example user interface on an example display according to one embodiment of the present disclosure.

[0033] Figure 7 A diagram illustrating an example user interface on an example display according to one embodiment of the present disclosure.

[0034] Figure 8 A diagram illustrating an example user interface on an example display according to one embodiment of the present disclosure.

[0035] Figures 9A to 9B An example data structure including data associated with automated vehicle actions according to one embodiment of the present disclosure is illustrated.

[0036] Figures 10A to 10B A flowchart diagram illustrating an example method for generating automated vehicle actions for a user according to one embodiment of the present disclosure.

[0037] Figure 11 A flowchart diagram illustrating an example method for implementing automated vehicle actions for a second user according to one embodiment of the present disclosure.

[0038] Figure 12 A block diagram of an example computing system according to one embodiment of the present disclosure is illustrated. Detailed Description

[0039] One aspect of the present disclosure relates to using personalized artificial intelligence to analyze a user's interaction with a vehicle and generate actions that will be automatically performed by the vehicle for a specific user. The personalized artificial intelligence that allows the vehicle to mimic the user's behavior can be based on a machine learning model that learns the user's preferences over time. For example, a vehicle (e.g., a car) can include multiple vehicle functions. A vehicle function can refer to the functionality or operation that the vehicle is configured to perform based on an input. For example, a user can interact with a vehicle function (e.g., activate or adjust a vehicle function). The user can be the driver of the vehicle. The vehicle can observe the user's interactions and build a set of context data that reflects the user's preferences for vehicle functions over time. Based on the context data, the vehicle can create an expected user preference and automatically implement an automated vehicle action for the vehicle functionality according to the user preference.

[0040] For example, a user may interact with the seat massage function. The user may activate the massage device in the seat and / or select a massage setting. The vehicle computing system of the vehicle may observe the user's interaction with the seat massage function and record certain information of each interaction as context data. For example, the computing system may record the specific massage setting selected by the user (e.g., "Classic Massage"). Additionally, when the user selects a specific setting, the computing system may record the time, temperature, and location conditions observed by the vehicle. As an example, for a given user interaction, the vehicle may observe that the user selects a massage setting at 5:05 PM PT on Monday, the outside temperature is 65 degrees Fahrenheit, and it is at a certain latitude / longitude pair corresponding to the user's workplace. The user may interact with the seat massage function every day of the week. The computing system may observe the user's selected massage settings to build a set of context data that reflects the user's preferences for the seat massage function. The vehicle may use the context data to learn the user's preferences for the seat massage function and implement the seat massage function accordingly.

[0041] To help better identify the user's routines and automate the user's preferences, the techniques of the present disclosure may utilize a multi-level multi-model computing framework to process the context data.

[0042] For example, the computing system may store or access multiple machine learning clustering models. Each corresponding machine learning clustering model may be associated with a specific vehicle function. This may allow the model to better learn its hyperparameters and calibrate the hyperparameters to the type of user interaction the user performs with the specific vehicle function. For example, to learn from the user's seat massage function interaction, the computing system may use the machine learning clustering model associated with the seat massage function.

[0043] The machine learning clustering model may be configured to apply clustering modeling techniques to generate data clusters based on patterns identified within the user's activities ("user-activity clusters"). The boundaries of the user-activity clusters may be defined by the machine learning clustering model based on the observed conditions. For example, as will be further described herein, once a threshold number of data points with a common time range (e.g., 5:03 PM to 5:53 PM PT), temperature range (e.g., 60 degrees Fahrenheit to 68 degrees Fahrenheit), and location range (e.g., within 1000 feet of the user's workplace) are collected, the machine learning clustering model may identify the presence of a cluster. For example, for seat massage function interactions, the user-activity cluster may include multiple data points, each associated with a specific user interaction with the seat massage function. The data points may include metadata indicating the massage setting selected by the user and the time, temperature, and location conditions observed when the setting was selected. The machine learning clustering model may group the data points into user-activity clusters based on similarities or other patterns in the metadata to establish a bounded region of parameters associated with the user's preferences for the seat massage function.

[0044] A computing system can process user-activity clusters to determine skills (also known as routines) for a vehicle that automate a user's preferred settings for vehicle-specific (or non-vehicle-specific) functions. To do so, the computing system can utilize a vehicle action model. In some embodiments, the vehicle action model can be a rule-based model that weights various user-selected settings to, for example, determine the settings most frequently selected by the user (e.g., classic massage). The rule-based model can include a set of predefined rules or heuristics that can be processed to achieve a goal (e.g., for applying weights for setting analysis and selection). In some embodiments, the vehicle action model can include one or more machine learning models. The vehicle can implement the setting as an automated setting.

[0045] Additionally, the vehicle action model can determine the conditions ("trigger conditions") that will trigger the automated setting. In one embodiment, the trigger conditions can be based on, for example, a time, temperature, or location range associated with the user-activity cluster. Thus, the computing system can generate an "automated vehicle action" that defines the relationship between the automated setting of a vehicle function and the trigger conditions. For example, the relationship can be expressed as an if / then logical statement: If the time is between 5:00 PM and 5:35 PM in PT time, the temperature is between 60 degrees Fahrenheit and 65 degrees Fahrenheit, and the vehicle is within 1000 feet of the user's workplace, then activate the classic massage setting for the seat massage function.

[0046] To confirm that the automated vehicle action is appropriate, the computing system can determine whether the new automated vehicle action conflicts with another automated vehicle action. For example, the computing system can compare the automated setting and trigger conditions of the new automated vehicle action with pre-existing automated vehicle actions. Based on this comparison, the computing system can confirm that the vehicle can perform both automated vehicle actions without modifying either of the automated vehicle actions.

[0047] In one embodiment, a computing system may request user approval of an automated vehicle action before storing the automated vehicle action in a memory of the vehicle for execution. To this end, the computing system may generate content for presentation to the user via a vehicle's in-vehicle host unit display device. The content may be a prompt requesting the user to approve by selecting an element on a touch screen, providing a verbal confirmation, etc. Upon approval, the computing system may output command instructions for the vehicle to implement the automated vehicle action based on whether the vehicle detects a trigger condition, thereby automatically controlling a vehicle function (e.g., a seat massage function) according to an automation setting (e.g., classic massage). A library may store the command instructions to maintain the automated vehicle action in a manner associated with a particular user or the user's profile. As will be described in further detail, a remote cloud server may store the library such that the user's automated vehicle actions may be automatically downloaded to another vehicle operable by the user. In this way, the systems and methods of the present disclosure may personalize the user and automate the user's actions in a manner that can be transferred across multiple vehicles.

[0048] The techniques of the present disclosure provide many technical effects and improvements to vehicle and computing technologies. For example, a computing system may utilize machine learning clustering models dedicated respectively to various vehicle functions. The models may be trained and retrained based on data specific to the vehicle function and the user's interaction with the vehicle function. Thus, the machine learning clustering models may better learn the user's routines and patterns regarding a particular vehicle function. By improving the automatic learning of user preferences, for example, the techniques of the present disclosure may reduce or replace manual instructions or configuration inputs from the user, which may result in a delay in implementing user preferences. In this way, for example, the techniques of the present disclosure may increase the responsiveness of the vehicle in implementing vehicle functions (e.g., reduce latency). Additionally, by improving the automatic learning of user preferences, for example, the techniques of the present disclosure may reduce or replace manual instructions or configuration inputs from the user that consume processing power and rendering capabilities (e.g., capabilities for rendering audio communications, graphics communications), thereby helping to reserve such resources for other vehicle operations.

[0049] Furthermore, the improved machine learning clustering models may produce more accurate clusters, which enables automated vehicle actions that are more likely to be accepted by the user. By reducing the likelihood that an action will be rejected by the user, the techniques of the present disclosure may help to reserve limited processing, memory, power, and bandwidth resources on the vehicle for more core vehicle operations.

[0050] The technology of the present disclosure can improve the computational configurability of a vehicle. More specifically, by leveraging a clustering model that is personalized for a specific vehicle function, a cloud-based platform can more easily and accurately (e.g., via a software update push over a network) update the in-vehicle software of a vehicle to add models or remove deprecated models as the vehicle function changes. This can help avoid system restarts or lengthy reconfigurations that can be time-consuming and resource-intensive.

[0051] The automated vehicle actions generated by the systems and methods of the present disclosure can also improve the efficiency of the in-vehicle computing resources of a vehicle. For example, automating vehicle actions based on a user's preferred routines can reduce the number of user interactions that the user would otherwise have with a specific vehicle function. By reducing the frequency of user interactions, the vehicle can reduce the amount of processing and memory resources spent each time the user manually adjusts a vehicle function. Additionally, this can reduce wear on the physical interfaces associated with the vehicle function.

[0052] The technology of the present disclosure can also help reduce the unnecessary use of computing resources across multiple vehicles. For example, as further described herein, an automated vehicle action created by a first vehicle can be passed (e.g., via a cloud platform that maintains a user's profile) to a second vehicle. In this way, the second vehicle can avoid using its computing resources to recreate an automated vehicle action that has already been performed by the first vehicle. This can include, for example, avoiding computational tasks such as clustering, algorithm weighting, user interface generation, etc.

[0053] Ultimately, the systems and methods of the present disclosure improve the computational efficiency and configurability of a vehicle while also providing a personalized user experience that can be wirelessly transferred to another vehicle.

[0054] Reference will now be made in detail to the embodiments, one or more examples of which are illustrated in the drawings. Each example is provided by way of explanation of the embodiments and not limitation of the present disclosure. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made to the embodiments without departing from the scope or spirit of the present disclosure. For example, the functions illustrated or described as part of one embodiment can be used with another embodiment to yield yet another embodiment. Accordingly, aspects of the present disclosure are intended to cover such modifications and variations.

[0055] The technology of the present disclosure can include such collection of data associated with a user in cases where the user has explicitly authorized the collection of such data. Such authorization can be provided by the user via an explicit user input to a user interface in response to a prompt that explicitly requests such authorization. The data collected can be anonymized, pseudonymized, encrypted, noise-added, securely stored, or otherwise protected. The user can opt out of such data collection at any time.

[0056] Figure 1 Illustrates an example computing ecosystem 100 in accordance with one embodiment of the present disclosure. The ecosystem 100 may include a vehicle 105, a remote computing platform 110 (also referred to herein as the computing platform 110), and a user device 115 associated with a user 120. The user 120 may be a driver of the vehicle. In some embodiments, the user 120 may be a passenger of the vehicle. The vehicle 105, the computing platform 110, and the user device 115 may be configured to communicate with each other via one or more networks 125.

[0057] The systems / devices of the ecosystem 100 may communicate using one or more application programming interfaces (APIs). This may include external-facing APIs to communicate data from one system / device to another. The external-facing APIs may allow systems / devices to establish secure communication channels via a secure access channel on the network 125 by any number of methods such as web-based forms, programmatic access via RESTful APIs, Simple Object Access Protocol (SOAP), Remote Procedure Call (RPC), script access, etc.

[0058] The computing platform 110 may include a computing system remote from the vehicle 105. In one embodiment, the computing platform 110 may include a cloud-based server system. The computing platform 110 may include one or more backend services to support the vehicle 105. Each service may include, for example, a remote assistance service, a navigation / routing service, a performance monitoring service, etc. The computing platform 110 may host or otherwise include one or more APIs for communicating data to / from the computing system 130 of the vehicle 105 or the user device 115.

[0059] The computing platform 110 may include one or more computing devices. For example, the computing platform 110 may include control circuitry 185 and a non-transitory computer-readable medium 190 (e.g., a memory). The control circuitry 185 of the computing platform 110 may be configured to perform the various operations and functions described herein.

[0060] In one embodiment, the control circuitry 185 may include one or more processors (e.g., microprocessors), one or more processing cores, programmable logic circuitry (PLC) or programmable logic / gate arrays (PLA / PGA), field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), or any other control circuitry.

[0061] In one embodiment, the control circuitry 185 may be programmed by one or more computer-readable instructions or computer-executable instructions stored on the non-transitory computer-readable medium 190.

[0062] In one embodiment, the non-transitory computer-readable medium 190 can be a memory device (also referred to as a data storage device), which can include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. The non-transitory computer-readable medium 190 can form, for example, a hard disk drive (HDD), a solid-state drive (SDD), or a solid-state integrated memory, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a dynamic random access memory (DRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), and / or a memory stick. In some cases, the non-transitory computer-readable medium 190 can store computer-executable instructions or computer-readable instructions, such as instructions for performing the operations and methods described herein.

[0063] In various embodiments, the terms "computer-readable instructions" and "computer-executable instructions" are used to describe software instructions or computer code configured to perform various tasks and operations. In various embodiments, if computer-readable instructions or computer-executable instructions form a module, the term "module" broadly refers to a collection of software instructions or code configured to cause the control circuit 185 to perform one or more functional tasks. When the control circuit or other hardware components are executing a module or computer-readable instructions, the module and the computer-readable / executable instructions can be described as performing various operations or tasks.

[0064] The user device 115 can include a computing device owned or otherwise accessible by the user 120. For example, the user device 115 can include a telephone, a laptop computer, a tablet computer, a wearable device (e.g., a smartwatch, smart glasses, headphones), a personal digital assistant, a gaming system, a personal desktop device, other handheld devices, or other types of mobile or non-mobile user devices. As further described herein, the user device 115 can include one or more input components, such as buttons, a touch screen, a joystick, or other cursor controls, a stylus, a microphone, a camera, or other imaging devices, motion sensors, etc. The user device 115 can include one or more output components, such as a display device (e.g., a display screen), a speaker, etc. In one embodiment, the user device 115 can include a component (such as, for example, a touch screen, etc.) configured to perform input and output functionality to receive user input and present information to the user 120. The user device 115 can execute one or more instructions to run an instance of a software application and present a user interface associated therewith. The software application that launches the corresponding transmission platform can initiate a user network session with the computing platform 110.

[0065] Network 125 can be any type of network or combination of networks that enables communication between devices. In some specific implementations, network 125 may include one or more of a local area network, a wide area network, the Internet, a secure network, a cellular network, a mesh network, a peer-to-peer communication link, or some combination thereof, and may include any number of wired or wireless links. Communication over network 125 may be accomplished, for example, via a network interface using any type of protocol, protection scheme, encoding, format, encapsulation, etc. Communication between computing system 130 and user device 115 may be facilitated by near field communication technology or short-range communication technology (e.g., Bluetooth Low Energy protocol, radio frequency signaling, NFC protocol).

[0066] Vehicle 105 can be a vehicle operable by user 120. In one implementation, vehicle 105 can be an automobile or another type of land-based vehicle manually driven by user 120. For example, vehicle 105 can be a sedan or a van. In some specific implementations, vehicle 105 can be an aircraft (e.g., a personal aircraft) or a water-based vehicle (e.g., a boat). Vehicle 105 may include operator assistance functionality such as cruise control, advanced driver assistance systems, etc. In some specific implementations, vehicle 105 can be a fully autonomous vehicle or a semi-autonomous vehicle.

[0067] Vehicle 105 may include a powertrain and one or more power sources. The powertrain may include a motor, an electric motor, a transmission, a drive shaft, an axle, a differential, electronic components, a sending device, etc. The power source may include one or more types of power sources. For example, vehicle 105 can be a fully electric vehicle (EV) that is capable of using a battery to operate the powertrain of vehicle 105 (e.g., for propulsion) and the vehicle's on-board functions. In one implementation, vehicle 105 may use combustible fuel. In one implementation, vehicle 105 may include a hybrid power source, such as, for example, a combination of combustible fuel and electricity.

[0068] Vehicle 105 may include vehicle interior. Vehicle interior may include the area of vehicle body interior of vehicle 105, including, for example, the user cabin of vehicle 105. The interior of vehicle 105 may include seats, steering mechanisms, accelerator interfaces, brake interfaces, etc. for users. The interior of vehicle 105 may include display devices, such as display screens associated with infotainment systems. Such components may be referred to as display devices of infotainment systems, or may be considered as devices for implementing embodiments including the use of infotainment systems. For illustrative and example purposes, such components may be referred to herein as host unit display devices (e.g., positioned in the front area / dashboard area of vehicle interior), rear unit display devices (e.g., positioned in the rear passenger area of vehicle interior), infotainment host units or rear units, etc.

[0069] The display device can display various content to the user 120, including information about the vehicle 105, prompts for user input, etc. The display device can include a touch screen through which the user 120 can provide user input to the user interface. The display device can be associated with an audio input device (e.g., a microphone) for receiving audio input from the user 120. In some implementations, the display device can be used as a dashboard of the vehicle 105. Figure 6 , Figure 7 and Figure 8 An example display device is illustrated in .

[0070] The interior of the vehicle 105 may include one or more lighting elements. The lighting elements may be configured to emit light in various colors, brightness levels, etc.

[0071] The vehicle 105 may include a vehicle exterior. The vehicle exterior may include an outer surface of the vehicle 105. The vehicle exterior may include one or more lighting elements (e.g., headlights, brake lights, highlight lights). The vehicle 105 may include one or more doors for accessing the vehicle interior by, for example, manipulating a door handle on the vehicle exterior. The vehicle 105 may include one or more windows, including a windshield, a door window, a passenger window, a rear window, a sunroof, etc.

[0072] For the sake of brevity, certain routines and conventional components (eg, an engine) of the vehicle 105 are not illustrated and / or discussed herein. One of ordinary skill in the art will understand the operation of conventional vehicle components in the vehicle 105.

[0073] The vehicle 105 may include a computing system 130 on the vehicle 105. The computing system 130 may be on the vehicle 105 because it is included on or within the vehicle 105. The computing system 130 may include one or more computing devices, and the one or more computing devices may include various computing hardware components. For example, the computing system 130 may include control circuitry 135 and a non-transitory computer-readable medium 140 (e.g., a memory). The control circuitry 135 may be configured to perform various operations and functions for implementing the techniques described herein.

[0074] In one embodiment, the control circuitry 135 may include one or more processors (e.g., microprocessors), one or more processing cores, programmable logic circuitry (PLC) or programmable logic / gate arrays (PLA / PGA), field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), or any other control circuitry. In some specific implementations, the control circuitry 135 and / or the computing system 130 may be part of or may form a vehicle control unit (also referred to as a vehicle controller) that is embedded or otherwise disposed in the vehicle 105 (e.g., a sedan or a van). For example, the vehicle controller may be or may include an infotainment system controller (e.g., an infotainment host unit), a telematics control unit (TCU), an electronic control unit (ECU), a central powertrain controller (CPC), a charging controller, a central external and internal controller (CEIC), a zone controller, or any other controller (the terms “or” and “and / or” may be used interchangeably herein).

[0075] In one embodiment, the control circuitry 135 may be programmed by one or more computer-readable instructions or computer-executable instructions stored on the non-transitory computer-readable medium 140.

[0076] In one embodiment, the non-transitory computer-readable medium 140 can be a memory device (also referred to as a data storage device), which may include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. The non-transitory computer-readable medium 140 can form, for example, a hard disk drive (HDD), a solid-state drive (SDD), or a solid-state integrated memory, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), dynamic random access memory (DRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), and / or a memory stick. In some cases, the non-transitory computer-readable medium 140 can store computer-executable instructions or computer-readable instructions, such as instructions for performing Figures 10A to 10B and Figure 11 the methods. Additionally or alternatively, similar such instructions can be stored in the computing platform 110 (e.g., the non-transitory computer-readable medium 190) and provided via the network 125.

[0077] The computing system 130 (e.g., the control circuit 135) can be configured to communicate with other components of the vehicle 105 via a communication channel. The communication channel can include one or more data buses (e.g., Controller Area Network (CAN)), on-board diagnostic connectors (e.g., OBD-II), or a combination of wired or wireless communication links. The vehicle systems can transmit or receive data, messages, signals, etc. to or from each other via the communication channel.

[0078] In one embodiment, the communication channel can include a direct connection, such as a connection provided via a dedicated wired communication interface (such as an RS-232 interface, a Universal Serial Bus (USB) interface) or via a local computer bus (such as a Peripheral Component Interconnect (PCI) bus). In one embodiment, the communication channel can be provided via a network. The network can be any type or form of network, such as a personal area network (PAN), a local area network (LAN) (e.g., an intranet), a metropolitan area network (MAN), a wide area network (WAN), or the Internet. The network can utilize different technology and protocol layers or protocol stacks, including, for example, Ethernet protocol, Internet protocol suite (TCP / IP), ATM (Asynchronous Transfer Mode) technology, SONET (Synchronous Optical Network) protocol, or SDH (Synchronous Digital Hierarchy) protocol.

[0079] In one embodiment, the systems / devices of vehicle 105 may communicate via an intermediate storage device or more generally via an intermediate non-transitory computer-readable medium. For example, a non-transitory computer-readable medium 140, which may be located external to computing system 130, may act as an external buffer or repository for storing information. In such examples, computing system 130 may retrieve or otherwise receive information from non-transitory computer-readable medium 140.

[0080] Vehicle 105 may include one or more human-machine interfaces (HMIs) 145. The human-machine interface 145 may include a display device as described herein. The display device (e.g., a touchscreen) may be viewable by a user of vehicle 105 (e.g., user 120, second user 175) at the front of vehicle 105 (e.g., driver's seat, front passenger seat). Additionally or alternatively, the display device (e.g., a rear unit) may be viewable by a user at the rear of vehicle 105 (e.g., rear passenger seats).

[0081] Vehicle 105 may include one or more sensors 150. The sensors 150 may be configured to: obtain sensor data. This may include sensor data associated with the surrounding environment of vehicle 105, sensor data associated with the interior of vehicle 105, or sensor data associated with a particular vehicle function. The sensor data may indicate conditions observed within the vehicle, outside the vehicle, or in the surrounding environment. For example, the sensor data may obtain image data, internal / external temperature data, weather data, data indicating the position of a user / object within vehicle 105, weight data, motion / gesture data, audio data, or other types of data. The sensors 150 may include one or more of the following: cameras (e.g., visible spectrum cameras, infrared cameras), motion sensors, audio sensors (e.g., microphones), weight sensors (e.g., for vehicle seats), temperature sensors, humidity sensors, light detection and ranging (LIDAR) systems, radio detection and ranging (RADAR) systems, or other types of sensors. Vehicle 105 may also include other sensors configured to obtain data associated with vehicle 105. For example, vehicle 105 may include an inertial measurement unit, a tire odometer device, or other sensors.

[0082] Vehicle 105 may include a positioning system 155. The positioning system 155 may be configured to: generate position data (also referred to as location data) indicating the position (also referred to as location) of the vehicle 105. For example, the positioning system 155 may determine the position in one or more of the following ways: using inertial sensors (e.g., inertial measurement units, etc.), satellite positioning systems; based on an IP address; using triangulation and / or proximity to network access points or other network components (e.g., cell towers, WiFi access points, etc.); or other suitable techniques. The positioning system 155 may determine the current location of the vehicle 105. The location may be represented as a set of coordinates (e.g., latitude, longitude), an address, a semantic location (e.g., "at work"), etc.

[0083] In one embodiment, the positioning system 155 may be configured to: locate the vehicle 105 within the environment of the positioning system. For example, the vehicle 105 may access map data that provides detailed information about the surrounding environment of the vehicle 105. The map data may provide information about: the identification and location of different roads, road segments, buildings, or other items; the location and direction of lanes (e.g., parking lanes, turning lanes, bicycle lanes, or the location and direction of other lanes within a particular road); traffic control data (e.g., the location, timing, or instructions of signs (e.g., stop signs, yield signs), traffic lights (e.g., stop lights), or other traffic signals or control devices / markers (e.g., crosswalks)); or any other data. The positioning system 155 may locate the vehicle 105 within the environment (e.g., across multiple axes) based on the map data. For example, the positioning system 155 may process sensor data (e.g., LIDAR data, camera data, etc.) to match it with a map of the surrounding environment to understand the position of the vehicle within the environment. The determined position of the vehicle 105 may be used by various systems of the computing system 130 or provided to the computing platform 110.

[0084] Vehicle 105 may include a communication system 160 that is configured to allow the vehicle 105 (and its computing system 130) to communicate with other computing devices. The computing system 130 may use the communication system 160 to communicate with the computing platform 110 or one or more other remote computing devices via the network 125 (e.g., via one or more wireless signal connections). In some embodiments, the communication system 160 may allow communication between one or more systems on the vehicle 105.

[0085] In one embodiment, the communication system 160 may be configured to allow the vehicle 105 to communicate with or otherwise receive data from the user equipment 115. The communication system 160 may utilize various communication technologies, such as, for example, the Bluetooth Low Energy protocol, radio frequency signaling, or other short-range communication technologies or near-field communication technologies. The communication system 160 may include any suitable components for docking with one or more networks, including, for example, a transmitter, a receiver, a port, a controller, an antenna, or other suitable components that may help facilitate communication.

[0086] The vehicle 105 may include multiple vehicle functions 165A-165C. The vehicle functions 165A-165C may be functions that the vehicle 105 is configured to perform based on detected inputs. The vehicle functions 165A-165C may include one or more of the following: (i) vehicle comfort functions; (ii) vehicle preparation functions; (iii) vehicle climate functions; (vi) vehicle navigation functions; (v) drive form functions; (v) vehicle parking functions; or (vi) vehicle entertainment functions.

[0087] Vehicle comfort functions may include window functions (e.g., of doors, windows, sunroofs), seat functions, wall functions, steering wheel functions, pedal functions, or other comfort functions. In one embodiment, the seat function may include, for example, a seat temperature function for controlling the seat temperature. This may include a specific temperature (e.g., in degrees Celsius / Fahrenheit) or a temperature level (e.g., low, medium, high). In one embodiment, the seat function may include a seat ventilation function for controlling the ventilation system of the seat. In one embodiment, the seat function may include a seat massage function for controlling a massager device within the seat. The seat massage function may have one or more levels, each level reflecting the intensity of the massage. In one embodiment, the seat massage function may have one or more programs / settings, each program / setting reflecting a different type or combination of massage. In one embodiment, the seat function may include a seat position function for controlling the position of the seat in one or more directions (e.g., forward / backward or up / down). The pedal function may control the position of one or more pedal controls (e.g., brake pedal, accelerator pedal) relative to the user's foot. The wall function may control the temperature of the vehicle interior wall or door. The steering wheel function may control the temperature, position, or vibration of the steering wheel.

[0088] The vehicle preparation function can control the interior lighting of vehicle 105. In one embodiment, the vehicle preparation function can include an interior lighting function. For example, the interior lighting function can control the color, brightness, intensity, etc. of the interior lights (e.g., ambient lighting) of vehicle 105. In one embodiment, the vehicle preparation function can include one or more predefined lighting programs or combinations. Each program can be set by the user or pre-programmed into the default settings of vehicle 105. In some specific implementations, the vehicle preparation function can include an exterior lighting function. For example, the exterior lighting function can control the ambient lighting located below or otherwise along the exterior of vehicle 105.

[0089] The vehicle climate function can control the interior climate of vehicle 105. In one embodiment, the vehicle climate function can include an air conditioning / heating function for controlling the air conditioning / heating system or other systems associated with setting the temperature inside the passenger compartment of vehicle 105. In one embodiment, the vehicle climate function can include a defrost or fan function for controlling the level, type, and / or location of the air flow inside the passenger compartment of vehicle 105. In one embodiment, the vehicle climate function can include an air fragrance function for controlling the fragrance inside vehicle 105.

[0090] The vehicle navigation function can control the vehicle's systems to provide a route to a specific destination. For example, vehicle 105 can include an on-board navigation system that provides a route for the user 120 to travel to the destination. The navigation system can utilize map data and GPS-based signals to provide guidance to the user 120 via a display device inside vehicle 105.

[0091] The vehicle parking function can control the parking-related functions of the vehicle. In one embodiment, the vehicle parking function can include a parking camera function that controls side cameras, rear cameras, or 360-degree cameras to assist the user 120 when parking vehicle 105. Additionally or alternatively, the vehicle parking function can include a parking assist function that helps maneuver vehicle 105 into a parking area.

[0092] The vehicle entertainment function can control one or more entertainment-related functions of vehicle 105. For example, the vehicle entertainment function can include a radio function for controlling the radio or a media function for controlling another audio or visual media source. The vehicle entertainment function can control sound parameters (e.g., volume, bass, treble, speaker distribution) or select radio stations or media content types / sources.

[0093] Each vehicle function may include controllers 170A - 170C associated with that particular vehicle function 165A - 165C. The controllers 170A - 170C for a particular vehicle function may include control circuitry configured to operate its associated vehicle function 165A - 165C. For example, the controller may include circuitry configured to turn on a seat heating function, turn off a seat heating function, set a particular temperature or temperature level, etc.

[0094] In one embodiment, the controllers 170A - 170C for a particular vehicle function may include a sensor or otherwise be associated with a sensor that collects data indicative of whether the vehicle function is on or off, the settings of the vehicle function, etc. For example, the sensor may be an audio sensor or a motion sensor. The audio sensor may be a microphone configured to collect audio input from the user 120. For example, the user 120 may provide a voice command to activate the radio function of the vehicle 105 and request a particular station. The motion sensor may be a vision sensor (e.g., a camera), an infrared sensor, a RADAR sensor, etc., configured to collect gesture input from the user 120. For example, the user 120 may provide a gesture motion to adjust the temperature function of the vehicle 105, thereby reducing the temperature inside the vehicle.

[0095] The controllers 170A - 170C may be configured to: transmit a signal to the control circuit 135 or another in - vehicle system. The signal may encode data associated with the corresponding vehicle function. The encoded data may indicate, for example, function settings, timing, etc.

[0096] The user 120 may interact with the vehicle functions 165A - 165C via user input. The user input may specify the settings of the vehicle function 165A - 165C selected by the user ("user - selected settings"). In one embodiment, the vehicle functions 165A - 165C may be associated with a physical interface, such as, for example, a button, a knob, a switch, a lever, a touch - screen interface element, or other physical mechanism. The physical interface may be physically manipulated to control the vehicle function 165A - 165C according to the user - selected settings. As an example, the user 120 may physically manipulate a button associated with the seat massage function to set the seat massage function to a five - level massage intensity. In one embodiment, the user 120 may interact with the vehicle functions 165A - 165C via user interface elements presented on a user interface of a display device (e.g., of the head unit infotainment system).

[0097] The techniques of the present disclosure can utilize artificial intelligence including machine learning models to learn routines or patterns of a user 120 interacting with vehicle functions, and generate a database of actions of the vehicle functions that can be automatically executed by vehicle 105 for a specific user 120. These automatically executable actions can be referred to as "automated vehicle actions". User 120 can authorize and activate computing system 130 to collect data and generate these automated vehicle actions. Such authorization / activation can be provided via a user input to a user interface of a display device (e.g., the infotainment system of vehicle 105). The techniques for generating automated vehicle actions will now be described in more detail.

[0098] Figure 2 FIG. illustrates an example computing architecture 200 for generating automated vehicle actions according to one embodiment of the present disclosure. Architecture 200 can include: (i) various databases for storing information; (ii) services for performing automated tasks, responding to hardware events, providing data, listening for data requests from other software, etc.; and (iii) software clients. In one embodiment, the services and clients can be implemented as modules within their respective computing systems. For example, the services and clients can be implemented as modules on vehicle 105 (e.g., within computing system 130) or away from vehicle 105 (e.g., within computing platform 110).

[0099] Computing system 130 can include various services and databases that can be implemented on vehicle 105 for generating automated vehicle actions based on learned routines of user 120. In one embodiment, computing system 130 can include: vehicle function services 205A - 205C, embedded learning service 210, model database 215, vehicle action manager 220, vehicle embedded service 225, and automated vehicle action database 230.

[0100] The vehicle function services 205A - 205C can be configured to: listen for data associated with the vehicle functions 165A - 165C. In one embodiment, the computing system 130 can include one vehicle function service 205A - 205C for each vehicle function 165A - 165C. The vehicle function services 205A - 205C can listen for context data associated with the respective vehicle functions (e.g., via the controllers 170A - 170C, associated sensors, etc.). The context data can indicate the settings of the vehicle functions 165A - 165C selected by the user 120 ("user - selected settings") and the conditions observed at the user - selected settings ("observed conditions"). The user - selected settings can include: (i) on / off selection; (ii) open / close selection; (iii) temperature selection; (iv) function - specific program / setting selection (e.g., seat massage level selection, seat heating temperature level selection), or another type of selection. The vehicle function services 205A - 205C can be configured to communicate the context data to the embedded learning service 210 or the vehicle - embedded service 225.

[0101] The embedded learning service 210 can be configured to: learn routines or patterns of the user's interaction with specific vehicle functions 165A - 165C and generate automated vehicle actions using one or more models. The embedded learning service 210 can access models from the model database 215. The model database 215 can store models that can be specific to the respective vehicle functions among the various vehicle functions 165A - 165C of the vehicle 105. In one embodiment, the model database 215 can store data structures that index or classify these models based on the associated vehicle functions 165A - 165C of the models. The embedded learning service 210 can access a specific model based on the vehicle functions 165A - 165C indicated in the received context data.

[0102] The embedded learning service 210 can include a learning kernel that continuously trains and predicts user routines. These learning results can be translated into automated vehicle actions and recommended to the user 120. The process / data pipeline by which the embedded learning service 210 analyzes context data to generate automated vehicle actions will be described in more detail below with reference to Figure 3 The process / data pipeline by which the embedded learning service 210 analyzes context data to generate automated vehicle actions will be described in more detail below with reference to

[0103] Still referring to Figure 2, the vehicle action manager 220 can be configured to: manage automated vehicle actions. In one embodiment, the vehicle action manager 220 may include services for performing its management responsibilities. The management of automated vehicle actions may include coordinating, for example, the creation and modification of automated vehicle actions, conflict analysis, persistence of automated vehicle actions, situational observation, and scheduling of automated vehicle actions. The vehicle action manager 220 can be programmed to implement a collection of components and libraries for managing the generation of automated vehicle actions. In some specific implementations, the vehicle action manager 220 may include a framework that utilizes one or more software factories to return one or more objects for use by the vehicle action manager 220.

[0104] In one embodiment, the vehicle action manager 220 may provide one or more software development kits (SDKs) that help allow vehicles (e.g., their clients and services) to generate and execute automated vehicle actions. For example, the SDK may include: a standardized object library based on interface definitions, client objects for establishing communication to another client or device (e.g., IPC communication to a cloud platform), client authentication mechanisms, standardized logging and metrics (analytics) hooks and tools, and / or other factors.

[0105] In one embodiment, the vehicle action manager 220 may include a client interface to the services of the vehicle action manager 220. For example, the vehicle action manager 220 may include a client interface that is configured to establish a client connection to the in-vehicle services of the vehicle action manager 220. This may include, for example, using inter-process communication (IPC) such as Unix domain sockets (UDS) or message queues (mqueue) to establish a connection to the service. In one embodiment, the manager client may not utilize client authentication on the vehicle 105. For example, the client-service relationship may be established at software build time such that the ability to interact with the services of the vehicle action manager 220 can be provided to client processes linked to the SDK.

[0106] The vehicle embedded service 225 can be a service for synchronizing, maintaining, and managing the execution of automated vehicle actions. In one embodiment, the vehicle embedded service 225 can provide APIs and bridges for various clients to the vehicle 105 to infer context data (e.g., as data points) and execute automated vehicle actions. For example, the vehicle embedded service 225 can maintain an automated vehicle action database 230. As will be further described herein, the automated vehicle action database 230 can store data structures that include command instructions for automated vehicle actions associated with a particular user 120 or user profile. In one embodiment, the automated vehicle action database 230 can concurrently store automated vehicle actions for more than one user (or user profile). The vehicle embedded service 225 can be configured to: receive data from the vehicle function services 205A - 205C and determine whether any trigger conditions for the stored automated vehicle actions exist. The vehicle embedded service 225 can be configured to: in the presence of a trigger condition, send a signal to control the vehicle functions 165A - 165C according to the automated vehicle action, as will be further described herein.

[0107] The vehicle embedded service 225 can be configured to: synchronize automated vehicle actions with a computing system remote from the vehicle 105. For example, the vehicle embedded service 225 can be configured to: send data indicating an automated vehicle action generated on the vehicle 105 to the computing platform 110 (e.g., a cloud - based server system).

[0108] The computing platform 110 can include various services and databases that can be implemented on the servers of the computing platform to support the management and generation of automated vehicle actions. In one embodiment, the computing platform 110 can include: a cloud embedded service 235 and a cloud database 240.

[0109] The cloud embedded service 235 can be a service for synchronizing, maintaining, and managing automated vehicle actions in a system remote from the vehicle 105. In one embodiment, the cloud embedded service 235 can provide APIs for various clients to manage automated vehicle actions. Possible clients can include, for example, services running on the vehicle 105, mobile software applications (e.g., iOS, Android), or web applications.

[0110] In one embodiment, the cloud-embedded service 235 may include or otherwise be associated with a cloud manager (outside the vehicle), which is configured to perform operations and functions similar to those of the vehicle action manager 220. For example, the cloud manager may include a client that is configured to establish a client connection to the cloud manager service (e.g., connect to the service using a TCP-based protocol such as HTTP). In some embodiments, client authentication may be required to establish the connection. This may include, for example, using a token-based authentication scheme.

[0111] The cloud-embedded service 235 may be configured to: maintain a data structure that identifies automated vehicle actions for a particular user 120. This may include, for example, receiving data indicating an automated vehicle action generated on the vehicle 105, identifying a particular user profile 245 of the user 120 of the vehicle 105 on which the automated vehicle action was generated, and providing data indicating the automated vehicle action for storage in the cloud database 240 in a manner associated with the user profile 245. The cloud-embedded service 235 may identify the user profile 245 from a plurality of user profiles based on data provided from the vehicle 105. This may include encrypted pseudonymized data (e.g., an encrypted user ID) associated with the user 120, which may be decrypted and used with a lookup function to access the appropriate user profile 245. The cloud-embedded service 235 may be configured to: update the cloud database to include new automated vehicle actions or remove automated vehicle actions (e.g., when a user disables or deletes an action).

[0112] The cloud database 240 may store information for multiple users. For example, the cloud database 240 may store a plurality of data structures that include automated vehicle actions. The corresponding data structures may include a table or list of automated vehicle actions associated with a particular user profile. The table / list may index the automated vehicle actions according to vehicle functions 165A - 165C. The corresponding data structures may be adjusted to reflect an updated representation of the automated vehicle actions associated with a particular user profile (e.g., when a new action is generated, a previous action is removed, etc.).

[0113] In one embodiment, the cloud - embedded service 235 may be configured to: provide data indicating a user profile and its associated vehicle actions to the vehicle 105. For example, when the user 120 enters the vehicle, the user 120 may be identified by the vehicle - embedded service 225 (e.g., based on the user's key or a handshake between the user device and the vehicle 105, a user profile selection on the host unit display). The user 120 may be different from the previous user operating the vehicle 105. The vehicle - embedded service 225 may send anonymized data indicating the user and request data indicating the user profile 245 of the user 120 (and the automated vehicle actions associated with that user profile). The cloud - embedded service 235 may receive the request, access the cloud database 240 to extract the requested data, and send data indicating the requested user profile 245 (and the automated vehicle actions associated with that user profile) to the vehicle - embedded service 225. The vehicle - embedded service 225 may store data indicating the automated vehicle actions associated with the user 120 in the automated vehicle action database 230 (e.g., as an active user of the vehicle 105).

[0114] In one embodiment, the vehicle - embedded service 225 may request more than one user profile from the cloud - embedded service 235. For example, two users may enter the vehicle 105: a first user 120 as the driver and a second user 175 as the passenger (as Figure 1 shown). The computing system 130 may detect the presence of the first user based on a handshake between the first user's key (or mobile device) and the vehicle 105, or the first user 120 may provide user input to the display device of the vehicle 105 to select the first user's user profile. The computing system 130 may detect the presence of the second user 175 based on a handshake between the second user's key (or mobile device) and the vehicle 105, or the second user 175 may provide user input to the display device of the vehicle 105 to select the second user 175's profile. In response, the computing system 130 may send a request for the first user profile of the first user 120 and the second user profile of the second user 175 to the cloud - embedded service 235. The cloud - embedded service 235 may extract data indicating the first user profile and the second user profile from the cloud database 240 and send the profile data to the computing system 130. As will be described herein, concurrently leveraging multiple user profiles on the vehicle 105 may allow the embedded learning service 210 to learn user routines for two different users during the same (or overlapping) time period.

[0115] The cloud - embedded service 235 can be configured to: track the performance of a model used to generate automated vehicle actions. For example, the cloud - embedded service 235 can be configured to: track the frequency with which automated vehicle actions generated by the embedded learning service 210 are rejected by the user 120.

[0116] Figure 3 FIG. illustrates an example data pipeline 300 for using multiple models to generate automated vehicle actions according to an embodiment of the present disclosure. The following description of the data flow in the data pipeline 300 is described with an example implementation in which the computing system 130 uses multiple models to generate automated vehicle actions on the vehicle 105. Additionally or alternatively, one or more portions of the data flow in the data pipeline 300 can be implemented in the computing platform 110.

[0117] The computing system 130 can receive context data 305 associated with multiple user interactions with vehicle functions 165A - 165C of the vehicle 105 over multiple time instances. Vehicle - function services 205A - 205C associated with the respective vehicle functions 165A - 165C can provide the context data 305. User interactions can include the user 120 interacting with the interfaces (e.g., physical buttons, soft buttons) of the vehicle functions 165A - 165C to adjust the vehicle functions 165A - 165C according to the settings 310 selected by the user. This can include turning the vehicle functions 165A - 165C to an "on" state, an "open" state, setting an intensity level, etc.

[0118] The context data 305 can indicate the settings 310 selected by multiple users across multiple time instances. Each respective time instance can be a point in time, a time range, a time of day, a phase of the day (e.g., morning, noon, afternoon, evening, late night), a day of the week, a week, a month, a date, etc. The respective time instance can indicate the time when the user interaction occurred (e.g., the time when the vehicle - function services 205A - 205C detected the user interaction) or the time when the context data was sent or received by the embedded learning service 210.

[0119] The context data 305 can indicate one or more observed conditions 315 associated with the settings 310 selected by the user. The observed conditions 315 can be collected at the time instance associated with the user interaction. The computing system 130 can receive data indicating the observed conditions 315 via multiple sensors 150 or systems (e.g., the positioning system 155) of the vehicle 105. In one embodiment, the data indicating the observed conditions 315 can be provided by the associated vehicle - function services 205A - 205C.

[0120] The observed condition 315 may indicate certain conditions that occur when the user 120 interacts with the corresponding vehicle functions 165A - 165C. For example, the observed condition 315 may include at least one of the following: (i) the date and / or time of day when the corresponding user interaction with the vehicle functions 165A - 165C occurs; (ii) the location where the corresponding user interaction with the vehicle functions 165A - 165C occurs; (iii) the route on which the corresponding user interaction with the vehicle functions 165A - 165C occurs; or (iv) the temperature when the corresponding user interaction with the vehicle functions 165A - 165C occurs. The route may be a route that the user requests via the vehicle's navigation function and that the user is following to a destination. In one embodiment, the observed condition 315 may include other information, such as, for example, weather conditions, traffic conditions, etc.

[0121] Figure 4 Illustrates example context data 305 associated with multiple user interactions with the vehicle functions 165A - 165C of the vehicle 105 over multiple time instances. Figure 4 Each row shown may represent a corresponding user interaction. As shown, each user interaction may be associated with a time (e.g., date and time) and include user - selected settings 310 for the vehicle functions 165A - 165C. As an example, on Monday, user 120 manually sets the seat ventilation to level three, turns on the seat massage function and sets it to "classic massage", and opens the driver - side window. These user - selected settings 310 are each recorded along with the observed condition 315. As shown in the example, each user - selected setting 310 is associated with the observed date, time, location (e.g., latitude / longitude coordinates), and temperature.

[0122] Return Figure 3 , the computing system 130 may utilize a machine - learning model to determine whether there are routines or patterns within the context data 305. For example, the computing system 130 may utilize a model such as the machine - learning clustering model 320. Additionally or alternatively, the computing system 140 may utilize a rule - based clustering model.

[0123] In one embodiment, the machine learning clustering model 320 can be an unsupervised learning model configured to group input data into multiple clusters such that similar data points are grouped together and different data points are distinguished. The machine learning clustering model 320 can use clustering analysis (e.g., probabilistic clustering) to process the context data 305 to generate user-activity clusters 500A-500C based on user interactions with specific vehicle functions 165A-165C. Using the machine learning clustering model 320 can include leveraging one or more clustering modeling techniques to process the context data 305. This can include applying one or more of the following to the context data 305: K-means clustering, KNN (k-nearest neighbor), K-medians clustering, expectation maximization model, hierarchical clustering, density-based spatial clustering of applications with noise (DBSCAN), ordering points to identify clustering structure (OPTICS), anomaly detection, principal component analysis, independent component analysis, apriori algorithm, or other means. The machine learning clustering model 320 can include multiple hyperparameters (such as, for example, multiple clusters) or be otherwise affected by the multiple hyperparameters.

[0124] The unsupervised learning model can be trained based on training data that does not include labels (or includes only a small number of labels). Training can include methods such as the following: Hopfield learning rule, Boltzmann learning rule, contrastive divergence, wake-sleep, variational inference, maximum likelihood method, maximum a posteriori probability, Gibbs sampling, or backpropagation reconstruction error or hidden state reparameterization. Further description of the training of the machine learning clustering model 320 is provided below with reference to Figure 12 the training of the machine learning clustering model 320.

[0125] As described herein, the model database 215 can include multiple machine learning clustering models. Each machine learning clustering model can be associated with a specific vehicle function 165A-165C. By assigning each machine learning clustering model to a specific vehicle function, each machine learning clustering model may be able to focus its learning on data associated with a single vehicle function rather than trying to learn patterns and identify clusters for multiple vehicle functions. This can improve model confidence as well as the accuracy of the clusters generated by the dedicated models.

[0126] The computing system 130 can select, invoke, or otherwise implement the machine learning clustering model 320 from among multiple machine learning clustering models (e.g., stored in the model database 215) based on one or more vehicle functions 165A-165C indicated in the context data 305. For example, given Figure 4For the example scenario data 305 shown in the figure, the computing system 130 may select at least one of the following: (i) a machine learning clustering model for the seat ventilation function; (ii) a machine learning clustering model for the seat massage function; or (iii) a machine learning model for the window function. The computing system 130 may select various models by parsing the scenario data 305 to determine the data type and determine the corresponding model for that data type. The computing system 130 may select various models by calling (e.g., using a function call) a specific model for processing a specific part of the scenario data 305 (e.g., calling the corresponding model for the corresponding column of list data, etc.).

[0127] The computing system 130 may use the machine learning clustering model 320 to generate user - activity clusters 500A - 500C for vehicle functions 165A - 165C based on the scenario data 305. The machine learning clustering model 320 may be configured to identify user - activity clusters 500A - 500C based on at least a portion of the user - selected settings 310 and at least a portion of one or more observed conditions 315 associated with the corresponding user - selected settings. More specifically, the machine learning clustering model 320 may be configured to perform a clustering analysis to group (or partition) data points with shared attributes in order to infer an algorithmic relationship between the user - selected settings 310 and the observed conditions 315 for a particular vehicle function 165A - 165C.

[0128] As an example, referring to Figures 5A to 5D , the computing system 130 may record multiple data points over the course of a work week. Each data point may be associated with a specific user interaction with vehicle functions 165A - 165C and may represent a user - specific variable for a given user interaction. For example, each data point may indicate an observed condition (e.g., time, location, temperature) and may include metadata indicating the user - selected settings 310 for the corresponding vehicle function. For example, Figure 5A shows an example data point 505 recorded for a user interaction with the ventilation function (e.g., set to level three). Figure 5B shows an example data point recorded for a user interaction with the seat massage function (e.g., set to "classic massage"). Figure 5C shows an example data point recorded for a user interaction with the window function (e.g., opening the window). Figure 5D shows an example data point recorded for a user interaction with the preparation function (e.g., selecting an ambient light setting).

[0129] The machine learning clustering model 320 can be configured to use probabilistic clustering to process the context data 305 (e.g., the context data generating the data points 505) to generate user - activity clusters 500A - 500C. The user - activity clusters 500A - 500C can be specific sets of values of context features based on time, location, and other observed conditions 315. The user - activity clusters 500A - 500C group together the data points 505 of the context data 305 that indicate sufficient generality among themselves to represent the personalized routines of the user 120. In one implementation, the machine learning clustering model 320 can apply a probability distribution to the data points, which can describe the probability (e.g., confidence) of the data points belonging within the user - activity clusters 500A - 500C. As an example, the machine learning clustering model 320 can be configured to generate a first user - activity cluster 500A indicating the routine of the user 120 when interacting with the seat ventilation function. The first user - activity cluster 500A can be considered complete (or closed) when the model has analyzed enough data points such that it has a threshold confidence level that those data points will be grouped as the first user - activity cluster 500A. A similar process can be utilized to generate a second user - activity cluster 500B for the seat massage function and a third user - activity cluster 500C for the window function.

[0130] The machine learning clustering model 320 can be configured to determine the boundaries of the user - activity clusters 500A - 500C for vehicle functions based on one or more observed conditions 315. For example, the boundaries of the user - activity clusters 500A - 500C can be defined by the outermost time, location, and temperature of the data points included in the respective user - activity clusters 500A - 500C. The boundaries of the user - activity clusters 500A - 500C can be defined by a cut - off probability of the probability distribution along the dimensions of the user - activity clusters 500A - 500C. The boundaries of the user - activity clusters 500A - 500C can be defined by the distance from an anchor point (e.g., centroid, etc.) of a given cluster. The boundaries of the user - activity clusters 500A - 500C can be implicitly encoded in the parameters of a machine learning classification model that is configured to receive a set of input context data and output a classification (e.g., cluster classification).

[0131] In one implementation, the machine learning clustering model 320 may not generate a user - activity cluster if it has not reached the threshold confidence level. As an example, as Figure 5D illustrated, the context data 305 only includes two user interaction data points for the preparation function within the represented time range. Thus, the machine learning clustering model 320 may not yet have enough data / confidence to generate a user - activity cluster that potentially represents the routine for the preparation function.

[0132] ReturnFigure 3 , computing system 130 can determine automated vehicle actions 330 for the vehicle functions 165A - 165C based on the user - activity clusters 500A - 500C for the respective vehicle functions 165A - 165C. The automated vehicle actions 330 can be considered skills / routines of the vehicle 105 that describe the automated execution of vehicle functions 165A - 165C based on the context. For example, the automated vehicle actions 330 can indicate automated settings 335 for the vehicle functions 165A - 165C and one or more trigger conditions 340 for automatically implementing the automated settings 335. More specifically, the automated vehicle actions 330 can define the relationship between one or more trigger conditions 340 and one or more settings 335 of the vehicle functions 165A - 165C. As an example, as further described herein, the automated vehicle actions 330 can include logical statements (e.g., if / then statements) or learning model outputs that indicate that if the trigger condition 340 is detected, the vehicle 105 will automatically control the vehicle functions 165A - 165C according to the automated settings 335 (e.g., to activate the settings).

[0133] In one implementation, to assist in determining the automated vehicle actions 330, the computing system 130 can use or otherwise utilize a vehicle action model 325. The vehicle action model 325 can be responsible for learning the user 120's preferences in different settings / procedures associated with the vehicle functions 165A - 165C.

[0134] The vehicle action model 325 can include a weighting algorithm that can be applied to the learned user - activity clusters 500A - 500C. For example, the vehicle action model 325 can be a rule - based model that is configured to apply respective weights 345 to each of the respective user - selected settings 310 within the user - activity clusters 500A - 500C. The weights 345 can be, for example, numbers assigned to each user - selected setting 310 to indicate the importance of that user - selected setting in terms of user preferences.

[0135] Each user-activity cluster 500A-500C may have an independent set of weights 345 associated therewith, which corresponds to vehicle functions 165A-165C. For example, the set of weights 345 applied to the first user-activity cluster 500A associated with the seat ventilation function may be different from the set of weights 345 applied to the second user-activity cluster 500B associated with the window function. The seat ventilation function may have several ventilation levels (e.g., level 1, level 2, level 3, level 4, level 5, etc.), while the window function may include two states (e.g., open state, closed state). Thus, in one embodiment, a higher additional weighting may be performed on the first user-activity cluster 500A associated with the seat ventilation function than on the second user-activity cluster 500B associated with the window function.

[0136] The computing system 130 may use the vehicle action model 325 to process the user-activity clusters 500A-500C and determine the automated settings 335 for the vehicle functions 165A-165C based on the weights 345 of the corresponding user-selected settings 310 within the user-activity clusters 500A-500C. In one embodiment, the weights 345 for each possible setting of the vehicle functions 165A-165C may be set to the same value at the time of cluster creation. When the user interacts with the vehicle functions 165A-165C to make the user-selected settings 310, the vehicle action model 325 may use a set of rules to adjust / update the weights 345 of the user-selected settings 310.

[0137] These rules may be based on the type of interaction of the user 120 with the vehicle functions 165A-165C. For example, a set of rules for adjusting the weights 345 may include a positive interaction rule, a negative interaction rule, and a neutral interaction rule. In one embodiment, the positive interaction rule may indicate that when the user 120 activates (e.g., turns on, opens) a particular user-selected setting 310, the weight of that user-selected setting 310 will be increased (e.g., +1). In one embodiment, the negative interaction rule may indicate that when the user 120 deactivates (e.g., turns off) a particular user-selected setting 310 or performs an action that is explicitly opposite to the positive interaction (e.g., closes the window), the weight of that user-selected setting 310 will be decreased (e.g., -1).

[0138] In one embodiment, a neutral interaction rule may indicate that the weight 345 of a setting 310 selected by a particular user will not change based on the user's interaction. This may occur when the conditions of user-activity clusters 500A - 500C occur but user 120 does not perform an action that has historically been performed in that context. As an example, when user 120 neither actively closes nor opens a window, but rather does not interact with the window and keeps the state unchanged, despite the occurrence of an observed condition 315 (falling within the scope of user-activity cluster 500C), a neutral interaction may be identified. In some implementations, a neutral interaction may be considered negative reinforcement (e.g., for seat heating, ventilation).

[0139] In one embodiment, a vehicle action model 325 may be configured to represent user interactions with vehicle functions 165A - 165C as a distribution. For example, user interactions may be modeled as a multinomial distribution. A multinomial distribution may include the distribution of the observed counts for each possible category in a set of categorical distributions. In this approach, the weight 345 may be parameterized by a Dirichlet distribution (e.g., the conjugate prior of a multinomial distribution).

[0140] In one embodiment, each respective user-activity cluster 500A - 500C may have an independent Dirichlet distribution corresponding to user interactions with the associated vehicle functions 165A - 165C. The parameters of the Dirichlet distribution may contain information representing the confidence in a particular user-selected setting 310 that user 120 is using and the probability of that particular user-selected setting relative to other settings for the corresponding vehicle functions 165A - 165C. The mean of the parameters for each setting / program of vehicle functions 165A - 165C may constitute the likelihood that the vehicle function is being used by user 120 within the user-activity cluster 500A - 500C (e.g., given the observed conditions). In one embodiment, for each vehicle function 165A - 165C having an "X" number of settings / programs, the Dirichlet distribution for each user-activity cluster 500A - 500C may have (X + 1) number of parameters: one parameter for each possible setting / program and one additional parameter for the "off" setting.

[0141] Parameters (e.g., weights) may be updated based on user interactions with the associated settings / programs, following the positive and negative rules described above. In one example, positive reinforcement may correspond to a gradual increase in the weight of the setting 310 selected by the user within the user-activity cluster 500A - 500C, and negative reinforcement may correspond to a gradual increase in the weight of the "off" setting.

[0142] The vehicle action model 325 can be configured to generate distributions of a similar type to identify the day-of-week preferences within each user-activity cluster. For example, the seven days of the week can also be modeled as a Dirichlet distribution. The vehicle action model 325 can initialize the weights 345 for each day with the same weight value and update the weight 345 for that day based on the occurrence of user interactions on the corresponding day.

[0143] In one embodiment, the vehicle action model 325 can be configured to determine an automated vehicle action 330 based on a threshold 350. The threshold 350 can include a preset threshold that represents the minimum probability required before determining the automated vehicle action 330 (e.g., its automation setting 335). For example, the threshold 350 can be expressed as the minimum value that the weight 345 of a setting 310 selected by a particular user must reach before the vehicle action model 325 determines that it should be used as the basis for the automation setting 335.

[0144] An example of how the computing system 130 can use the vehicle action model 325 to process user-activity clusters 500A - 500C is provided below. In this example, the seat massage function can include three possible settings for activating the seat massage function: a "gentle" setting, a "classic" setting, and a "strong" setting, as well as an "off" setting. An initial weight of 1 can be assigned to each of these settings. This can be represented as a Dirichlet distribution with concentration parameters [1, 1, 1, 1]. Within the same user-activity cluster (e.g., given similar observed conditions 315), user 120 can interact with the "gentle" setting once and with the "classic" setting four times. The "gentle" setting and the "classic" setting can be considered as the settings 310 selected by the user, and the computing system 130 can adjust their associated weights accordingly. For example, the computing system 130 (e.g., the vehicle action model 325) can update the corresponding Dirichlet parameters to [1 + 1, 1 + 4, 1, 1] = [3, 5, 1, 1]. These weight values can be converted to probability weights by dividing by the total weight (e.g., 3 + 5 + 1 + 1 = 10). Thus, the probability for each weight value can be [0.3, 0.5, 0.1, 0.1], indicating that the vehicle action model 325 has 30% confidence, 50% confidence, 10% confidence, and 10% confidence in each of the settings respectively. Therefore, the vehicle action model 325 can determine that the "classic" massage setting is the preferred setting for user 120 within the particular user-activity cluster 500A. Thus, the vehicle action model 325 can select the "classic" massage setting as the automation setting 335 for a given automated vehicle action 330.

[0145] In one embodiment, the action interpretation model 355 may construct a decision framework that interprets patterns of user activities revealed by the processed user-activity clusters 500A - 500C. The action interpretation model 355 may utilize rule-based techniques or heuristic-based techniques as well as machine learning approaches. The action interpretation model 355 may be configured to translate the user-activity clusters 500A - 500C and the automation settings 335 into automated vehicle actions 330 that simulate patterns of user activities. The action interpretation model 355 may be configured to assist in generating the automated vehicle actions 330 in a form that can be digested by the vehicle 105 or the user 120. This may include, for example, assisting in determining trigger conditions 340 for the automation settings 335 for automatically performing vehicle functions 165A - 165C.

[0146] In one embodiment, the action interpretation model 355 may operate directly on the context data 305 (e.g., accessing the user-selected settings 310). Additionally or alternatively, the action interpretation model 355 may probabilistically sample the simulated user activities within a cluster based on relative probabilities determined for various settings, optionally without accessing the user-selected settings 310. Such simulated user activities provide training data for training / testing the decision framework generated by the action interpretation model 355.

[0147] In one embodiment, the action interpretation model 355 may construct decision boundaries in the context feature space. The decision boundaries may provide interpretable ranges of parameter values for the context data, which together may approximate regions of the context feature space output by the machine learning clustering model.

[0148] In one embodiment, the action interpretation model 355 may use a rule-based approach to process the data, which discovers deterministic feature boundaries for the context features used in the clustering. The context features may include observed conditions 315 (e.g., latitude, longitude, temperature, time of day). For each user-activity cluster 500A - 500C, it may be assumed that the individual dimensions of a multi-dimensional Gaussian are independent Gaussian distributions. The feature boundaries may be defined by the range: [minimum, maximum] = [m - l·s, m + l·s], where m is the mean of the feature, s is the standard deviation of the individual distribution, and l is a hyperparameter that can take a value between 1 and 3 depending on the desired granularity level of the automated vehicle action 330. The minimum and maximum of the feature boundaries may be used to determine the trigger conditions 340 for the automation settings 335 for the vehicle functions 165A - 165C.

[0149] In one embodiment, the action interpretation model 355 may obtain decision boundaries that form a hyperrectangular body or hypercube in the situation-feature space. For example, the edges of the hyperrectangular body can be ranges of values of a given feature (e.g., the endpoints of the segments forming the edges are the minimum and maximum values of the range).

[0150] In one embodiment, the action interpretation model 355 may evaluate the relative impact of one or more situation features (e.g., one or more dimensions in the situation-feature space). For example, some situation feature dimensions may have no significant impact on the feature settings and may thus be deprioritized or ignored even if present in the determined clusters. For example, some features can be random, can be proxies for other features, or can be incompletely observed. For example, the action interpretation model 355 may determine that the time of day has less impact than the external temperature when it comes to internal heating settings. Thus, the action interpretation model 355 may determine to omit or relax the decision boundaries on the time axis in the situation-feature space for the heating settings. For example, even if the user has never turned on the heating at a time outside of a set of previously observed behaviors (e.g., 3:00 am), the action interpretation model 355 may determine decision boundaries that do not exclude the data point based on this anomalous time value when more influential criteria are met (e.g., the external temperature is below 32 degrees Fahrenheit).

[0151] In one embodiment, the action interpretation model 355 may include one or more hypercubes 360 that can be considered "approximators" for each user-activity cluster 500A - 500C in the user-activity clusters and a multicube 365 that can be defined as a set of hypercubes 360. The multicube can be a three-way bridge between the vehicle action model 325, the set of hypercubes 360, and the automated vehicle actions 330 presented to the user 120.

[0152] In one example, the multicube 365 may train all the hypercubes 360, obtain the learning results of the hypercubes 360, and present these learning results to the user 120. This can be achieved via a three-step method.

[0153] In the first stage, the multicube 365 may simulate usage data for each user-activity cluster 500A - 500C. The multicube 365 may simulate data that mimics different situation regions, which intelligently cover the phase space learned by the machine learning clustering model 320 using a customized Metropolis-Hastings inspired sampling algorithm. Each simulated data point can be a numerical vector corresponding to each situation feature being learned (e.g., the observed condition 315) and a "target" value indicating whether the user 120 is performing a user interaction in that situation space.

[0154] In a second stage, for each user-activity cluster 500A - 500C, data can be passed to a hypercube object that identifies a subset of the most important situational features in the user interactions with the corresponding vehicle functions 166A - 166C (e.g., the observed conditions that have the highest influence in user-activity clusters 500A - 500C). For example, for the seat heating function, the internal temperature and the external temperature can be two main factors that affect the seat heating use of a particular user 120. The most important features can be determined by a customized scoring function that uses: (i) the features with the highest variance; (ii) the features with a higher mutual information score relative to the "target" values in the first stage; and (iii) the features with a higher chi-square score relative to the "target" values in the first stage.

[0155] In a third stage, for each of these important features, the action interpretation model 355 can calculate minimum and maximum bounds. For example, as described herein, hypercubes 360 can be trained for each user-activity cluster 500A - 500C under a number of situational features (e.g., observed conditions 315). The hypercubes 360 can approximate these regions by dividing the situational regions learned by the machine learning clustering model 320 into a set of hyperrectangular bodies (e.g., higher-dimensional cubes). The hypercubes 360 can take as input the information generated by the machine learning clustering model 320 and the vehicle action model 325 and approximate this information as a higher-dimensional cube that can be presented to the user 120 as an automated vehicle action 330. In this way, in the second and third stages, the action interpretation model 355 can determine the preferred features that serve as the first side of the hyperrectangular body based on the customized scoring function, determine decision boundary rules that appropriately characterize the model output and maximize the information gain in each situational space using the preferred features and the target variable, and translate the discovered rules / limits into the boundaries of the sides of the hyperrectangular body.

[0156] In one implementation, the computing system 130 can utilize additional or alternative statistical approaches to determine the trigger condition 340. For example, as described herein, the trigger condition 340 can be based on the observed conditions 315 within the user-activity clusters 500A - 500C. In one example, the trigger condition 340 can be defined by the minimum and maximum values of the observed conditions 315 within the user-activity cluster. As Figure 4As shown, the observed condition 315 for the seat ventilation function recorded within the user-activity cluster 500A may indicate that user 120 activates the three-stage ventilation setting every day when the temperature is between 21 degrees Celsius and 24 degrees Celsius. Accordingly, the trigger condition 340 for activating the three-stage ventilation setting as an automated setting may include detecting that the temperature is higher than 21 degrees Celsius and lower than 24 degrees Celsius. In some specific implementations, the computing system 130 may adjust the minimum and maximum values (e.g., + / - 5 degrees Celsius) based on the appropriateness for expanding (or narrowing) the trigger condition 340. In some specific implementations, the trigger condition 340 may include a threshold (based on the minimum value), upon exceeding which the automated setting 335 can be activated.

[0157] In one embodiment, the trigger condition 340 may be based on being within a specific range of the observed condition 315. For example, as Figure 4 shown, the observed condition 315 recorded within the user-activity cluster 500C for the seat massage function may indicate that user 120 activates the "classic" massage setting every day when the temperature is between 24 degrees Celsius and 25 degrees Celsius. Accordingly, the trigger condition 340 for activating the "classic" massage setting as an automated setting may include detecting that the temperature is within a specific range between 24 degrees Celsius and 25 degrees Celsius (e.g., +1 / - 5 degrees Celsius).

[0158] In one embodiment, the trigger condition 340 may be based on the mean or mode or centroid of the observed condition 315 within the user-activity clusters 500A - 500C. For example, the observed condition 315 recorded within the user-activity cluster 500B may indicate that user 120 opens the driver's side window when the vehicle 105 is within "X" meters of a central location (e.g., corresponding to the user's workplace). The central location may be determined based on the mean or mode of the latitude and longitude coordinates within the observed condition 315 of the user-activity cluster 500B. In one example, the distance "X" may be defined by the maximum distance from the central location at which user 120 opens the driver's side window. The trigger condition 340 for automatically opening the driver's side window may include detecting that the vehicle 105 is within the "X" distance from the central location.

[0159] The computing system 130 may output an automated vehicle action 330. As described herein, the automated vehicle action 330 may include an automated setting 335 (e.g., the highest weighted user-selected setting 310) and one or more trigger conditions 340 that indicate the conditions under which the vehicle 105 will automatically activate the automated setting 335. This may be expressed, for example, as a logical relationship: "When (the trigger condition occurs), then (the automated setting) is applied to the associated vehicle function." In one implementation, the computing system 130 may assign a name to the automated vehicle action 330 that reflects the associated vehicle function (e.g., ventilation), automated setting (e.g., level three), trigger condition (e.g., nearby location, time of day, temperature), or other characteristics.

[0160] The computing system 130 may notify the user 120 of the automated vehicle action 330. To do so, the computing system 130 may utilize an action interpretation model 355 that may be configured to generate content for presentation to the user 120 via a user interface of a display device. The content may be based on the described approach for generating the automated vehicle action 330 that is digestible by the user (e.g., by using the hypercube 360 to develop the content). The content may recommend the automated vehicle action 330 to the user and may request the user 120 to approve the automated vehicle action 330 (e.g., via user input to the display device). The content may indicate the vehicle functions 165A-165C, the automated setting 335, and the one or more trigger conditions 340 associated with the automated vehicle action 330. The display device may include, for example, a touchscreen of an infotainment system on the vehicle 105 or a touchscreen of the user device 115 of the user 120.

[0161] As an example, referring to Figure 6 , the user-activity cluster 500A may result in an automated vehicle action 600 for the seat ventilation function. If a trigger condition 610 (e.g., location, time, and temperature conditions) is detected, the automated vehicle action 600 may indicate that the automated setting 605 (e.g., level three ventilation) of the seat ventilation function will be activated. The automated vehicle action 600 may be presented as content on a user interface 615 of a display device 620 (e.g., on the vehicle 105). In one implementation, the content may be generated as a prompt for the user 120. The user 120 may interact with the user interface 615 (e.g., by providing a touch input to a user interface soft button element) to accept the automated vehicle action 600, manually adjust the trigger condition 610, ignore, disable, or delete the automated vehicle action 600, or select that the user 120 may decide whether to implement the automated vehicle action 600 at a later time.

[0162] In another example, Figure 7 an example is presented of content for presenting to user 120 regarding automated vehicle action 700. User-activity cluster 500B may result in automated vehicle action 700 for a seat massage function. If a trigger condition 710 (e.g., location, time, and temperature conditions) is detected, then automated vehicle action 700 may indicate that an automated setting 705 (e.g., a "classic" massage setting) for the seat massage function will be activated. Automated vehicle action 700 may be presented as content on user interface 715 of display device 720 (e.g., on vehicle 105). User 120 may interact with user interface 715 (e.g., via user interface elements) to accept automated vehicle action 700, manually adjust trigger condition 710, ignore, disable, or delete automated vehicle action 700, or select whether user 120 may decide to implement automated vehicle action 700 at a later time.

[0163] In another example, Figure 8 an example is presented of content for presenting to user 120 regarding automated vehicle action 800. User-activity cluster 500C may result in automated vehicle action 800 for a window function. If a trigger condition 810 (e.g., location, time, and temperature conditions) is detected, then automated vehicle action 800 may indicate that an automated setting 805 (e.g., opening the driver's window) for the window function will be activated. Automated vehicle action 800 may be presented as content on user interface 815 of display device 820 (e.g., on vehicle 105). User 120 may interact with user interface 815 (e.g., via user interface elements) to accept automated vehicle action 800, manually adjust trigger condition 810, ignore, disable, or delete automated vehicle action 800, or select whether user 120 may decide to implement automated vehicle action 800 at a later time.

[0164] Returning Figure 3 , in one embodiment, computing system 130 may determine whether there is a conflict between newly generated automated vehicle action 330 and pre-existing automated vehicle actions. Such conflict resolution may occur before the following actions: presenting automated vehicle action 330 to user 120; or storing automated vehicle action 330 in database 230 for vehicle 105 to execute.

[0165] To perform conflict resolution analysis, computing system 130 may utilize a conflict resolver 370. The conflict resolver 370 may be implemented as a module of the computing system 130. The conflict resolver 370 may be configured to determine whether an automated vehicle action 330 conflicts with another pre-existing automated vehicle action (e.g., stored in database 230). A conflict may be determined if vehicle 105 cannot perform both automated vehicle actions without modifying one of them.

[0166] The conflict resolver 370 may determine whether there is a conflict between automated vehicle actions in various ways. In one implementation, the conflict resolver 370 may determine that there is a conflict when two automated vehicle actions are assigned the same name. The conflict resolver 370 may be configured to detect name conflicts using string comparison or substring checking analysis.

[0167] In one implementation, the conflict resolver 370 may determine that there is a conflict between automated vehicle actions based on a domain associated with the context of the automated vehicle action 330 or the vehicle functions 165A - 165C to be controlled. The conflict domain may be identified by the association of the controller 170A - 170C (or ECU) with the execution of the automated vehicle action 330. The computing system may determine that there is a conflict when the vehicle actions involve the same controller 170A - 170C (or ECU) and the controller 170A - 170C cannot implement two automated settings concurrently. For example, climate-based automated vehicle actions cannot set the climate control temperature to maximum heating and maximum cooling simultaneously.

[0168] In one implementation, the conflict resolver 370 may determine whether there is a conflict between the automated vehicle action 330 and an action taken by vehicle 105 due to an explicit command from the user. For example, user 120 may provide audio input (e.g., a verbal voice command) to vehicle 105 to perform navigation guidance to a place of interest (e.g., a restaurant). The trigger condition 340 of the automated vehicle action 330 may indicate that an automatic navigation setting for guiding the user to work will be executed simultaneously. Thus, the conflict resolver 370 may determine that there is a conflict between the voice-activated navigation and the automated vehicle action 330.

[0169] The computing system 130 can resolve conflicts based on one or more conflict resolution strategies 375. The strategies 375 can be programmed to automatically resolve which automated vehicle actions to enable and which to disable. An example strategy can include enabling the most recently determined automated vehicle action and disabling another automated vehicle action. Another example strategy can include enabling automated vehicle actions that are more likely to occur frequently (e.g., during a morning commute) and disabling another (e.g., one that is only activated during a specific season). Additionally or alternatively, another example strategy can include enabling or disabling automated vehicle actions based on a hierarchy of automated vehicle actions. Additionally or alternatively, another example strategy can include supporting the setting of vehicle functions according to the explicit commands of the user rather than performing automated vehicle actions. Additionally or alternatively, another example strategy can include supporting (e.g., via the computing system 130) automated vehicle actions created on the vehicle 105 rather than automated vehicle actions created outside the vehicle 105 (e.g., via the computing platform 110).

[0170] Additionally or alternatively, example strategies can be configured to help resolve conflicts based on the context of the vehicle 105. For example, a strategy can be configured to prevent the activation of a certain function given the weather, traffic conditions, noise level, or other current or future conditions of the vehicle 105. As an example, an automated vehicle action associated with opening a window (e.g., a sunroof) may not be activated if it is (or is predicted to be) raining, the noise is too high, etc. The prediction of rain, elevated noise levels, etc. can be determined based on data indicating the future operating conditions of the vehicle 105 (e.g., weather data forecasting rain, route data showing a route through a noisy area). This can help resolve conflicts between automated vehicle actions by supporting automated vehicle actions that are more appropriate according to the context of the vehicle.

[0171] In one implementation, if none of the strategies 375 automatically resolve the conflict, content can be presented to the user 120 on the user interface to manually resolve the conflict. The content can include prompts asking the user which automated vehicle actions to enable and which to disable. The disabled automated vehicle actions can remain disabled until, for example, they are manually adjusted by the user 120.

[0172] The computing system 130 may output command instructions 380 based on the automated vehicle action 330. The command instructions 380 may be computer-executable instructions for the vehicle 105 to implement the automated vehicle action 330 based on whether the vehicle 105 detects one or more trigger conditions 340, so as to automatically control the vehicle functions 165A - 165C according to the automation setting 335.

[0173] In one embodiment, the computing system 130 may store the command instructions 380 in an accessible memory on the vehicle 105 for execution at a later time. For example, the command instructions 380 may be stored in the automated vehicle action database 230 together with command instructions associated with other automated vehicle actions. The database 230 may be updated when creating new automated vehicle actions, changing, disabling, enabling existing automated vehicle actions, etc.

[0174] The computing system 130 may execute the command instructions 380 at a later time. For example, at a later time, the computing system 130 may detect the occurrence of one or more trigger conditions 340. The detection may be based on signals generated by the vehicle functions services 205A - 205C. The signals may encode data indicating the trigger conditions 340 (e.g., time, temperature, location, weather, traffic, etc.). Based on the trigger conditions 340, the computing system 130 (e.g., the vehicle embedded service 225) may send signals for the vehicle 105 to implement the automation setting 335. The signals may be sent to the controllers 170A - 170C, which are configured to activate / adjust the vehicle functions 170A - 170C to the automation setting 335 (to start a classic massage program).

[0175] In one embodiment, the computing system 130 may generate content (e.g., a notification) indicating that the vehicle 105 is implementing or has implemented the automation setting 335. The content may be presented to the user 120 via a user interface of a display device (e.g., of the vehicle's infotainment system). In one embodiment, the user 120 may provide user input to stop, pause, or delay (e.g., to implement at another time), or cancel or disable the automated vehicle action 330.

[0176] In one embodiment, the command instructions 380 may be stored in one or more sets for indexing automated vehicle actions. The corresponding sets may include sets of automation settings 335, trigger conditions 340, etc.

[0177] For example, Figures 9A to 9BIllustrated is a data structure 900A - 900B that includes multiple automated vehicle actions. The data structure 900A can be, for example, a table, list, etc. that indexes corresponding automated vehicle actions. The automated vehicle actions can be represented as objects that store an automation settings object and a trigger condition object. Each object can also maintain a collection of metadata (such as a serial number, unique identifier, assigned name, etc.) for uniquely identifying the automated vehicle action. In one embodiment, the automated vehicle actions can be stored or indexed according to the type of action (e.g., ClimateControlAction, NavigationRouteAction, etc.). In one embodiment, the automated vehicle actions can be associated with an action affinity that defines the responsibility of the computing system 130 (e.g., vehicle - embedded service 225) or the computing platform 110 (e.g., cloud - embedded service 235) for a particular automated vehicle action.

[0178] The command instruction 380 can be stored in a manner associated with the user profile of the user 120. For example, the data structure 900A can be associated with a first user profile associated with the first user 120, and the data structure 900B can be associated with a second user profile associated with the second user 175 ( Figure 1 as shown). In this way, the database 230 can indicate which automated vehicle actions are associated with which users.

[0179] In one embodiment, the computing system 130 can send a communication indicating the command instruction 380 to the computing platform 110 (e.g., a server system) via the network 125 for storage in a manner associated with the user profile of the user 120 outside the vehicle 105. For example, the computing platform 110 can receive the communication and store the command instruction in the cloud database 240 in a manner associated with the user profile of the user 120. In the case where the user 120 enters the second vehicle 180 ( Figure 1 as shown), the computing platform 110 can provide the second vehicle 180 with data indicating the command instruction 380 for the automated vehicle actions associated with the user profile of the user 120. Thus, the user 120 can experience the automated vehicle actions determined by the first vehicle 105 while in the second vehicle 180.

[0180] In one embodiment, computing platform 110 may be configured to: aggregate autonomous vehicle actions across multiple users. For example, computing platform 110 (e.g., cloud - embedded service 235) may obtain data indicative of multiple autonomous vehicle actions generated by multiple different vehicles for multiple different users. Such data may be stored in, for example, cloud database 240. Computing platform 110 may analyze the data to generate an aggregated cluster that includes multiple autonomous vehicle actions from multiple different users. In one example, computing platform 110 may use clustering analysis and machine - learning models similar to those described previously herein to determine the aggregated cluster. The autonomous vehicle actions within the aggregated cluster may be related because they are associated with similar vehicle functions, have similar trigger conditions, etc. Additionally or alternatively, the users associated with the cluster may have common attributes, such as being located in the same or similar geographical regions, being under similar weather conditions, etc.

[0181] Computing platform 110 may be configured to: recommend autonomous vehicle actions to a user based on the analysis of the aggregated cluster. For example, computing platform 110 may apply a vehicle - action model to weight multiple automated settings within the aggregated cluster based on the respective frequencies of that vehicle - action model within the aggregated cluster. Computing platform 110 may identify the automated setting with the highest weight and select that automated setting as the recommended automated setting. Using techniques similar to those described previously herein, computing platform 110 may determine the recommended trigger condition for the recommended automated setting based on the aggregated cluster (e.g., based on the trigger - condition actions included in the aggregated cluster). Computing platform 110 may generate a recommended vehicle action for user 120 based on the recommended automated setting and the recommended trigger condition.

[0182] Computing platform 110 may convey a communication indicative of the recommended vehicle action to vehicle 105 associated with user 120 or to user device 115 associated with user 120. The computing system 130 of vehicle 105 may receive the communication and process the communication for presentation to user 120. In one example, computing system 130 may generate content for presentation to user 120 via a display device on vehicle 105. The content may include the recommended vehicle action and prompt user 120 to approve, discard, or ignore the recommendation. In the case where user 120 approves, computing system 130 may store the recommended vehicle action as an autonomous vehicle action (e.g., in a manner associated with the user profile of user 120) in action database 230.

[0183] In one embodiment, the computing system 130 may send a communication to the computing platform 110 to indicate the user's approval of the recommended vehicle action. The computing platform 110 may store the recommended vehicle action as an automated vehicle action (e.g., in a manner associated with the user profile of the user 120) in the cloud database 240. Additionally or alternatively, the computing platform 110 may utilize the feedback to help retrain models for generating aggregated clusters, recommended vehicle actions, and the like.

[0184] In one embodiment, the user device 115 may additionally or alternatively perform the operations and functions of the computing system 130 with respect to the recommended vehicle action.

[0185] Figures 10A to 10B A flowchart illustrating an example method 1000 for generating automated vehicle actions in accordance with one embodiment of the present disclosure is shown. Method 1000 may be executed by a computing system described with reference to other figures. In one embodiment, method 1000 may be executed by the control circuit 135 of the Figure 1 computing system 130. One or more portions of method 1000 may be implemented as algorithms on hardware components of the devices described herein (e.g., as in Figures 1 to 3 , Figures 6 to 8 and Figure 12 ), for example, to generate automated vehicle actions as described herein. For example, the steps of method 1000 may be implemented as operations / instructions executable by computing hardware.

[0186] Figures 10A to 10B Elements are illustrated and discussed as being performed in a particular order for purposes of illustration. Those of ordinary skill in the art will understand that, in the context of using the disclosure provided herein, the elements of any of the methods discussed herein may be adapted, rearranged, extended, omitted, combined, or modified in various ways without departing from the scope of the present disclosure. Figures 10A to 10B is described with reference to elements / terms described with respect to other systems and figures for purposes of example illustration and is not meant to be limiting. One or more portions of method 1000 may additionally or alternatively be executed by other systems. For example, method 1000 may be executed by the control circuit 185 of the computing platform 110.

[0187] In one embodiment, method 1000 may begin or otherwise include step 1005, in which computing system 130 receives context data 305 associated with a plurality of user interactions with vehicle function 165B of vehicle 105 over a plurality of time instances. Context data 305 may include data indicating settings 310 for a plurality of user selections for vehicle function 165B and data indicating one or more observed conditions 315 associated with the settings 310 selected by the corresponding user. For example, user 120 may interact with a knob, button, etc. daily within a week to activate a massage setting of a seat massage function. Context data 305 may indicate a specific massage setting and the time, temperature, and location conditions when the setting was selected.

[0188] In one embodiment, method 1000 may include step 1010, in which computing system 130 selects machine learning clustering model 320 from a plurality of machine learning clustering models based on vehicle function 165B. As described herein, each corresponding machine learning clustering model 320 may be dedicated to a specific vehicle function 165A - 165C of vehicle 105. Computing system 130 may process context data 305 to identify (e.g., by identifying an indicator encoded in the data, identifying the vehicle function service providing the data) which vehicle function among vehicle functions 165A - 165C the context data belongs to and access the machine learning clustering model 320 associated with that vehicle function. In one example, in the case where context data 305 indicates a user interaction with a seat massage function, computing system 130 may access the machine learning clustering model 320 for the seat massage function.

[0189] In one embodiment, method 1000 may include step 1015, in which computing system 130 uses machine learning clustering model 320 to generate user-activity cluster 500B for vehicle function 165B based on context data 305. As described herein, machine learning clustering model 320 may be trained to identify user-activity cluster 500B based on at least a portion of user-selected settings 310 and at least a portion of one or more observed conditions 315 associated with the corresponding user-selected settings 310. For example, machine learning clustering model 320 may generate user-activity cluster 500B that includes certain data points 505, which are associated with user interactions with the seat massage function. Machine learning clustering model 320 may group these data points based on similarities or patterns among data points 505. This may include, for example, activating the seat massage function within a similar time range, temperature range, location range, etc. As described herein, machine learning clustering model 320 may apply a probability distribution to data points 505, which may help determine the probability (e.g., confidence) that a data point 505 belongs to user-activity cluster 500B (e.g., for the seat massage function).

[0190] In one embodiment, method 1000 may include step 1020, in which computing system 130 determines automated vehicle action 330 based on user-activity cluster 500B. Automated vehicle action 330 may indicate an automated setting 335 for vehicle function 165B (e.g., a vehicle comfort function) and one or more trigger conditions 340 for automatically implementing the automated setting 335. More specifically, as described herein, automated vehicle action 330 may be expressed as a dependency (e.g., an if / then statement) between the implementation of automated setting 335 and trigger condition 340.

[0191] In one embodiment, the computing system 130 may use the vehicle action model 325 to determine an automated vehicle action 330 in step 1020. The vehicle action model 325 may include, for example, a weighted algorithm that may be applied to the user-activity cluster 500B. A particular weighting scheme may be specific to the vehicle function 165B. In one example, the vehicle action model 325 may provide a weighting scheme specific to the seat massage function (e.g., a given number of massage settings). When the user interacts with the seat massage function, the vehicle action model 325 may assign weights to each user-selected setting 310. In one embodiment, the vehicle action model 325 may identify the user-selected setting 310 with the highest weight in the user-activity cluster 500B (e.g., the classic massage setting) as the setting preferred by the user 120. Thus, the computing system 130 may set the highest-weighted user-selected setting 310 as the automated setting 335 to be achieved when performing the automated vehicle action 330.

[0192] The computing system 130 may determine a trigger condition 340 for the automated setting 335 in step 1020 based on the user-activity cluster 500B. As described herein, the trigger condition 340 may be based on the observed conditions 315 that appear in the user-activity cluster 500B. In one example, the time, location, and / or temperature conditions for automatically activating the classic massage setting may be based on the observed time, location, and temperature at which the user 120 selects the setting over time.

[0193] In one embodiment, method 1000 may include step 1025, in which computing system 130 determines whether there is a conflict between automated vehicle action 330 and a pre-existing automated vehicle action. In one embodiment, computing system 130 may compare the new automated vehicle action 330 and the trigger conditions 340 of the pre-existing automated vehicle action to determine whether these actions can be triggered concurrently. If so, computing system 130 may compare the automation settings 335 of the newer automated vehicle action 330 with the automation settings of the pre-existing automated vehicle action to determine whether these settings can be implemented concurrently. For example, if the associated controller 170B (or ECU) cannot activate the settings during the overlapping time period, these settings may not be able to be implemented concurrently. As an example, the controller 170B for the seat massage function of the driver's seat may be programmed to activate only one massage setting at a time. Thus, controller 170B may not be able to activate the classic massage setting of the new automated vehicle action and the gentle massage setting of the pre-existing automated vehicle action (if these actions can be triggered by the same conditions). In this example, computing system 130 may determine that there is a conflict, and method 1000 may return to step 1005. In the absence of a conflict, method 1000 may continue.

[0194] In one embodiment, method 1000 may include step 1030, in which computing system 130 generates content for presentation to user 120 via the user interface of the display device based on automated vehicle action 330 and using action interpretation model 355. The content may indicate vehicle function 165B, automation settings 335, and one or more trigger conditions 340. Figure 7 Example content is shown, in which user interface 715 presents a prompt to user 120 that indicates that the classic massage setting will be activated for the seat massage function if certain location, time, and temperature conditions are detected. In one embodiment, user 120 may interact with user interface 715 to edit trigger conditions 710.

[0195] Now referring to Figure 10B , in one embodiment, method 1000 may include step 1035, in which computing system 130 receives user input indicating approval of automated vehicle action 330 by user 120. As described herein, the user input may include a touch input to a user interface element, a voice command, etc.

[0196] In one embodiment, user 120 may reject automated vehicle action 330. In the case of rejection, method 1000 may return to step 1005.

[0197] In one embodiment, method 1000 may include step 1040, in which computing system 130 retrains machine learning clustering model 320. For example, computing system 130 may include a feedback training loop in which machine learning clustering model 320 that generates relevant user-activity clusters 500B is trained / retrained based on the feedback received from user 120 in step 1035. For example, machine learning clustering model 320 may be retrained based on whether user 120 accepts or rejects automated vehicle action 330. This may include, for example (e.g., based on the feedback), modifying the hyperparameters of machine learning clustering model 320.

[0198] In one embodiment, method 1000 may include step 1045, in which computing system 130 outputs command instruction 380 for vehicle 105 to implement automated vehicle action 330 based on whether vehicle 105 detects one or more trigger conditions 340, so as to automatically control vehicle function 165B according to automation setting 335. Command instruction 380 may include computer-executable instructions that cause computing system 130 to monitor trigger conditions 340 (e.g., time, location, temperature, etc.), and if these trigger conditions are detected, implement automation setting 335 (e.g., activate the classic massage setting).

[0199] In one embodiment, method 1000 may include step 1050, in which computing system 130 stores command instruction 380 in an accessible memory on the vehicle for execution at a later time. For example, command instruction 380 may be stored in the memory on vehicle 105 (e.g., automated vehicle action database 230). Command instruction 380 (e.g., to activate automation setting 335) may be executed at a later time (e.g., when user 120 approves / enables automated vehicle action 330, trigger condition 340 is detected, etc.).

[0200] In one embodiment, method 1000 may include step 1055, in which the computing system 130 sends a communication indicating command instruction 380 over a network to a server system for storage in a manner associated with the user profile of user 120. As described herein, command instruction 380 may be provided to a computing platform 110 (e.g., a cloud-based server system). The computing platform 110 may store command instruction 380 in a memory outside of vehicle 105 (e.g., cloud database 240). Command instruction 380 may be stored in a manner associated with the user profile 245 of user 120 such that, if needed, the user's automated vehicle actions may be transferred from the computing platform 110 to one or more other vehicles. For example, the computing platform 110 may send data indicating automated vehicle action 330 to another vehicle (different from vehicle 105) such that the other vehicle may implement automated vehicle action 330 even though they were created by another vehicle (e.g., vehicle 105).

[0201] In one embodiment, method 1000 may include step 1060, in which the computing system 130 detects the occurrence of one or more trigger conditions 340. For example, the computing system 130 may collect data from sensors 150 (e.g., a thermometer) of vehicle 105 or other systems / devices on vehicle 105 (e.g., a clock, positioning system 155) to determine if a trigger condition 340 for automated vehicle action 330 has occurred.

[0202] In one embodiment, method 1000 may include step 1065, in which the computing system 130 sends a signal to implement an automated setting 335 for vehicle function 165B based on one or more trigger conditions 340. In one example, if the computing system 130 detects the occurrence of a defined set of time, temperature, and location conditions, the computing system 130 may automatically send a signal to controller 170B to indicate that the seat massage function will be set to a classic massage.

[0203] Figure 11 A flowchart illustration of an example method 1100 for implementing automated vehicle actions for a second user in accordance with one embodiment of the present disclosure. Method 1100 may be executed by a computing system described with reference to other figures. In one embodiment, method 1100 may be executed by the control circuit 135 of computing system 130. One or more portions of method 1100 may be implemented as an algorithm on hardware components of the devices described herein (e.g., as in Figure 1 ), Figures 1 to 3 Figures 6 to 8 Figure 12 ​​among others). For example, the steps of method 1100 may be implemented as operations / instructions executable by computing hardware.

[0204] Figure 11 Elements are illustrated for purposes of illustration and discussion in a particular order. In light of the disclosure provided herein, one of ordinary skill in the art will understand that, without departing from the scope of the present disclosure, the elements of any of the methods discussed herein may be adapted, rearranged, extended, omitted, combined, or modified in various ways. Figure 11 is described with reference to elements / terms described with respect to other systems and figures for illustrative purposes and is not meant to be limiting. One or more portions of method 1100 may additionally or alternatively be performed by other systems. For example, method 1100 may be performed by control circuitry 185 of computing platform 110.

[0205] In one implementation, method 1100 may begin with or otherwise include step 1105, in which computing system 130 receives data indicative of a second user profile of a second user 175 of vehicle 105. In one example, second user 175 may enter vehicle 105 to become the driver of vehicle 105, and another user (e.g., first user 120) may not be located in vehicle 105 with second user 175. In another example, second user 175 may be located in vehicle 105 with another user (e.g., first user 120). In such examples, second user 175 may be the driver or a passenger.

[0206] In one implementation, computing system 130 may receive data indicative of the second user profile due to detecting that second user 175 is located within vehicle 105. For example, as described herein, computing system 130 may identify the key or user device of the user of second user 175 (e.g., via a handshake process), or second user 175 may provide user input (e.g., voice input, touch input) to indicate that second user 175 has entered vehicle 105. In one implementation, computing platform 110 may receive data indicative of the presence of a user in vehicle 105 and respond by sending data indicative of an automated vehicle action associated with the second user profile to computing system 130 on vehicle 105. The automated vehicle action associated with the second user profile / second user 175 may be an action determined by a vehicle different from vehicle 105 (e.g., now being used by second user 175).

[0207] In one embodiment, method 1100 may include step 1110, in which computing system 130 stores command instructions for a second automated vehicle action associated with a second user profile in an accessible memory of vehicle 105. For example, computing system 130 may store command instructions for the second automated vehicle action in an automated vehicle action database 230 on vehicle 105.

[0208] The command instructions for the second automated vehicle action may be based on at least one user interaction of the second user with a vehicle function of another vehicle (e.g., vehicle 180). For example, second user 175 may interact with the seat massage function of second vehicle 180 at multiple time instances. The computing system (e.g., its control circuitry) of second vehicle 175 may be configured to: use the techniques and processes described herein to determine the second automated vehicle action based on the user interaction of second user 175 with the seat massage function of second vehicle 180. The computing system may output command instructions for the second automated vehicle action and send data indicative of the second automated vehicle action (e.g., the associated command instructions) to computing platform 110. As described herein, computing platform 110 may store such information and send the information to another vehicle (e.g., vehicle 105) in a manner associated with the second user profile.

[0209] In one embodiment, computing system 130 may not store command instructions for automated vehicle actions other than those associated with the automated vehicle actions of second user 175. This may occur, for example, when second user 175 is the only user in vehicle 105. Additionally or alternatively, this may occur when second user 175 is the only user detected in vehicle 105. Additionally or alternatively, this may occur when second user 175 is the driver of vehicle 105. Computing system 130 may execute the command instructions to activate the automated settings of vehicle functions 165A-165C of second user 175 when a trigger condition is later detected.

[0210] In one embodiment, computing system 130 may be configured to: concurrently store command instructions for automated vehicle actions associated with first user 120 and second user 175. For example, this may occur when first user 120 and second user 175 are in vehicle 105 during a concurrent time period. Additionally or alternatively, this may occur when first user 120 and second user 175 are regular users of vehicle 105.

[0211] In one embodiment, computing system 130 may execute the automated vehicle actions of first user 120 and the automated vehicle actions of second user 175 at concurrent / overlapping time periods such that vehicle 105 simultaneously executes two automated settings (for two different users). As an example, the first automated vehicle action of first user 120 may instruct vehicle 105 to set the seat heating function to "low" when the external temperature is below 16 degrees Fahrenheit. The second automated vehicle action of second user 175 may instruct vehicle 105 to set the seat heating function to "high" when the external temperature is below 15 degrees Fahrenheit.

[0212] In a manner similar to that described herein, computing system 130 may compare the vehicle functions, trigger conditions, and automated settings of the first automated vehicle action and the second automated vehicle action to determine whether the first automated vehicle action and the second automated vehicle action conflict with each other. In the case where vehicle 105 (e.g., the associated vehicle function) may not be able to concurrently implement the automated settings of the first automated vehicle action and the second automated vehicle action, the two actions may be considered to conflict. In the above example, the first automated vehicle action and the second automated vehicle action may be concurrently implemented because, for example, the seat of first user 120 (sitting in the driver's seat) has a different seat heating function than the seat of second user 175 (sitting in the passenger seat). Thus, in the case where computing system 130 detects that the external temperature is 14 degrees Fahrenheit, computing system 130 may send a first signal to set the seat heating function of the first user to "low" and a second signal to set the seat heating function of the second user to "high". In one embodiment, computing system 130 may generate content to notify first user 120 and second user 175 (e.g., via a user interface on a display device of the infotainment system) of the activation of the automated settings.

[0213] In the presence of a conflict, computing system 130 may attempt to resolve the conflict according to one or more policies 375. In one embodiment, policies 375 may include one or more policies for resolving conflicts between the automated vehicle actions of different users in vehicle 105. For example, policies 375 may include a hierarchy that indicates that the automated vehicle actions of the user who is the driver, regardless of which user that is, will take precedence over the automated vehicle actions of other users. In another example, the hierarchy may indicate that the automated vehicle actions of the user who first enters vehicle 105 will take precedence over the automated vehicle actions of other users. In one embodiment, computing system 130 may generate content to notify first user 120 or second user 175 that a particular automated vehicle action was not implemented due to a conflict.

[0214] In one embodiment, method 1100 may include step 1115, in which computing system 130 generates another automated vehicle action for second user 175 based on context data associated with second user 175. For example, computing system 130 may utilize data pipeline 300 described with reference Figure 3 to generate an automated vehicle action for second user 175 of vehicle 105. Such vehicle actions may be stored in a manner associated with the second user profile of second user 175. In this way, a user / user profile may be associated with multiple automated vehicle actions, where at least one automated vehicle action (e.g., the command instruction associated therewith) is determined by first vehicle 105 and at least one automated vehicle action (e.g., the command instruction associated therewith) is determined by second vehicle 175.

[0215] In some embodiments, computing system 130 may utilize data pipeline 300 to concurrently learn the routines and patterns of first user 120 and second user 175. This may include collecting context data for first user 120 and second user 175 simultaneously, generating user-activity clusters, determining automated vehicle actions, outputting command instructions, performing conflict resolution analysis, etc.

[0216] Figure 12 FIG. illustrates a block diagram of an example computing system 1200 according to an embodiment of the present disclosure. System 1200 includes a computing system 1205 (e.g., a computing system on a vehicle), a server computing system 1305 (e.g., a remote computing system, a cloud computing platform), and a training computing system 1405 communicatively coupled via one or more networks 1255.

[0217] Computing system 1205 may include one or more computing devices 1210 or circuits. For example, computing system 1205 may include control circuit 1215 and a non-transitory computer-readable medium 1220 (also referred to herein as a memory). In one embodiment, control circuit 1215 may include one or more processors (e.g., microprocessors), one or more processing cores, programmable logic circuitry (PLC) or programmable logic / gate arrays (PLA / PGA), field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), or any other control circuit. In some embodiments, control circuit 1215 may be part of or may form a vehicle control unit, which is embedded or otherwise disposed in a vehicle (e.g., in a car or truck). For example, the vehicle controller can be or can include an infotainment system controller (e.g., an infotainment host unit), a telematics control unit (TCU), an electronic control unit (ECU), a central power train controller (CPC), a charging controller, a central external and internal controller (CEIC), a zone controller, or any other controller. In one embodiment, the control circuit 1215 can be programmed by one or more computer-readable instructions or computer-executable instructions stored on a non-transitory computer-readable medium 1220.

[0218] In one embodiment, the non-transitory computer-readable medium 1220 can be a memory device (also referred to as a data storage device), which can include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. The non-transitory computer-readable medium 1220 can be formed, for example, as a hard disk drive (HDD), a solid-state drive (SDD), or a solid-state integrated memory, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a dynamic random access memory (DRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), and / or a memory stick.

[0219] The non-transitory computer-readable medium 1220 can store information accessible by the control circuit 1215. For example, the non-transitory computer-readable medium 1220 (e.g., the memory device) can store data 1225 that can be obtained, received, accessed, written, manipulated, created, and / or stored. The data 1225 can include, for example, any of the data or information described herein. In some specific implementations, the computing system 1205 can obtain data from one or more memories remote from the computing system 1205.

[0220] The non-transitory computer-readable medium 1220 may also store computer-readable instructions 1230 executable by the control circuit 1215. The instructions 1230 may be software written in any suitable programming language or may be implemented in hardware. The instructions may include computer-readable instructions, computer-executable instructions, etc. As described herein, in various embodiments, the terms "computer-readable instructions" and "computer-executable instructions" are used to describe software instructions or computer code configured to perform various tasks and operations. In various embodiments, if the computer-readable instructions or computer-executable instructions form a module, the term "module" broadly refers to a collection of software instructions or code configured to cause the control circuit 1215 to perform one or more functional tasks. When the control circuit 1215 or other hardware components are executing a module or computer-readable instructions, the module and the computer-readable / executable instructions may be described as performing various operations or tasks.

[0221] The instructions 1230 may be executed in logical and / or virtual separated threads on the control circuit 1215. For example, the non-transitory computer-readable medium 1220 may store instructions 1230 that, when executed by the control circuit 1215, cause the control circuit 1215 to perform any of the operations, methods, and / or processes described herein. In some cases, the non-transitory computer-readable medium 1220 may store computer-executable instructions or computer-readable instructions, such as instructions for performing Figures 10A to 10B or Figure 11 at least a portion of the method of.

[0222] In one embodiment, the computing system 1205 may store or include one or more machine learning models 1235. For example, the machine learning model 1235 may be or otherwise include various machine learning models (including the machine learning clustering model 320). In one embodiment, the machine learning model 1235 may include an unsupervised learning model (e.g., for generating data clusters). In one embodiment, the machine learning model 1235 may include a neural network (e.g., a deep neural network) or other types of machine learning models (including non-linear models and / or linear models). The neural network may include a feedforward neural network, a recurrent neural network (e.g., a long short-term memory recurrent neural network), a convolutional neural network, or other forms of neural networks. Some example machine learning models may utilize attention mechanisms such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a transformer model).

[0223] In one aspect of the present disclosure, model 1235 can be used to group various types of input data. For example, machine learning clustering model 320 can be used to classify, identify patterns, or extract features from input data (such as data encoding user interactions with vehicle functions 165A - 165C), which describes user - selected settings 310 and observed conditions 315 associated with the user - selected settings, or any other suitable type of structured data.

[0224] As described herein, machine learning clustering model 320 can receive an input and process each input to produce a corresponding cluster assignment for each input. The corresponding cluster assignment for each input can include a corresponding probability distribution of the corresponding embedding relative to a plurality of clusters. Each probability distribution of the corresponding input elements can describe the corresponding probability (e.g., confidence) of belonging to each of the clusters. In other words, the corresponding cluster assignment can probabilistically map (e.g., soft - code) each input to a plurality of clusters. Thus, the cluster assignment can identify similarities between various inputs or input elements, such as similar objects or features within an image, similar sounds within an audio, and / or correlations between statistical data points.

[0225] In some embodiments, the cluster assignment that describes the mapping of the embedding relative to a plurality of clusters can describe the corresponding centroids of the plurality of clusters. For example, the clusters can be mathematically defined based on their corresponding centroids in a multi - dimensional space. In other embodiments, the cluster assignment that describes the mapping of the embedding relative to a plurality of clusters does not involve or does not require the calculation of cluster centroids.

[0226] In one implementation, one or more machine learning models 1235 can be received from server computing system 1305 via network 1255, stored in computing system 1205 (e.g., non - transitory computer - readable medium 1220), and then used or otherwise implemented by control circuit 1215. In one implementation, computing system 1205 can implement multiple parallel instances of a single model.

[0227] Additionally or alternatively, one or more machine learning models 1235 can be included in or otherwise stored and implemented by server computing system 1305, which communicates with the computing system according to a client - server relationship. For example, machine learning model 1235 can be implemented by server computing system 1305 as part of a web service. Thus, one or more models 1235 can be stored and implemented at computing system 1205, and / or one or more models 1235 can be stored and implemented at server computing system 1305.

[0228] The computing system 1205 may include one or more communication interfaces 1240. The communication interface 1240 may be used to communicate with one or more other systems. The communication interface 1240 may include any circuitry, components, software, etc. for communicating via one or more networks (e.g., network 1255). In some specific implementations, the communication interface 1240 may include, for example, one or more of a communication controller, receiver, transceiver, transmitter, port, conductor, software, and / or hardware for conveying data / information.

[0229] The computing system 1205 may also include one or more user input components 1245 for receiving user input. For example, the user input component 1245 may be a touch-sensitive component (e.g., a touch-sensitive display screen or a touchpad) that is sensitive to a user input object (e.g., a finger or a stylus). The touch-sensitive component may be used to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, a cursor device, a joystick, or other devices through which a user may provide user input.

[0230] The computing system 1205 may include one or more output components 1250. The output component 1250 may include hardware and / or software for audibly or visually generating content. For example, the output component 1250 may include one or more speakers, earpieces, headphones, a handset, etc. The output component 1250 may include a display device, which may include hardware for displaying a user interface and / or a message for a user. As an example, the output component 1250 may include a display screen, a CRT, an LCD, a plasma screen, a touch screen, a TV, a projector, a tablet computer, and / or other suitable display components.

[0231] The server computing system 1305 may include one or more computing devices 1310. In one implementation, the server computing system 1305 may include or otherwise be implemented by one or more server computing devices. In instances where the server computing system 1305 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0232] The server computing system 1305 may include control circuitry 1315 and a non-transitory computer-readable medium 1320 (also referred to herein as memory 1320). In one implementation, the control circuitry 1315 may include one or more processors (e.g., microprocessors), one or more processing cores, programmable logic circuitry (PLC) or programmable logic / gate arrays (PLA / PGA), field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), or any other control circuitry. In one implementation, the control circuitry 1315 may be programmed by one or more computer-readable instructions or computer-executable instructions stored on the non-transitory computer-readable medium 1320.

[0233] In one embodiment, the non-transitory computer-readable medium 1320 can be a memory device (also referred to as a data storage device), which can include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. The non-transitory computer-readable medium can form, for example, a hard disk drive (HDD), a solid-state drive (SDD), or a solid-state integrated memory, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a dynamic random access memory (DRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), and / or a memory stick.

[0234] The non-transitory computer-readable medium 1320 can store information accessible by the control circuit 1315. For example, the non-transitory computer-readable medium 1320 (e.g., a memory device) can store data 1325 that can be obtained, received, accessed, written, manipulated, created, and / or stored. The data 1325 can include, for example, any of the data or information described herein. In some specific implementations, the server system 1305 can obtain data from one or more memories remote from the server system 1305.

[0235] The non-transitory computer-readable medium 1320 can also store computer-readable instructions 1330 executable by the control circuit 1315. The instructions 1330 can be software written in any suitable programming language or can be implemented in hardware. The instructions can include computer-readable instructions, computer-executable instructions, etc. As described herein, in various embodiments, the terms "computer-readable instructions" and "computer-executable instructions" are used to describe software instructions or computer code configured to perform various tasks and operations. In various embodiments, if the computer-readable instructions or computer-executable instructions form a module, the term "module" broadly refers to a collection of software instructions or code configured to cause the control circuit 1315 to perform one or more functional tasks. When the control circuit 1315 or other hardware components are executing a module or computer-readable instructions, the module and the computer-readable / executable instructions can be described as performing various operations or tasks.

[0236] The instructions 1330 can be executed in logical and / or virtual separated threads on the control circuit 1315. For example, the non-transitory computer-readable medium 1320 can store instructions 1330 that, when executed by the control circuit 1315, cause the control circuit 1315 to perform any of the operations, methods, and / or processes described herein. In some cases, the non-transitory computer-readable medium 1320 can store computer-executable instructions or computer-readable instructions, such as for performingFigures 10A to 10B or Figure 11 instructions for at least a portion of the method of.

[0237] The server computing system 1305 may store or otherwise include one or more machine learning models 1335, including a plurality of machine learning clustering models 320. The machine learning model 1335 may include or be the same as the model 1235 stored in the computing system 1205. In one embodiment, the machine learning model 1335 may include an unsupervised learning model (e.g., for generating data clusters). In one embodiment, the machine learning model 1335 may include a neural network (e.g., a deep neural network) or other types of machine learning models (including non-linear models and / or linear models). The neural network may include a feed-forward neural network, a recurrent neural network (e.g., a long short-term memory recurrent neural network), a convolutional neural network, or other forms of neural networks. Some example machine learning models may utilize attention mechanisms such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a transformer model).

[0238] The machine learning models described in this specification may have various types of input data and / or combinations thereof, representing data that can be used by sensors and / or other systems on a vehicle. The input data may include, for example, latent encoded data (e.g., an input latent space representation, etc.), statistical data (e.g., data calculated and / or operated on from some other data source), sensor data (e.g., raw data and / or processed data collected by sensors of the vehicle), or other types of data.

[0239] The server computing system 1305 may include one or more communication interfaces 1340. The communication interface 1340 may be used to communicate with one or more other systems. The communication interface 1340 may include any circuitry, components, software, etc. for communicating via one or more networks (e.g., network 1255). In some specific implementations, the communication interface 1340 may include, for example, one or more of a communication controller, a receiver, a transceiver, a transmitter, a port, a conductor, software, and / or hardware for conveying data / information.

[0240] The computing system 1205 and / or the server computing system 1305 may train the models 1235, 1335 by interacting with a training computing system 1405 communicatively coupled via the network 1255. The training computing system 1405 may be separate from the server computing system 1305 or may be a part of the server computing system 1305.

[0241] The training computing system 1405 may include one or more computing devices 1410. In one implementation, the training computing system 1405 may include or otherwise be implemented by one or more server computing devices. In instances where the training computing system 1405 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0242] The training computing system 1405 may include control circuitry 1415 and a non-transitory computer-readable medium 1420 (also referred to herein as memory 1420). In one implementation, the control circuitry 1415 may include one or more processors (e.g., microprocessors), one or more processing cores, programmable logic circuitry (PLC) or programmable logic / gate arrays (PLA / PGA), field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), or any other control circuitry. In one implementation, the control circuitry 1415 may be programmed by one or more computer-readable instructions or computer-executable instructions stored on the non-transitory computer-readable medium 1420.

[0243] In one implementation, the non-transitory computer-readable medium 1420 may be a memory device (also referred to as a data storage device), which may include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. The non-transitory computer-readable medium may form, for example, a hard disk drive (HDD), a solid-state drive (SDD), or a solid-state integrated memory, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), dynamic random access memory (DRAM), portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD), and / or a memory stick.

[0244] The non-transitory computer-readable medium 1420 may store information accessible by the control circuitry 1415. For example, the non-transitory computer-readable medium 1420 (e.g., the memory device) may store data 1425 that can be obtained, received, accessed, written, manipulated, created, and / or stored. The data 1425 may include, for example, any of the data or information described herein. In some implementations, the training computing system 1405 may obtain data from one or more memories remote from the training computing system 1405.

[0245] The non-transitory computer-readable medium 1420 may also store computer-readable instructions 1430 executable by the control circuit 1415. The instructions 1430 may be software written in any suitable programming language or may be implemented in hardware. The instructions may include computer-readable instructions, computer-executable instructions, etc. As described herein, in various embodiments, the terms "computer-readable instructions" and "computer-executable instructions" are used to describe software instructions or computer code configured to perform various tasks and operations. In various embodiments, if the computer-readable instructions or computer-executable instructions form a module, the term "module" broadly refers to a collection of software instructions or code configured to cause the control circuit 1415 to perform one or more functional tasks. When the control circuit 1415 or other hardware components are executing a module or computer-readable instructions, these modules and computer-readable / executable instructions may be described as performing various operations or tasks.

[0246] The instructions 1430 may be executed in a logical or virtual separate thread on the control circuit 1415. For example, the non-transitory computer-readable medium 1420 may store instructions 1430 that, when executed by the control circuit 1415, cause the control circuit 1415 to perform any of the operations, methods, and / or processes described herein. In some cases, the non-transitory computer-readable medium 1420 may store computer-executable instructions or computer-readable instructions, such as instructions for performing Figures 10A to 10B or Figure 11 at least a portion of the method of.

[0247] The training computing system 1405 may include a model trainer 1435 that uses various training techniques or learning techniques to train the machine learning models 1235, 1335 stored at the computing system 1205 and / or the server computing system 1305. For example, a clustering loss function may be used to train the models 1235, 1335 (e.g., machine learning clustering models). The clustering loss function may be configured to: balance two competing objectives. First, the clustering loss function may be configured to: seek to produce a confidence assignment of input data elements to clusters. The clustering loss function may balance this first objective with a second objective of preventing trivial solutions in which all elements of the input data are mapped to a single cluster. Thus, the clustering loss function may encourage each input to be confidently assigned to one of the clusters, but also encourage the mapping of input data points across multiple clusters.

[0248] The model trainer can train models 1235, 1335 (e.g., machine learning clustering models) in an unsupervised manner. Thus, unlabeled data for a specific application or problem domain can be used to effectively train the models (e.g., to generate user-activity clusters), which improves the performance and adaptability of the models. Additionally, models 1235, 1335 can facilitate the discovery of natural partitions or clusters in the data without pre-existing embeddings to seed the clustering objectives. As a result, such models can be more effectively trained to cluster complex data with less manual human intervention (e.g., labeling, selecting pre-existing embeddings, etc.).

[0249] More specifically, the clustering loss function can evaluate the achievement of a first objective (encouraging confidence mapping) by evaluating a first average (e.g., "per-sample average entropy") of the corresponding first entropy of each respective probability distribution across multiple inputs. The clustering loss function can evaluate the achievement of a second objective (encouraging diversity in cluster assignments) by evaluating a second entropy of a second average of the probability distributions of multiple inputs (e.g., "entropy of the batch-average distribution"). Thus, the clustering loss function can be used to train models 1235, 1335 to produce non-trivial and confident clustering assignments in an unsupervised manner.

[0250] The training computing system 1405 can modify the parameters of models 1235, 1335 (e.g., machine learning clustering model 320) based on a loss function (e.g., the clustering loss function) such that models 1235, 1335 can be effectively trained in an unsupervised manner for a specific application without labeled data. This can be particularly useful for effectively training models to cluster complex and unlabeled data sets.

[0251] In one example, the model trainer 1435 can backpropagate the clustering loss function through the machine learning clustering model 320 to modify the parameters (e.g., weights) of the clustering model. The model trainer 1435 can continue to backpropagate the clustering loss function through the machine learning clustering model 320 with or without modifying the parameters (e.g., weights) of the model. For example, the model trainer 1435 can perform gradient descent techniques in which the parameters of the machine learning clustering model 320 are modified in the negative gradient direction of the clustering loss function. Thus, in one implementation, the model trainer 1435 can modify the parameters of the machine learning clustering model 320 based on the clustering loss function without modifying the parameters of the embedding model.

[0252] In one embodiment, one or more components of the clustering loss function may be scaled by corresponding hyperparameters. For example, the second entropy may be scaled by a diversity hyperparameter. The diversity hyperparameter may be used to adjust the relative effects of the terms of the clustering loss function that respectively promote the two objectives. Thus, the diversity hyperparameter may be used to adjust or tune the loss provided by the clustering loss function and the resulting behavior of the machine learning clustering model 320 trained based on the clustering loss function. The diversity hyperparameter may be selected to produce a desired balance between a first objective, which is to minimize the average entropy of the input data points, and a second objective, which is to prevent the mapping produced by the machine learning clustering model 320 from degenerating into a trivial solution that maps all inputs to a single cluster.

[0253] In other specific embodiments, one or more components of the clustering loss function may be scaled by learned diversity weights or parameters. For example, iterative steps of training and evaluation may be used to optimize the diversity hyperparameter that controls the balance between the above first and second objectives. Thus, the learned diversity weights may be used to further improve the training of the machine learning clustering model 320.

[0254] The model trainer 1435 may utilize training techniques such as backpropagation of error. For example, the loss function may be backpropagated through the model to update one or more parameters of the model (e.g., based on the gradient of the loss function). Various loss functions may be used, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques may be used to iteratively update the parameters through multiple training iterations.

[0255] In one embodiment, performing backpropagation of error may include performing truncated backpropagation over time. The model trainer 1435 may perform various generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the trained model. In particular, the model trainer 1435 may train the machine learning models 1235, 1335 based on a set of training data 1440.

[0256] The training data 1440 may include unlabeled training data for unsupervised training. In one example, the training data 1440 may include an unlabeled data set indicating the settings selected by the training user for a specific vehicle function and data indicating the conditions observed during training. The training data 1440 may be specific to a particular vehicle function to help focus the models 1235, 1335 on the specific vehicle function.

[0257] In one embodiment, if the user has provided consent / authorization, training examples may be provided by the computing system 1205 (e.g., of the user's vehicle). Thus, in such embodiments, the models 1235 provided to the computing system 1205 may be trained by the training computing system 1405 in a manner that personalizes the models 1235.

[0258] The model trainer 1435 may include computer logic for providing the desired functionality. The model trainer 1435 may be implemented in hardware, firmware, and / or software that controls a general-purpose processor. For example, in one embodiment, the model trainer 1435 may include program files stored on a storage device, loaded into memory, and executed by one or more processors. In other embodiments, the model trainer 1435 may include one or more sets of computer-executable instructions stored on a tangible computer-readable storage medium (such as RAM, a hard disk, or optical or magnetic media).

[0259] The training computing system 1405 may include one or more communication interfaces 1445. The communication interfaces 1445 may be used to communicate with one or more other systems. The communication interfaces 1445 may include any circuitry, components, software, etc. for communicating via one or more networks (e.g., network 1255). In some embodiments, the communication interfaces 1445 may include, for example, one or more of a communication controller, a receiver, a transceiver, a transmitter, a port, a conductor, software, and / or hardware for conveying data / information.

[0260] The one or more networks 1255 may be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and may include any number of wired or wireless links. Generally, communication over the network 1255 may be carried via any type of wired and / or wireless connection using various communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or security schemes (e.g., VPN, secure HTTP, SSL).

[0261] FIG. 15 illustrates an example computing system that may be used to implement the present disclosure. Other computing systems may also be used. For example, in one embodiment, the computing system 1205 may include the model trainer 1435 and the training data 1440. In such embodiments, the models 1235, 1335 may be locally trained and used at the computing system 1205. In some of such embodiments, the computing system 1205 may implement the model trainer 1435 to personalize the models 1235, 1335.

[0262] Additional discussion of various embodiments

[0263] Embodiment 1 relates to a computing system. The computing system may include control circuitry. The control circuitry may be configured to: receive context data associated with a plurality of user interactions with a vehicle function over a plurality of time instances. The context data may include data indicating settings of a plurality of user selections for the vehicle function and data indicating one or more observed conditions associated with the settings of the corresponding user selections. The control circuitry may be configured to: use a machine learning clustering model to generate user-activity clusters for the vehicle function based on the context data. The machine learning clustering model may be configured to: identify the user-activity clusters based on at least a portion of the settings of the user selections and at least a portion of the one or more observed conditions associated with the settings of the corresponding user selections. The control circuitry may be configured to: determine an automated vehicle action based on the user-activity clusters. The automated vehicle action may indicate an automated setting for the vehicle function and one or more trigger conditions for automatically implementing the automated setting. The control circuitry may output a command instruction for the vehicle to implement the automated vehicle action based on whether the vehicle detects the one or more trigger conditions, thereby automatically controlling the vehicle function according to the automated setting.

[0264] Embodiment 2 includes the computing system according to Embodiment 1. In this embodiment, the machine learning clustering model may be an unsupervised learning model, and the unsupervised learning model is configured to use probability clustering to process the context data to generate the user-activity clusters.

[0265] Embodiment 3 includes the computing system according to any one of Embodiments 1 or 2. In this embodiment, the machine learning clustering model may be configured to determine boundaries of the user-activity clusters for the vehicle function based on the one or more observed conditions.

[0266] Embodiment 4 includes the computing system according to any one of Embodiments 1 to 3. In this embodiment, to determine the automated vehicle action, the control circuitry may be configured to: process the user-activity clusters using a vehicle action model, where the vehicle action model is a rule-based model, and the rule-based model is configured to apply corresponding weights to each of the settings of the corresponding user selections within the user-activity clusters. The control circuitry may be configured to: determine the automated setting for the vehicle function based on the weights of the settings of the corresponding user selections within the user-activity clusters.

[0267] Embodiment 5 includes the computing system according to any one of Embodiments 1 to 4. In this embodiment, in order to receive the context data, the control circuit may be further configured to: receive data indicating the one or more observed conditions via a plurality of sensors or systems of the vehicle. The observed conditions may include at least one of the following: (i) the date and / or time of day when the corresponding user interaction with the vehicle function occurs; (ii) the location where the corresponding user interaction with the vehicle function occurs; (iii) the route where the corresponding user interaction with the vehicle function occurs; or (iv) the temperature when the corresponding user interaction with the vehicle function occurs.

[0268] Embodiment 6 includes the computing system according to any one of Embodiments 1 to 5. In this embodiment, the control circuit is further configured to: generate content for presentation to the user via a user interface of a display device based on the automated vehicle action and using an action interpretation model. The content indicates the vehicle function, the automation setting, and the one or more trigger conditions.

[0269] Embodiment 7 includes the computing system according to any one of Embodiments 1 to 6. In this embodiment, the content may include a request for the user to approve the automated vehicle action.

[0270] Embodiment 8 includes the computing system according to any one of Embodiments 1 to 7. In this embodiment, the control circuit may be further configured to: store the command instruction in an accessible memory on the vehicle for execution at a later time.

[0271] Embodiment 9 includes the computing system according to any one of Embodiments 1 to 8. In this embodiment, the control circuit may be further configured to: detect the occurrence of the one or more trigger conditions, and based on the one or more trigger conditions, send a signal to implement the automation setting for the vehicle function.

[0272] Embodiment 10 includes the computing system according to any one of Embodiments 1 to 9. In this embodiment, the control circuit may be further configured to: determine whether there is a conflict between the automated vehicle action and a pre-existing automated vehicle action.

[0273] Embodiment 11 includes the computing system according to any one of Embodiments 1 to 10. In this embodiment, the control circuit may be further configured to: send a communication indicating the command instruction to a server system via a network for storage in a manner associated with the user profile of the user.

[0274] Embodiment 12 includes the computing system according to any one of Embodiments 1 to 11. In this embodiment, the plurality of user interactions may be associated with a first user. The command instructions for the automated vehicle actions may be associated with a first user profile of the first user. The control circuit may be further configured to: receive data indicating a second user profile of a second user of the vehicle; and store command instructions for a second automated vehicle action associated with the second user profile in an accessible memory of the vehicle. The command instructions for the second automated vehicle action may be based on at least one user interaction of the second user with a vehicle function of another vehicle.

[0275] Embodiment 13 includes the computing system according to any one of Embodiments 1 to 12. In this embodiment, the vehicle may include a plurality of vehicle functions and a plurality of machine learning clustering models. The control circuit may be further configured to: select the machine learning clustering model from the plurality of machine learning clustering models based on the vehicle function.

[0276] Embodiment 14 includes the computing system according to any one of Embodiments 1 to 13. In this embodiment, the vehicle function may include: (i) a window function; (ii) a seat function; or (iii) a temperature function.

[0277] Embodiment 15 includes the computing system according to any one of Embodiments 1 to 14. In this embodiment, the seat function may include: a seat temperature function, a seat ventilation function, or a seat massage function.

[0278] Embodiment 16 includes the computing system according to any one of Embodiments 1 to 15. In this embodiment, the plurality of user-selected settings of the vehicle function may each include at least one of the following: (i) an on / off selection; (ii) an open / close selection; (iii) a temperature selection; or (iv) a massage level selection.

[0279] Embodiment 17 includes a computer-implemented method. The computer-implemented method may include: receiving context data associated with a plurality of user interactions with a vehicle function over a plurality of time instances. The context data may include data indicating settings for a plurality of user selections for the vehicle function and data indicating one or more observed conditions associated with the settings corresponding to the user selections. The computer-implemented method may include: using a machine learning clustering model to generate user-activity clusters for the vehicle function based on the context data. The machine learning clustering model may be configured to identify the user-activity clusters based on at least a portion of the settings selected by the user and at least a portion of the one or more observed conditions associated with the settings corresponding to the user selections. The computer-implemented method may include: determining an automated vehicle action based on the user-activity clusters. The automated vehicle action may indicate an automated setting for the vehicle function and one or more trigger conditions for automatically implementing the automated setting. The computer-implemented method may include: outputting a command instruction for the vehicle to implement the automated vehicle action based on whether the vehicle detects the one or more trigger conditions, thereby automatically controlling the vehicle function according to the automated setting.

[0280] Embodiment 18 includes the computer-implemented method according to Embodiment 17. In this embodiment, the machine learning clustering model may include an unsupervised learning model configured to process the context data using probability clustering to generate the user-activity clusters; and the boundaries of the user-activity clusters for the vehicle function may be based on the one or more observed conditions.

[0281] Embodiment 19 includes the computer-implemented method according to any one of Embodiments 17 or 18. In this embodiment, the computer-implemented method may further include: generating content for presentation to a user via a user interface of a display device based on the automated vehicle action and using an action explanation model. The content may indicate the vehicle function, the automated setting, and the one or more trigger conditions.

[0282] Embodiment 20 includes one or more non-transitory computer-readable media storing instructions executable by a control circuit to perform operations. The control circuit may receive context data associated with a plurality of user interactions with a vehicle function at a plurality of time instances. The context data may include data indicating settings for a plurality of user selections for the vehicle function and data indicating one or more observed conditions associated with the settings corresponding to the user selections. The control circuit may use a machine learning clustering model to generate user-activity clusters for the vehicle function based on the context data. The machine learning clustering model may be configured to identify the user-activity clusters based on at least a portion of the settings selected by the user and at least a portion of the one or more observed conditions associated with the settings corresponding to the user selections. The control circuit may determine an automated vehicle action based on the user-activity clusters. The automated vehicle action may indicate an automated setting for the vehicle function and one or more trigger conditions for automatically implementing the automated setting. The control circuit may output a command instruction for the vehicle to implement the automated vehicle action based on whether the vehicle detects the one or more trigger conditions, thereby automatically controlling the vehicle function according to the automated setting.

[0283] Additional disclosure

[0284] As used herein, adjectives and their possessive forms are intended to be used interchangeably unless the context clearly dictates otherwise and / or clearly indicates otherwise. For example, in appropriate circumstances, "components of a vehicle" and "vehicle components" may be used interchangeably. Similarly, words, phrases, and other disclosures herein are intended to cover obvious variations and synonyms, even if such variations and synonyms are not explicitly listed.

[0285] The techniques discussed herein refer to servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information transmitted to and from these systems. The inherent flexibility of computer-based systems allows for a wide variety of possibilities in terms of configuration, combination, and task and functionality partitioning among components. For example, the processes discussed herein may be implemented using a single device or component, or multiple devices or components working in combination. Databases and applications may be implemented on a single system or distributed across multiple systems. Distributed components may operate sequentially or in parallel.

[0286] While the present subject matter has been described in detail with respect to various specific example embodiments of the subject matter, each example is provided by way of explanation and not limitation of the disclosure. Those skilled in the art will readily recognize alterations, variations, or equivalents to such embodiments after understanding the foregoing. Accordingly, the disclosure does not exclude including such modifications, variations, and / or additions to the disclosure, which would be apparent to a person of ordinary skill in the art. For example, functions illustrated or described as part of one embodiment may be used with another embodiment to yield yet another embodiment. Accordingly, the disclosure is intended to cover such alterations, variations, and equivalents.

[0287] Aspects of the present disclosure have been described in accordance with illustrative specific implementations of the present disclosure. Many other specific implementations, modifications, or variations within the scope and spirit of the appended claims will occur to those of ordinary skill in the art upon viewing the present disclosure. Any and all of the functions in the following claims may be combined or rearranged in any possible manner. Accordingly, the scope of the present disclosure is by way of example and not limitation, and the disclosure does not exclude including such modifications, variations, or additions to the disclosure, which would be apparent to a person of ordinary skill in the art. Additionally, lists of example elements are used herein to describe terms using conjunctions such as "and," "or," "but," etc. It should be understood that such conjunctions are provided for illustrative purposes only. The terms "or" and "and / or" may be used interchangeably herein. A list connected by a particular conjunction such as "or," for example, may refer to "at least one" or "any combination" of the example elements listed in the list, where "or" is understood as "and / or" unless otherwise indicated. Additionally, terms such as "based on" should be understood as "at least partially based on."

[0288] In using the disclosure provided herein, those of ordinary skill in the art will understand that elements of any of the claims, operations, or processes discussed herein may be adapted, rearranged, extended, omitted, combined, or modified in various ways without departing from the scope of the disclosure. Sometimes, for purposes of illustrative example, alphabetical labels may be used to list elements in the specification or claims and are not meant to be limiting. If alphabetical labels are used, the alphabetical label does not imply a particular order of operations or a particular importance of the elements listed. For example, alphabetical identifiers (such as (a), (b), (c),..., (i), (ii), (iii), … etc.) may be used to illustrate different elements in an operation or list. Such identifiers are provided for the convenience of the reader and do not denote a particular order, importance, or priority of steps, operations, or elements. For example, the operations illustrated by list identifiers such as (a), (i), etc. may be performed before, after, or in parallel with another operation illustrated by list identifiers such as (b), (ii), etc.

Claims

1. A computing system, the computing system comprising: control circuitry configured to: receive context data associated with a plurality of user interactions with a vehicle function over a plurality of time instances, wherein the context data includes data indicative of settings for a plurality of user selections for the vehicle function and data indicative of one or more observed conditions associated with the settings of the respective user selections; generate user-activity clusters for the vehicle function based on the context data using a machine learning clustering model, the machine learning clustering model being configured to identify the user-activity clusters based on at least a portion of the settings of the user selections and at least a portion of the one or more observed conditions associated with the settings of the respective user selections; determine an automated vehicle action based on the user-activity clusters, wherein the automated vehicle action indicates an automated setting for the vehicle function and one or more trigger conditions for automatically implementing the automated setting; and output a command instruction for the vehicle to implement the automated vehicle action based on whether the vehicle detects the one or more trigger conditions, thereby automatically controlling the vehicle function according to the automated setting.

2. The computing system according to claim 1, wherein the machine learning clustering model is an unsupervised learning model configured to process the context data using probability clustering to generate the user-activity clusters.

3. The computing system according to claim 1, wherein the machine learning clustering model is configured to determine boundaries of the user-activity clusters for the vehicle function based on the one or more observed conditions.

4. The computing system according to claim 1, wherein, to determine the automated vehicle action, the control circuitry is configured to: process the user-activity clusters using a vehicle action model, wherein the vehicle action model is a rule-based model configured to apply respective weights to each of the respective user selections of the settings within the user-activity clusters; and determine the automated setting for the vehicle function based on the weights of the respective user selections of the settings within the user-activity clusters.

5. The computing system according to claim 1, wherein, to receive the context data, the control circuitry is further configured to: Receiving data indicative of the one or more observed conditions via a plurality of sensors or systems of the vehicle, wherein the observed conditions include at least one of the following: (i) the date and / or time of day at which the corresponding user interaction with the vehicle function occurred; (ii) the location at which the corresponding user interaction with the vehicle function occurred; (iii) the route on which the corresponding user interaction with the vehicle function occurred; or (iv) the temperature at which the corresponding user interaction with the vehicle function occurred.

6. The computing system of claim 1, wherein the control circuit is further configured to: Generate content for presentation to a user via a user interface of a display device based on the automated vehicle action and using an action interpretation model, wherein the content indicates the vehicle function, the automated setting, and the one or more trigger conditions.

7. The computing system of claim 6, wherein the content includes a request for the user to approve the automated vehicle action.

8. The computing system of claim 1, wherein the control circuit is further configured to: Store the command instruction in an accessible memory on the vehicle for execution at a later time.

9. The computing system of claim 1, wherein the control circuit is further configured to: Detect the occurrence of the one or more trigger conditions; and Based on the one or more trigger conditions, send a signal to effect the automated setting for the vehicle function.

10. The computing system of claim 1, wherein the control circuit is further configured to: Determine whether a conflict exists between the automated vehicle action and a pre-existing automated vehicle action.

11. The computing system of claim 1, wherein the control circuit is further configured to: Send a communication indicating the command instruction to a server system via a network for storage in a manner associated with a user profile of a user.

12. The computing system of claim 1, wherein the plurality of user interactions are associated with a first user, wherein the command instruction for the automated vehicle action is associated with a first user profile of the first user, and wherein the control circuit is further configured to: Receive data indicative of a second user profile of a second user of the vehicle; and Store a command instruction for a second automated vehicle action associated with the second user profile in an accessible memory of the vehicle, wherein the command instruction for the second automated vehicle action is based on at least one user interaction of the second user with a vehicle function of another vehicle.

13. The computing system of claim 1, wherein the vehicle includes a plurality of vehicle functions and a plurality of machine learning clustering models, and wherein the control circuit is further configured to: Select the machine learning clustering model from the plurality of machine learning clustering models based on the vehicle function.

14. The computing system according to claim 1, wherein the vehicle function includes: (i) Window function; (ii) Seat function; Or (iii) Temperature function.

15. The computing system according to claim 14, wherein the seat function includes: Seat temperature function, seat ventilation function or seat massage function.

16. The computing system according to claim 1, wherein the plurality of user-selected settings of the vehicle function respectively include at least one of the following: (i) On / Off selection; (ii) Open / Close selection; (iii) Temperature selection; or (iv) Massage level selection.

17. A computer-implemented method, the method comprising: Receiving context data associated with a plurality of user interactions with a vehicle function of a vehicle over a plurality of time instances, wherein the context data includes data indicating a plurality of user-selected settings for the vehicle function and data indicating one or more observed conditions associated with the corresponding user-selected settings; Using a machine learning clustering model to generate user-activity clusters for the vehicle function based on the context data, the machine learning clustering model being configured to identify the user-activity clusters based on at least a portion of the user-selected settings and at least a portion of the one or more observed conditions associated with the corresponding user-selected settings; Determining an automated vehicle action based on the user-activity clusters, wherein the automated vehicle action indicates an automated setting for the vehicle function and one or more trigger conditions for automatically implementing the automated setting; And Outputting command instructions for the vehicle to implement the automated vehicle action based on whether the vehicle detects the one or more trigger conditions, thereby automatically controlling the vehicle function according to the automated setting.

18. The computer-implemented method according to claim 17, wherein the machine learning clustering model is an unsupervised learning model, the unsupervised learning model being configured to use probabilistic clustering to process the context data to generate the user-activity clusters; and wherein the boundaries of the user-activity clusters for the vehicle function are based on the one or more observed conditions.

19. The computer-implemented method according to claim 17, the method further comprising: Generating content for presentation to a user via a user interface of a display device based on the automated vehicle action and using an action explanation model, Wherein the content indicates the vehicle function, the automated setting, and the one or more trigger conditions.

20. One or more non-transitory computer-readable media storing instructions that can be executed by a control circuit to: Receive situational data associated with multiple user interactions with a vehicle function of a vehicle over multiple time instances, where the situational data includes data indicating settings of multiple user selections for the vehicle function and data indicating one or more observed conditions associated with the settings of the corresponding user selections; Use a machine learning clustering model to generate user-activity clusters for the vehicle function based on the situational data, the machine learning clustering model being configured to identify the user-activity clusters based on at least a portion of the settings of the user selections and at least a portion of the one or more observed conditions associated with the settings of the corresponding user selections; Determine an automated vehicle action based on the user-activity clusters, where the automated vehicle action indicates an automated setting for the vehicle function and one or more trigger conditions for automatically implementing the automated setting; And Output command instructions for the vehicle to implement the automated vehicle action based on whether the vehicle detects the one or more trigger conditions, thereby automatically controlling the vehicle function according to the automated setting.

Images