App suggestion system

Abstract

Methods and systems to display contextually-relevant content to a user in a vehicle. Sequence data indicative of a plurality of actions previously taken in sequence by a user within a vehicle is acces

Description

HELD

[0001] The present disclosure relates generally to using machine-learned models to anticipate user behavior. More particularly, the present disclosure relates to observing a sequence of behaviors of a user within a vehicle and using machine-learned models to predict a future action of the user. The present disclosure also relates to training a machine-learned model using feedback from the user. background

[0002] A vehicle may include a machine-learned model trained to provide personalized recommendations to a user. The machine-learned model may rely on data that indicates behavioral or personal traits of the user. SUMMARY

[0003] Aspects and advantages of implementations of the present disclosure will be set forth in part in the following description, or may be learned from the description, or may be learned through practice of the implementations.

[0004] One example aspect of the present disclosure is directed to a computing system of a vehicle. The computing system includes one or more processors and one or more tangible, non-transitory, computer readable media storing instructions that when executed by the one or more processors cause the computing system to perform operations. The operations include accessing sequence data indicative of a plurality of actions previously taken in sequence by a user within a vehicle. The operations include determining, based on the sequence data, context data associated with the plurality of actions, wherein the context data is indicative of at least a time or a location of respective actions of the plurality of actions. The operations include determining, using one or more machine-learned models and based on the sequence data and the context data, user preference data indicative of a ranking of computing programs associated with the vehicle, wherein the ranking is based on a probability of user interaction. The operations include outputting one or more signals to initiate a display of content indicative of the user preference data.

[0005] In an embodiment, the operations include associating, based on the sequence data and the context data, the plurality of actions with a respective computing program of a plurality of computing programs.

[0006] In an embodiment, the operations include receiving ground truth data, the ground truth data indicative of user feedback associated with the ranking of the computer programs. In an embodiment, the operations include based on the ground truth data, sequence data, and context data, determining using the one or more machine-learned models updated user preference data.

[0007] In an embodiment, the user feedback is indicative of at least one of: (i) a dismiss action, (ii) a non-action, (iii) a long way action, or (iv) an engage action.

[0008] In an embodiment, the user feedback is indicative of at least one of: (i) positive feedback, (ii) neutral feedback, or (iii) negative feedback.

[0009] In an embodiment, one or more weights are applied to the user feedback, the one or more weights associated with a level of interaction of the user.

[0010] In an embodiment, the sequence is indicative of the plurality of actions taken during a respective trip.

[0011] In an embodiment, the respective trip is based on a duration of time or distance between a point of origin and a destination.

[0012] In an embodiment, the ranking comprises at least three computing programs associated with a highest probability of user interaction.

[0013] One example aspect of the present disclosure is directed to a computer-implemented method. The computer-implemented method includes accessing sequence data indicative of a plurality of actions previously taken in sequence by a user within a vehicle. The computer-implemented method determining, based on the sequence data, context data associated with the plurality of actions, wherein the context data is indicative of at least a time or a location of respective actions of the plurality of actions. The computer-implemented method includes determining, using one or more machine-learned models and based on the sequence data and the context data, user preference data indicative of a ranking of computing programs associated with the vehicle, wherein the ranking is based on a probability of user interaction. The computer-implemented method includes outputting one or more signals to initiate a display of content indicative of the user preference data.

[0014] In an embodiment, the method includes associating, based on the sequence data and the context data, the plurality of actions with a respective computing program of a plurality of computing programs.

[0015] In an embodiment, the method includes receiving ground truth data, the ground truth data indicative of user feedback associated with the ranking of the computer programs. In an embodiment, the method includes based on the ground truth data, sequence data, and context data, determining using the one or more machine-learned models updated user preference data.

[0016] In an embodiment, the user feedback is indicative of at least one of: (i) a dismiss action, (ii) a non-action, (iii) a long way action, or (iv) an engage action.

[0017] In an embodiment, the user feedback is indicative of at least one of: (i) positive feedback, (ii) neutral feedback, or (iii) negative feedback.

[0018] In an embodiment, one or more weights are applied to the user feedback, the one or more weights associated with a level of interaction of the user.

[0019] In an embodiment, the sequence is indicative of the plurality of actions taken during a respective trip.

[0020] In an embodiment, the respective trip is based on a duration of time or distance between a point of origin and a destination.

[0021] In an embodiment, the ranking comprises at least three computing programs associated with a highest probability of user interaction.

[0022] One example aspect of the present disclosure is directed to one or more non-transitory computer-readable media storing instructions that when executed the one or more processors cause a computing system to perform operations including: accessing sequence data indicative of a plurality of actions previously taken in sequence by a user within a vehicle; determining, based on the sequence data, context data associated with the plurality of actions, wherein the context data is indicative of at least a time or a location of respective actions of the plurality of actions; determining, using one or more machine-learned models and based on the sequence data and the context data, user preference data indicative of a ranking of computing programs associated with the vehicle, wherein the ranking is based on a probability of user interaction; outputting one or more signals to initiate a display of content indicative of the user preference data.

[0023] Other example aspects of the present disclosure are directed to other systems, methods, vehicles, apparatuses, tangible non-transitory computer-readable media, and devices for the technology described herein.

[0024] These and other features, aspects, and advantages of various implementations will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate implementations of the present disclosure and, together with the description, serve to explain the related principles. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Detailed discussion of implementations directed to one of ordinary skill in the art are set forth in the specification, which makes reference to the appended figures, in which:

[0026] FIG. I illustrates an example computing ecosystem according to an embodiment hereof.

[0027] FIGS. 2A-D illustrate diagrams of an example computing architecture for an onboard computing system of a vehicle according to an embodiment hereof.

[0028] FIG. 3 illustrates an example vehicle interior with an example display according to an embodiment hereof.

[0029] FIG. 4 illustrates a diagram of an example computing platform that is remote from a vehicle according to an embodiment hereof.

[0030] FIG. 5 illustrates a diagram of an example user device according to an embodiment hereof.

[0031] FIG. 6 illustrates an example architecture of a multi-class machine-learned model according to an embodiment hereof.

[0032] FIG. 7 illustrates an example architecture of a binary machine-learned model according to an embodiment hereof.

[0033] FIG. 8. illustrates an example user interface according to an embodiment hereof.

[0034] FIG. 9 illustrates an example dataflow pipeline of generating training data according to an embodiment hereof.

[0035] FIG. 10 illustrates a flowchart diagram of an example method according to an embodiment hereof.

[0036] FIG. 11 illustrates a diagram of an example computing ecosystem with computing components according to an embodiment hereof. DETAILED DESCRIPTION

[0037] An aspect of the present disclosure relates to systems and methods for suggesting user preferences. User preferences may include a suggestion of the most likely software applications a user may use during a trip. For instance, a vehicle may include one or more machine-learned models trained on contextual user data generated by a vehicle operator or user of the vehicle. Contextual user data may include data generated as a result of a sequence of actions taken by a user of the vehicle within the context (e.g., day, time, temperature, location, etc.) in which the action was taken. As one example, a machine-learned model may be used by the vehicle to suggest software applications (e.g., phone application, interior controls application, navigation applications, entertainment application, etc.) to present to the vehicle operator via a user interface within the vehicle based on the vehicle operator using these applications in a previous similar context during a trip. For example, such models may output a predicted top 3, top 5, etc. list of applications and update a user interface within the vehicle to present these options when similar conditions (e.g., context) are detected.

[0038] Due to large training data requirements and the computationally intensive process, using deep learning models such as convolutional neural networks (CNNs), graph neural networks, (GNNs), recurrent neural networks (RNNs), etc., may not be appropriate for training on short sequences of user actions. Thus, shallow learning models such as linear regression models, decision trees, K-nearest neighbors (KNNs), etc. may be more appropriate. However, programmatically implementing standardization or normalization among user data may be a challenge due to the contextual complexity of the data. While examples herein describe an implementation using one or more shallow learning models, the present disclosure is not limited to such embodiment and may be implemented using deep learning models and the like.

[0039] To address this problem, the technology of the present disclosure enable these shallow models to be trained using a bag of apps (e.g., sequential aspect). The bag of apps may include a numerical representation of usage of applications, tiles, or other user interface elements during a trip. A trip may include a duration of time in which the vehicle operator is operating the vehicle from an origin to a destination location, or a duration of time in which the vehicle operator is located within the vehicle. For instance, a vehicle may include 5 applications (e.g., applications 1-5) that are available to the user. During a trip, application 1, application 3, and then application 5 may be used by the user. In response to applications 1,3, and 5 being used by the user during the trip, a bag of apps including a bag of apps vector may be generated with a vector size of 5 (e.g., due to 5 total applications being available). The bag of apps may include a first vector with binary values of [1, 0, 0, 0, 0] to indicate that the user used application 1 first, a second vector with binary values of [1, 0, 1, 0, 0] to indicate that the user used application 3 subsequently to using application I. and a third vector with binary values of [1, 0, 1,0, 1] to indicate that the user used application 5 subsequently to applications I and 3.

[0040] The bag of apps including the bag of apps vectors may be used as input to a machine-learned model. The bag of apps may reduce the complexity of the data by simplifying the sequence of application usage into binary vectors. The machine-learned model may receive as input the contextual user data. The contextual user data may include, but is not limited to the inside and outside temperature of the vehicle, the location (e.g., latitude, longitude), time (e.g., hour of the day, day of the week, etc.), duration of trip, etc.

[0041] The machine-learned model may receive the bag of apps and the contextual user and generate output indicative user preference data. The user preference data may indicate predicted user clicks (e.g., user actions). For instance, the machine-learned model may predict that the user will click on the navigation application first at the start of a trip to take an eco-friendly route home based on the contextual user data indicating the user has previously clicked on the navigation application first under similar context (e.g., time of 5 PM, located at user’s workplace, etc.). The machine-learned model may also predict subsequent user clicks such as predicting the user will tune the radio to a relaxing station, or predicting the user will adjust the cabin temperature settings, based on the bag of apps which indicates the user has previously interacted with the navigation, radio, and climate controls applications in sequence within similar context. The user preference output may cause the user interface of the vehicle to update to present the top 3, 5, etc. applications to the user.

[0042] As the user interacts with the available applications during the trip, user feedback data may be generated and used as training data to further train the machine-learned model. For instance, if the user clicks one of the suggested top applications, positive user feedback data may be generated. In some embodiments, if the user clicks on the suggested top applications in the order predicted by the machine-learned model, positive user feedback weighted more heavily may be generated. However, if the user fails to click on the suggested application, removes it from view (e.g., swipe away) or puts forth additional effort to locate another application, negative or neutral user feedback may be generated. As such the user feedback data may be used to further train the machine-learned model and iteratively improve the accuracy of the suggested top applications.

[0043] The technology of the present disclosure also improves the onboard computing technology of the vehicle. For instance, the vehicle’s onboard computing system may preserve computing resources by avoiding the computationally intensive process of training deep learning models and processing complex data sets. The vehicle computing system may also avoid the tedious task of collecting, storing and processing large amounts of user data necessary to train deep learning models. This allows the vehicle computing system to more efficiently utilize its computing resources to focus on more discrete tasks while still providing a fully trained model that iteratively improves over time.

[0044] The technology of the present disclosure may include the collection of data associated with a user in the event that the user expressly authorizes such collection. Such authorization may be provided by the user via explicit user input to a user interface in response to a prompt that expressly requests such authorization. Collected data may be anonymized, pseudonymized, encrypted, noised, securely stored, or otherwise protected. A user may opt out of such data collection at any time.

[0045] Reference now will be made in detail to embodiments, one or more examples of which are illustrated in the drawings. Each example is provided by way of explanation of the embodiments, not limitation of the present disclosure. In fact, it will be apparent to those skilled in the art that various modifications and variations may be made to the embodiments without departing from the scope or spirit of the present disclosure. For instance, features illustrated or described as part of one embodiment may be used with another embodiment to yield a still further embodiment. Thus, it is intended that aspects of the present disclosure cover such modifications and variations.

[0046] FIG. 1 illustrates an example computing ecosystem 100 according to an embodiment hereof. The ecosystem 100 may include a vehicle 105, a remote computing platform 110 (also referred to herein as computing platform 110), and a user device 115 associated with a user 120. The user 120 may be a driver of the vehicle 105. In some implementations, the user 120 may be a passenger of the vehicle 105. In some implementations, the computing ecosystem 100 may include a third party (3P) computing platform 125, as further described herein. The vehicle 105 may include a vehicle computing system 200 located onboard the vehicle 105. The computing platform 110, the user device 115, the third party computing platform 125, and / or the vehicle computing system 200 may be configured to communicate with one another via one or more networks 130.

[0047] The systems / devices of ecosystem 100 may communicate using one or more application programming interfaces (APIs). This may include external facing APIs to communicate data from one system / device to another. The external facing APIs may allow the systems / devices to establish secure communication channels via secure access channels over the networks 130 through any number of methods, such as web-based forms, programmatic access via RESTful APIs, Simple Object Access Protocol (SOAP), remote procedure call (RPC), scripting access, etc.

[0048] The computing platform 110 may include a computing system that is remote from the vehicle 105. In an embodiment, the computing platform 110 may include a cloud-based server system. The computing platform 110 may be associated with (e.g., operated by) an entity. For example, the remote computing platform 110 may be associated with an OEM that is responsible for the make and model of the vehicle 105. In another example, the remote computing platform 110 may be associated with a service entity contracted by the OEM to operate a cloud-based server system that provides computing services to the vehicle 105.

[0049] The computing platform 110 may include one or more back-end services for supporting the vehicle 105. The services may include, for example, tele-assist services, navigation / routing services, performance monitoring services, etc. In an embodiment, the remote computing platform 110 may train one or more machine-learned models utilized by the vehicle 105. The computing platform 110 may host or otherwise include one or more APIs for communicating data to / from a vehicle computing system 200 of the vehicle 105 or the user device 115. The computing platform 110 may include one or more inter-service APIs for communication among its microservices. In some implementations, the computing platform may include one or more RPCs for communication with the user device 115.

[0050] The computing platform 110 may include one or more computing devices. For instance, the computing platform 110 may include a control circuit and a non-transitory computer-readable medium (e.g., memory). The control circuit of the computing platform 110 may be configured to perform the various operations and functions described herein. A further description of the computing hardware and components of computing platform 110 is provided herein with reference to other figures.

[0051] The user device 115 may include a computing device owned or otherwise accessible to the user 120. For instance, the user device 115 may include a phone, laptop, tablet, wearable device (e.g., smart watch, smart glasses, headphones), personal digital assistant, gaming system, personal desktop devices, other hand-held devices, or other types of mobile or non-mobile user devices. As further described herein, the user device 115 may include one or more input components such as buttons, a touch screen, a joystick or other cursor control, a stylus, a microphone, a camera or other imaging device, a motion sensor, etc. The user device 115 may include one or more output components such as a display device (e.g., display screen), a speaker, etc.

[0052] In an embodiment, the user device 115 may include a component such as, for example, a touchscreen, configured to perform input and output functionality to receive user input and present information for the user 120. The user device 115 may execute one or more instructions to run an instance of a software application and present user interfaces associated therewith, as further described herein. In an embodiment, the launch of a software application may initiate a user-network session with the computing platform 110.

[0053] The third-party computing platform 125 may include a computing system that is remote from the vehicle 105, remote computing platform 110, and user device 115. In an embodiment, the third-party computing platform 125 may include a cloud-based server system. The term “third-party entity” may be used to refer to an entity that is different than the entity associated with the remote computing platform 110. For example, as described herein, the remote computing platform 110 may be associated with an OEM that is responsible for the make and model of the vehicle 105. The third-party computing platform 125 may be associated with a supplier of the OEM, a maintenance provider, a mapping service provider, an emergency provider, or other types of entities. In another example, the third-party computing platform 125 may be associated with an entity that owns, operates, manages, etc. a software application that is available to or downloaded on the vehicle computing system 200.

[0054] The third-party computing platform 125 may include one or more back-end services provided by a third-party entity. The third-party computing platform 125 may provide services that are accessible by the other systems and devices of the ecosystem 100. The services may include, for example, mapping services, routing services, search engine functionality, maintenance services, entertainment services (e.g., music, video, images, gaming, graphics), emergency services (e.g., roadside assistance, 911 support), or other types of services. In an embodiment, the third-party computing platform 125 may train one or more machine-learned models utilized by the vehicle 105. The third-party computing platform 125 may host or otherwise include one or more APIs for communicating data to / from the third-party computing system 125 to other systems / devices of the ecosystem 100.

[0055] The networks 130 may be any type of network or combination of networks that allows for communication between devices. In some implementations, the networks 130 may include one or more of a local area network, wide area network, the Internet, secure network, cellular network, mesh network, peer-to-peer communication link or some combination thereof and may include any number of wired or wireless links. Communication over the networks 130 may be accomplished, for instance, via a network interface using any type of protocol, protection scheme, encoding, format, packaging, etc. In an embodiment, communication between the vehicle computing system 200 and the user device 115 may be facilitated by near field or short range communication techniques (e.g., Bluetooth low energy protocol, radio frequency signaling, NFC protocol).

[0056] The vehicle 105 may be a vehicle that is operable by the user 120. In an embodiment, the vehicle 105 may be an automobile or another type of ground-based vehicle that is manually driven by the user 120. For example, the vehicle 105 may be a Mercedes-Benz® car or van. In some implementations, the vehicle 105 may be an aerial vehicle (e.g., a personal airplane) or a water-based vehicle (e.g., a boat). The vehicle 105 may include operator-assistance functionality such as cruise control, advanced driver assistance systems, etc. In some implementations, the vehicle 105 may be a fully or semi-autonomous vehicle.

[0057] The vehicle 105 may include a power train and one or more power sources. The powertrain may include a motor (e.g., an internal combustion engine, electric motor, or hybrid thereof), e-motor (e.g., electric motor), transmission (e.g., automatic, manual, continuously variable), driveshaft, axles, differential, e-components, gear, etc. The power sources may include one or more types of power sources. For example, the vehicle 105 may be a fully electric vehicle (EV) that is capable of operating a power train of the vehicle 105 (e.g., for propulsion) and the vehicle’s onboard functions using electric batteries. In an embodiment, the vehicle 105 may use combustible fuel. In an embodiment, the vehicle 105 may include hybrid power sources such as, for example, a combination of combustible fuel and electricity.

[0058] The vehicle 105 may include a vehicle interior. The vehicle interior may include the area inside of the body of the vehicle 105 including, for example, a cabin for users of the vehicle 105. The interior of the vehicle 105 may include seats for the users, a steering mechanism, accelerator interface, braking interface, etc. The interior of the vehicle 105 may include a display device such as a display screen associated with an infotainment system, as further described with respect to FIG 3.

[0059] The vehicle 105 may include a vehicle exterior. The vehicle exterior may include the outer surface of the vehicle 105. The vehicle exterior may include one or more lighting elements (e.g., headlights, brake lights, accent lights). The vehicle 105 may include one or more doors for accessing the vehicle interior by, for example, manipulating a door handle of the vehicle exterior. The vehicle 105 may include one or more windows, including a windshield, door windows, passenger windows, rear windows, sunroof, etc.

[0060] The systems and components of the vehicle 105 may be configured to communicate via a communication channel. The communication channel may include one or more data buses (e.g., controller area network (CAN)), on-board diagnostics connector (e.g., OBD-II), or a combination of wired or wireless communication links. The onboard systems may send or receive data, messages, signals, etc. amongst one another via the communication channel.

[0061] In an embodiment, the communication channel may include a direct connection, such as a connection provided via a dedicated wired communication interface, such as a RS-232 interface, a universal serial bus (USB) interface, or via a local computer bus, such as a peripheral component interconnect (PCI) bus. In an embodiment, the communication channel may be provided via a network. The network may be any type or form of network, such as a personal area network (PAN), a local-area network (LAN), Intranet, a metropolitan area network (MAN), a wide area network (WAN), or the Internet. The network may utilize different techniques and layers or stacks of protocols, including, e.g., the Ethernet protocol, the internet protocol suite (TCP / IP), the ATM (Asynchronous Transfer Mode) technique, the SONET (Synchronous Optical Networking) protocol, or the SDH (Synchronous Digital Hierarchy) protocol.

[0062] In an embodiment, the systems / devices of the vehicle 105 may communicate via an intermediate storage device, or more generally an intermediate non-transitory computer-readable medium. For example, the non-transitory computer-readable medium 140, which may be external to the vehicle computing system 200, may act as an external buffer or repository for storing information. In such an example, the vehicle computing system 200 may retrieve or otherwise receive the information from the non-transitory computer-readable medium 140.

[0063] Certain routine and conventional components of vehicle 105 (e.g., an engine) are not illustrated and / or discussed herein for the purpose of brevity. One of ordinary skill in the art will understand the operation of conventional vehicle components in vehicle 105.

[0064] The vehicle 105 may include a vehicle computing system 200. As described herein, the vehicle computing system 200 is onboard the vehicle 105. For example, the computing devices and components of the vehicle computing system 200 may be housed, located, or otherwise included on or within the vehicle 105. The vehicle computing system 200 may be configured to execute the computing functions and operations of the vehicle 105.

[0065] FIG. 2A illustrates an overview of an operating system of the vehicle computing system 200. The operating system may be a layered operating system. The vehicle computing system 200 may include a hardware layer 205 and a software layer 210. The hardware and software layers 205, 210 may include sub-layers. In some implementations, the operating system of the vehicle computing system 200 may include other layers (e.g., above, below, or in between those shown in FIG. 2A). In an example, the hardware layer 205 and the software layer 210 can be standardized base layers of the vehicle’s operating system.

[0066] FIG. 2B illustrates a diagram of the hardware layer 205 of the vehicle computing system 200. In the layered operating system of the vehicle computing system 200, the hardware layer 205 can reside between the physical computing hardware 215 onboard the vehicle 105 and the software (e.g., of software layer 210) that runs onboard the vehicle 105.

[0067] The hardware layer 205 may be an abstraction layer including computing code that allows for communication between the software and the computing hardware 215 in the vehicle computing system 200. For example, the hardware layer 205 may include interfaces and calls that allow the vehicle computing system 200 to generate a hardware-dependent instruction to the computing hardware 215 (e.g., processors, memories, etc.) of the vehicle 105.

[0068] The hardware layer 205 may be configured to help coordinate the hardware resources. The architecture of the hardware layer 205 may be serviced oriented. The services may help provide the computing capabilities of the vehicle computing system 105. For instance, the hardware layer 205 may include the domain computers 220 of the vehicle 105, which may host various functionality of the vehicle 105 such as the vehicle’s intelligent functionality (e.g., machine-learned models, etc.). The specification of each domain computer may be tailored to the functions and the performance requirements where the services are abstracted to the domain computers. By way of example, this permits certain processing resources (e.g., graphical processing units) to support the functionality of a central in-vehicle infotainment computer for rendering graphics across one or more display devices for navigation, games, etc. or to support an intelligent automated driving computer to achieve certain industry assurances.

[0069] The hardware layer 205 may be configured to include a connectivity module 225 for the vehicle computing system 200. The connectivity module may include code / instructions for interfacing with the communications hardware of the vehicle 105. This can include, for example, interfacing with a communications controller, receiver, transceiver, transmitter, port, conductors, or other hardware for communicating data / information. The connectivity module 225 may allow the vehicle computing system 200 to communicate with other computing systems that are remote from the vehicle 105 including, for example, remote computing platform 110 (e.g., an OEM cloud platform).

[0070] The architecture design of the hardware layer 205 may be configured for interfacing with the computing hardware 215 for one or more vehicle control units 230. The vehicle control units 230 may be configured for controlling various functions of the vehicle 105. This may include, for example, a central exterior and interior controller (CEIC), a charging controller, or other controllers as further described herein. In an embodiment, the vehicle 105 may automatically implement vehicle controls via the vehicle control units 230. For instance, a machine-learned model may output vehicle actions to be implemented by the vehicle 105. The vehicle 105 may receive the output and utilize the vehicle control units 230 to implement the action. An example of a machine-learned model generating output indicative of a vehicle action is further described with reference to FIGS 6-7.

[0071] The software layer 210 may be configured to provide software operations for executing various types of functionality and applications of the vehicle 105. FIG. 2C illustrates a diagram of the software layer 210 of the vehicle computing system 200. The architecture of the software layer 210 may be service oriented and may be configured to provide software for various functions of the vehicle computing system 200. To do so, the software layer 210 may include a plurality of sublayers 235A-C. For instance, the software layer 210 may include a first sublayer 235A including firmware (e.g., audio firmware) and a hypervisor, a second sublayer 235B including operating system components (e.g., open-source components), and a third sublayer 235C including middleware (e.g., for flexible integration with applications developed by an associated entity or third-party entity).

[0072] The vehicle computing system 200 may include an application layer 240. The application layer 240 may allow for integration with one or more software applications 245 that are downloadable or otherwise accessible by the vehicle 105. The application layer 240 may be configured, for example, using containerized applications developed by a variety of different entities. In an embodiment the user 120 may interact with the software applications 245 in a particular sequence which indicates a sequence of user actions. For instance, the user 120 may interact with a first application, a second application, and a third application in sequence during a trip. In an embodiment, one or more machine-learned models may be used to predict a future application likely to be used by the user 120 during a trip. An example of a machine-learned model predicting future software application 245 usage is further described with reference to FIGS 6-7.

[0073] The layered operating system and the vehicle’s onboard computing resources may allow the vehicle computing system 200 to collect and communicate data as well as operate the systems implemented onboard the vehicle 105. FIG. 2D illustrates a block diagram of example systems and data of the vehicle 105.

[0074] The vehicle 105 may include one or more sensor systems 305. A sensor system 305 may include or otherwise be in communication with a sensor of the vehicle 105 and a module for processing sensor data 310 associated with the sensor configured to acquire the sensor data 310. This may include sensor data 310 associated with the surrounding environment of the vehicle 105, sensor data associated with the interior of the vehicle 105, or sensor data associated with a particular vehicle function. The sensor data 310 may be indicative of conditions observed in the interior of the vehicle, exterior of the vehicle, or in the surrounding environment. For instance, the sensor data 310 may include image data, inside / outside temperature data, weather data, data indicative of a position of a user / object within the vehicle 105, weight data, motion / gesture data, audio data, or other types of data. The sensors may include one or more: cameras (e.g.. visible spectrum cameras, infrared cameras), motion sensors, audio sensors (e.g., microphones), weight sensors (e.g., for a vehicle a seat), temperature sensors, humidity sensors, Light Detection and Ranging (LIDAR) systems, Radio Detection and Ranging (RADAR) systems, or other types of sensors.

[0075] The vehicle 105 may include a positioning system 315. The positioning system 315 may be configured to generate location data 320 (also referred to as position data) indicative of a location (also referred to as a position) of the vehicle 105. For example, the positioning system 315 may determine location by using one or more of inertial sensors (e.g., inertial measurement units, etc.), a satellite positioning system, based on IP address, by using triangulation and / or proximity to network access points or other network components (e.g., cellular towers, Wi-Fi access points, etc.), or other suitable techniques. The positioning system 315 may determine a current location of the vehicle 105. The location may be expressed as a set of coordinates (e.g., latitude, longitude), an address, a semantic location (e.g., “at work”), etc.

[0076] In an embodiment, the positioning system 315 may be configured to localize the vehicle 105 within its environment. For example, the vehicle 105 may access map data that provides detailed information about the surrounding environment of the vehicle 105. The map data may provide information such as the identity and location of different roadways, road segments, buildings, or other items; the location and directions of traffic lanes (e.g., the location and direction of a parking lane, a turning lane, a bicycle lane, or other lanes within a particular roadway); traffic control data (e.g., the location, timing, or instructions of signage (e.g., stop signs, yield signs), traffic lights (e.g., stop lights), parking restrictions, or other traffic signals or control devices / markings (e.g., cross walks)); or any other data. The positioning system 315 may localize the vehicle 105 within the environment (e.g., across multiple axes) based on the map data. For example, the positioning system 155 may process certain sensor data 310 (e.g., LIDAR data, camera data, etc.) to match it to a map of the surrounding environment to get an understanding of the vehicle’s position within that environment. The determined position of the vehicle 105 may be used by various systems of the vehicle computing system 200 or another computing system (e.g., the remote computing platform 110, the third-party computing platform 125, the user device 115).

[0077] The vehicle 105 may include a communications unit 325 configured to allow the vehicle 105 (and its vehicle computing system 200) to communicate with other computing devices. The vehicle computing system 200 may use the communications unit 325 to communicate with the remote computing platform 110 or one or more other remote computing devices over a network 130 (e.g., via one or more wireless signal connections). For example, the vehicle computing system 200 may utilize the communications unit 325 to receive vehicle data 330 from the computing platform 110. This may include, for example, an over-the-air (OTA) software update for the operating system or machine-learned models (e.g., model syncing) of the vehicle computing system 200. Additionally, or alternatively, the vehicle computing system 200 may utilize the communications unit 325 to send vehicle data 330 to the computing platform 110. The vehicle data 330 may include any data acquired onboard the vehicle 105 including, for example, sensor data 310, location data 320, diagnostic data, user input data, data indicative of current software versions or currently running applications, occupancy data, data associated with the user 120 of the vehicle 105, or other types of data obtained (e.g., acquired, accessed, generated, downloaded, etc.) by the vehicle computing system 200.

[0078] In an embodiment, vehicle data 330 may include masked or encrypted data. For instance, data acquired onboard the vehicle 105 may include private or sensitive data. In some implementations, data may need to be masked, encrypted, anonymized, etc. to protect the privacy or sensitivity of the data. Example, sensitive data may include home locations, work locations, frequently visited locations, etc.

[0079] In some implementations, the communications unit 325 may allow communication among one or more of the systems on-board the vehicle 105.

[0080] In an embodiment, the communications unit 325 may be configured to allow the vehicle 105 to communicate with or otherwise receive data from the user device 115 (shown in FIG. 1). The communications unit 325 may utilize various communication technologies such as, for example, Bluetooth low energy protocol, radio frequency signaling, or other short range or near filed communication technologies. The communications unit 325 may include any suitable components for interfacing with one or more networks, including, for example, transmitters, receivers, ports, controllers, antennas, or other suitable components that may help facilitate communication.

[0081] The vehicle 105 may include one or more human-machine interface (HMI) systems 340. The HMI system 340 may include a display device, as described herein. The display device (e.g., touchscreen) may be viewable by a user of the vehicle 105 (e.g., user 120) that is located in the front of the vehicle 105 (e.g., driver’s seat, front passenger seat). Additionally, or alternatively, a display device (e.g., rear unit) may be viewable by a user that is located in the rear of the vehicle 105 (e.g., back passenger seats). The HMI system 340 may present content 335 via a user interface for display to a user 120.

[0082] FIG. 3 illustrates an example vehicle interior 300 with a display device 345. The display device 345 may be a component of the vehicle’s infotainment system. Such a component may be referred to as a display device of the infotainment system or be considered as a device for implementing an embodiment that includes the use of an infotainment system. For illustrative and example purposes, such a component may be referred to herein as a head unit display device (e.g., positioned in a front / dashboard area of the vehicle interior), a rear unit display device (e.g., positioned in the back passenger area of the vehicle interior), an infotainment head unit or rear unit, or the like. The display device 345 may be located on, form a portion of, or function as a dashboard of the vehicle 105. The display device 345 may include a display screen, CRT, LCD, plasma screen, touch screen, TV, projector, tablet, and / or other suitable display components.

[0083] The display device 345 may display a variety of content to the user 120 including information about the vehicle 105, prompts for user input, etc. The display device 345 may include a touchscreen through which the user 120 may provide user input (e.g., user action) to a user interface.

[0084] For example, the display device 345 may include user interface rendered via a touch screen that presents various content. The content may include vehicle speed, mileage, fuel level, charge range, or one or more software applications 245 such as navigation / routing information, audio selections, streaming content (e.g., video / image content), internet search results, comfort settings (e.g., temperature, humidity, seat position, seat massage), or other vehicle data 330. The display device 345 may render content to facilitate the receipt of user input. For instance, the user interface of the display device 345 may present one or more soft buttons with which a user 120 can interact to adjust various vehicle functions through a software application 245 (e.g., navigation, audio / streaming content selection, temperature, seat position, seat massage, etc.). In an embodiment, the display device 345 may present one or more predicted actions output by a machine-learned model. For instance, a machine-learned model may be trained to learn, anticipate, and predict adjustments to various vehicle functions. In an embodiment, a machine-learned model may output a predicted action and display a recommendation indicating the predicted action or sequence of predicted actions (e.g., associated with respective software applications 245) on the display device 345. Additionally, or alternatively, the display device 345 may be associated with an audio input device (e.g., microphone) for receiving audio input from the user 120.

[0085] Returning to FIG. 2D, the vehicle 105 may include a plurality of vehicle functions 350A^C. A vehicle function 350A C' may be a functionality that the vehicle 105 is configured to perform based on a detected input. The vehicle functions 350AC may include one or more: (i) vehicle comfort functions; (ii) vehicle staging functions; (iii) vehicle climate functions; (vi) vehicle navigation functions; (v) drive style functions; (v) vehicle parking functions; or (vi) vehicle entertainment functions. The user 120 may interact with a vehicle function 250A-C through a software application 245 by providing user input (e.g., to an adjustable input device, UI element) that specifies a setting of the vehicle function 25()AC selected by the user.

[0086] Each vehicle function may include a controller 355A-C associated with that particular vehicle function 350A-C. The controller 355A-C for a particular vehicle function may include control circuitry configured to operate its associated vehicle function 350A^C. For example, a controller may include circuitry configured to turn the seat heating function on, to turn the seat heating function off, set a particular temperature or temperature level, etc.

[0087] In an embodiment, a controller 355A-C for a particular vehicle function may include or otherwise be associated with a sensor that captures data indicative of the vehicle function being turned on or off, a setting of the vehicle function, etc. For example, a sensor may be an audio sensor or a motion sensor. The audio sensor may be a microphone configured to capture audio input from the user 120. For example, the user 120 may provide a voice command to activate the radio function of the vehicle 105 and request a particular station. The motion sensor may be a visual sensor (e.g., camera), infrared, RADAR, etc. configured to capture a gesture input from the user 120. For example, the user 120 may provide a hand gesture motion to adjust a temperature function of the vehicle 105 to lower the temperature of the vehicle interior.

[0088] The controllers 355A T' may be configured to send signals to another onboard system. The signals may encode data associated with a respective vehicle function. The encoded data may indicate, for example, a function setting, timing, etc. In an example, such data may be used to generate content for presentation via the display device 345 (e.g., showing a current setting). Additionally, or alternatively, such data can be included in vehicle data 330 and transmitted to the remote computing platform 110.

[0089] FIG. 4 illustrates a diagram of computing platform 110, which is remote from a vehicle according to an embodiment hereof. As described herein, the computing platform 110 may include a cloud-based computing platform.

[0090] In some implementations, the computing platform 110 may be implemented on a server, combination of servers, or a distributed set of computing devices which communicate over a network. For instance, the computing platform 110 may be distributed using one or more physical servers, private servers, or cloud computing. In some examples, the computing platform 110 may be implemented as a part of or in connection with one or more microservices, where, for example, an application is architected into independent services that communicate over APIs. Microservices may be deployed in a container (e.g., standalone software package for a software application) using a container service, or on VMs (virtual machines) within a shared network. Example, microservices may include a microservice associated with the vehicle software 405, remote assistance system 415, etc. A container service may be a cloud service that allows developers to upload, organize, run, scale, manage, and stop containers using containerbased virtualization to orchestrate their respective actions. A VM may include virtual computing resources which are not limited to a physical computing device. In some examples, the computing platform 110 may include or access one or more data stores for storing data associated with the one or more microservices. For instance, data stores may include distributed data stores, fully managed relational, NoSQL, and in-memory databases, etc. [0091 ] The computing platform may include vehicle software 405. The vehicle software 405 may include local software running on the vehicle 105. This may include metadata such as the current vehicle software, eligible software updates for the vehicle, etc. The computing platform 110 may be responsible for ensuring that the vehicle 105 is running the most up to date vehicle software 405. For instance, the computing platform 110 may receive software updates 410 from the OEM and update the vehicle software 405. In an embodiment vehicle software 405 maintains parity with the software running on the vehicle computing system 200. In an embodiment, the vehicle software 405 includes machine-learned models running on the vehicle 105. For instance, the computing platform 110 may remotely train one or more machine-learned models running on the vehicle 105.

[0092] The computing platform 110 may include a remote assistance system 415. The remote assistance system 415 may provide assistance to the vehicle 105. This can include providing information to the vehicle 105 to assist with charging (e.g., charging locations recommendations), remotely controlling the vehicle 105 (e.g., for AV assistance), roadside assistance (e.g., for collisions, flat tires), etc. The remote assistance system 415 may obtain assistance data 420 to provide its core functions. The assistance data 420 may include information that may be helpful for the remote assistance system 415 to assist the vehicle 105. This may include information related to the vehicle’s current state, an occupant’s current state, the vehicle’s location, the vehicle’s route, charge / fuel level, incident data, etc. In some implementations, the assistance data 420 may include the vehicle data 330.

[0093] The remote assistance system 415 may transmit data or command signals to provide assistance to the vehicle 105. This may include providing data indicative of relevant charging locations, remote control commands to move the vehicle, connect to an emergency provider, etc.

[0094] The computing platform 110 may include a security system 425. The security system 425 can be associated with one or more security-related functions for accessing the computing platform 110 or the vehicle 105. For instance, the security system 425 can process security data 430 for identifying digital keys, data encryption, data decryption, etc. for accessing the services / systems of the computing platform 110. Additionally, or alternatively, the security system 425 can store security data 430 associated with the vehicle 105. A user 120 can request access to the vehicle 105 (e.g., via the user device 115). In the event the request includes a digital key for the vehicle 105 as indicated in the security data 430, the security system 425 can provide a signal to lock (or unlock) the vehicle 105.

[0095] The computing platform 110 may include a navigation system 435 that provides a back-end routing and navigation service for the vehicle 105. The navigation system 435 may provide map data 440 to the vehicle 105. The map data 440 may be utilized by the positioning system 315 of the vehicle 105 to determine a location of the vehicle 105, a point of interest, etc. The navigation system 435 may also provide routes to destinations requested by the vehicle 105 (e.g., via user input to the vehicle’s head unit). The routes can be provided as a portion of the map data 440 or as separate routing data. Data provided by the navigation system 435 can be presented as content via a software application 245 on the display device 345 of the vehicle 105.

[0096] The computing platform 110 may include an entertainment system 445. The entertainment system 445 may access one or more databases for entertainment data 450 for a user 120 of the vehicle 105. In some implementations, the entertainment system 445 may access entertainment data 450 from another computing system associated with a third-party service provider of entertainment content. The entertainment data 450 may include media content such as music, videos, gaming data, etc. The entertainment data 450 may be provided to vehicle 105, which may output the entertainment data 450 via one or more software applications 245 as content 335 using one or more output devices of the vehicle 105 (e.g., display device, speaker, etc.).

[0097] The computing platform 110 may include a user system 455. The user system 455 may create, store, manage, or access user profile data 460. The user profile data 460 may include a plurality of user profiles, each associated with a respective user 120. The user profile data 460 may be updated based on information periodically provided by the vehicle 105. In some implementations, the user profile data 460 may be provided to the user device 115. A user profile may indicate various information about a respective user 120 including the user’s preferences (e.g., for music, comfort settings, parking preferences), frequented / past destinations, past routes, etc. The user profiles may be stored in a secure database. In some implementations, when a user 120 enters the vehicle 105, the user’s key (or user device) may provide a signal with a user or key identifier to the vehicle 105. The vehicle 105 may transmit data indicative of the identifier (e.g.. via its communications unit 325) to the computing platform 110. The computing platform 110 may look-up the user profile of the user 120 based on the identifier and transmit user profile data 460 to the vehicle computing system 200 of the vehicle 105. The vehicle computing system 200 may utilize the user profile data 460 to implement preferences of the user 120, present past destination locations, etc. For instance, the vehicle 105 may include a machine-learned model configured to anticipate future actions of the user 120 and suggest one or more software applications 245 to facilitate the anticipated future actions. In some implementations, the machine-learned model may determine a user is associated with a personalized machine-learned model that has been trained to anticipate a sequence of actions of the user 120 based on the user profile data 460.

[0098] By way of example, the user 120 may enter the vehicle 105, the computing platform 110 may look-up the user profile of the user 120 and determine the user has operated and trained a machine-learned model of a second vehicle. The vehicle computing system 200 may receive the trained machine-learned from the computing platform 110 and replace the machine-learned model in the vehicle 105. As such the vehicle 105 may seamlessly anticipate sequence of future actions of the user 120 across multiple vehicles and iteratively improve over time irrespective of the particular vehicle in operation.

[0099] FIG. 5 illustrates a diagram of example components of user device 115 according to an embodiment hereof. The user device 115 may include a display device 500 configured to render content via a user interface 505 for presentation to a user 120. The display device 500 may include a display screen, AR glasses lens, CRT, LCD, plasma screen, touch screen, TV, projector, tablet, or other suitable display components. The user device 115 may include a software application 510 that is downloaded and runs on the user device 115. In some implementations, the software application 510 may be associated with the vehicle 105 or an entity associated with the vehicle 105 (e.g., manufacturer, retailer, maintenance provider). In an example, the software application 510 may enable the user device 115 to communicate with the computing platform 110 and the services thereof.

[00100] The user device 115 may be configured to pair with the vehicle 105 via a short-range wireless protocol. The short-range wireless protocol may include, for example, at least one of Bluetooth ®. Wi-Fi, ZigBee, UWB, IR. The user device 115 may pair with the vehicle 105 through one or more known pairing techniques. For example, the user device 115 and the vehicle 105 may exchange information (e.g., addresses, device names, profiles) and store such information in their respective memories. Pairing may include an authentication process whereby the user 120 validates the connection between the user device 115 and the vehicle 105.

[00101] Once paired, the vehicle 105 and the user device 115 may exchange signals, data, etc. through the established communication channel. For example, the head unit 347 of the vehicle 105 may exchange signals with the user device 115.

[00102] The technology of the present disclosure allows the vehicle computing system 200 to preserve its computing capabilities by utilizing shallow learning models and a sequential aspect (e.g., bag of apps) to standardize / normalize complex contextual user data to more efficiently train one or more machine-learned models. More particularly, the vehicle computing system 200 may incorporate a sequential aspect that provides a numerical representation (e.g., binary representation) of the sequential usage of software applications 245 during a trip. As described herein, this technology can overcome potential inefficiencies introduced by using deep learning models and more complex data sets. Additionally, the technology preserves the privacy and sensitivity of data generated on-vehicle by allowing for more efficient on-vehicle training of the machine-learned models while also limiting the amount of data needed to effectively train the models.

[00103] FIG. 6 illustrates an example architecture of a multi-class machine-learned model according to an embodiment hereof. Those of ordinary skill in the art, using the disclosures provided herein, will understand that the elements of any of the architecture discussed herein may be adapted, rearranged, expanded, omitted, combined, or modified in various ways without deviating from the scope of the present disclosure. FIG. 6 is described with reference to elements / terms described with respect to other systems and figures, for example illustrated purposes and is not meant to be limiting. One or more portions of the example architecture 600 may be implemented additionally, or alternatively, by other systems. For example, architecture 600 may be implemented by one or more servers of the remote computing platform 110, third-party computing platform 125, etc.

[00104] The example architecture 600 may depict an example implementation of a multi-class machine-learned model 604. A multi-class model 604 may include a singular machine-learned model which may predict a sequence of future user actions across a plurality of classes (e.g., software applications 245). In an embodiment the multi-class model 604 may be comprised of one or more sub-models. The multi-class model 604 may receive, as input, user data 601 including contextual features 602 and a bag of apps 603. The user data 601 may be pre-processed to generate the bag of apps 603 and extract the contextual features 602 prior to input into the multi-class model 604. Pre-processing the user data 601 may provide standardization / normalization to user data 601. The multi-class model 604 may process the contextual features 602 and the bag of apps 603 and generate output indicative of a user preference data. User preference data may include a ranking (e.g., top 3, top 5, etc.) of the applications (e.g., software applications 245) which are associated with anticipated user actions.

[00105] The multi-class model 604 may be or may otherwise include various machine-learned models such as, for example, shallow learning models. Example shallow learning models may include linear models (e.g., SGDClassifier, Logistic Regression, Naive Bayes (Gaussian, Bernoli), etc.), tree-based models (e.g., decision trees and its variants, etc.), boosting / ensemble models (e.g., XGBoost, RandomForest, CatBoost, LightGBM, AdaBoost, etc.), or neural-based models (e.g., MLP (multi-layer perceptron), etc.). In an embodiment, the multi-class model 604 may include regression networks, generative adversarial networks, neural networks (e.g., deep neural networks), support vector machines, k-nearest neighbors models, Bayesian networks, or other types of models Example neural networks include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or other forms of neural networks. As another example, the multi-class model 604 may include generative models, such as stable diffusion models, generative adversarial networks (GAN), GPT models, etc.

[00106] The user data 601 may include data (e.g., sensor data, log data etc.) generated by the user 120 while previously operating the vehicle 105 during a trip. For instance, as the user 120 begins a trip (e.g., enters the vehicle, starts the vehicle, etc.) the vehicle computing system 200 may capture vehicle data 330 indicating user actions taken by the user 120. By way of example, the user 120 may enter the vehicle 105 and adjust the air conditioner (AC) (e.g., by interacting with a climate control software application), activate the GPS (e.g. by interacting with a navigation software application) and adjust the seat to turn on a massage setting (e.g., by interacting with a comfort control software application). The various sub-systems (e.g., sensor system 305, positioning system 315, HMI system 340, etc.) of the vehicle computing system 200 may capture user data 601 (e.g., vehicle data 330, etc.) indicating the sequence of actions taken by the user 120.

[00107] In an embodiment, the user data 601 may be pre-processed to generate a bag of apps 603 (e.g., sequence data) and extract the contextual features 602. The user data 601 may be pre-processed instantaneously (e.g., upon generation of the data) or subsequently by the vehicle computing system 200. In an embodiment, the multi-class model 604 may pre-process the user data 601 as an initial step prior to processing.

[00108] For example, the user data 601 may be analyzed (e.g., pre-processed) to generate a bag of apps 603. The bag of apps 603 may include sequence data indicative of the sequence of actions (e.g., adjusting the AC, activation GPS, adjusting the seat, etc.) taken by the user 120 using binary values. For instance, the bag of apps 603 may include a bag of apps vector with a vector size equal to the number of software applications 245 available to the user 120 during a trip. The bag of apps vector may be populated with a value of “1” if a respective software application 245 was used during the trip by the user 120 and a value of “0” if there was no usage. The bag of apps 603 may include a plurality of bag of apps vectors which represent the sequential usage of software applications during the trip.

[00109] By way of example, a bag of apps vector may have a vector size of 6 based on the user 120 having 6 software applications 245 (e.g., applications 1-6) available during a trip. If the user uses application 2 and application 3 during a trip. The bag of apps 603 may include a first bag of apps vector with binary values of [0, 1,0,0, 0, 0] to indicate that the user used application 2 first, and a second bag of apps vector with binary values of [0, 1, 1, 0, 0, 0] to indicate that the user used application 3 subsequently to using application 2.

[00110] The user data 601 may be analyzed (e.g., pre-processed) to extract contextual features 602 (e.g., the inside and outside temperature, latitude / longitude, hour of the day / day of week, time since start of trip, etc.) from the user data 601. The contextual features 602 may indicate the context (e.g., conditions, setting, etc.) in which the respective actions of the sequence of actions were taken by the user 120. For instance, the contextual features 602 may include context associated with the trip. Example context associated with the trip may include the location of the vehicle 105 (e.g., located at work, home, gym, etc.), the temperature observed during the trip, the day / time stamp (e.g., early morning, late afternoon on a weekday or weekend), etc. The context associated with the trip may provide additional information to better understand and anticipate user behavior.

[00111] The user data 601 including the contextual features 602 and the bag of apps 603 may be accessed or otherwise inputted into the multi-class model 604. The multi-class model 604 may utilize the user data 601 to anticipate the user’s behavior. For instance, the multi-class model 604 may determine, based on the sequence data (e.g., bag of apps 603), context features associated with the respective actions of the plurality of actions. For example, the multi-class model 604 may associate contextual features 602 with usage of a particular software application 245.

[00112] Byway of example, the multi-class model 604 may determine based on the user data 601 including the bag of apps 603 that the user 120 used a radio software application 4 seconds after beginning a trip (e.g., entering the vehicle, starting the vehicle etc.). For instance, the bag of apps 603 may include a first bag of apps vector populated with a binary value of “1” associated with the radio software application. The multi-class model 604 may determine contextual features 602 associated with the usage of the radio software application. For instance, the multi-class model 604 may concatenate the location, date / time stamp, temperature, etc. for usage of the radio application. In some embodiments, the multi-class model 604 may determine contextual features 602 associated with a plurality of sequential user actions (e.g., usage of a plurality of software applications 245) by concatenating the values indicated by the bag of apps vectors with contextual features 602 during, immediately before, and / or immediately after the user action.

[00113] The multi-class model 604 may predict, based on the sequence data (e.g., bag of apps 603) and the context data (e.g., contextual features 602), user preference data (e.g., future user behavior). The user preference data may be indicative of a ranking of software applications 245 which facilitate an anticipated future behavior of the user 120. The ranking may be based on the probability of the user 120 performing a future action (e.g., via a software application 245) during the trip. For instance, the multi-class model 604 may determine the probability or likelihood of application usage during the trip using an association score. The association score may be a numerical representation of the similarities between the user action taken and the associated contextual features 602 of previous trips to the current trip.

[00114] For example, the multi-class model may associate the software applications 245 and contextual features 602 with different weights which can be used in determining the association score. By way of example, the multi-class model 604 may assign a greater weight to the navigation application using the location and day / time features rather than temperature. For instance, the multi-class model 604 may determine the probability of the navigation application usage may be higher when the navigation application was used with the same (e.g., or substantially the same) location (e.g., work, etc.), day (weekday, etc.), and time (5 PM, etc.) as a previous trip. For example, the temperature feature may vary greatly among previous trips, but the location and day / time may be consistent among previous trips. As such the multi-class model may associate a higher association score with the location and day / time features for the navigation application to more accurately anticipate the user 120 using the navigation application.

[00115] In an embodiment, the multi-class model 604 may determine probability scores for each of the available software applications 245 during the trip using the association score. For instance, the multi-class model 604 may generate user preference data and an associated probability score. The probability score may be any numerical value which indicates a likelihood or probability of usage. The software applications 245 may be ranked based on having the highest probability of usage. In some embodiments, multi-class model 604 may determine the ranking based on a threshold probability (e.g., greater than 50%, 60%, etc.).

[00116] By way of example, the multi-class model 604 may determine the probability of application usage across a plurality of software applications and determine, based on the contextual features 602 and bag of apps 603, user preference data. For instance, probably scores of 5, 6, 7, and 2 may be associated with 4 software applications 245 respectively. The user preference output indicative of the probability scores may be compared to a threshold probability score to determine the ranking. For instance, a threshold probability score of 5 may indicate a ranking top-3 ranking of applications based on a probability score of 2 being below the threshold probability. As such the 4 software applications 245 may be ranked according to their respective probability scores (e.g., 7, 6, 5).

[00117] The user preference data may be translated as the ranking of available software applications 245 where the top ranked software applications 245 (e.g., top 2, top 3, top 7, etc.) are recommended applications 605. The recommended applications 605 may be received by the HMI system 340 as content 335 and used to generate or update a user interface (e.g., display device 345, etc.) to display the anticipated applications to be used by the user 120 during the trip.

[00118] As the user 120 interacts with the software applications during the trip, user feedback data may be generated. The user feedback data may provide various types of feedback on the accuracy of the predicted user behavior (e.g., recommended applications 605). The user feedback data may be used to further train the multi-class model 604. An example of user feedback data is further described with reference to FIG. 8.

[00119] FIG. 7 illustrates an example architecture of a binary machine-learned model according to an embodiment hereof. Those of ordinary skill in the art, using the disclosures provided herein, will understand that the elements of the architecture discussed herein may be adapted, rearranged, expanded, omitted, combined, or modified in various ways without deviating from the scope of the present disclosure. FIG. 7 is described with reference to elements / terms described with respect to other systems and figures, for example illustrated purposes and is not meant to be limiting. One or more portions of the example architecture 700 may be implemented additionally, or alternatively, by other systems. For example, architecture 700 may be implemented by one or more servers of the remote computing platform 110, third-party computing platform 125, etc.

[00120] The example architecture 700 may depict an example implementation of a plurality of binary machine-learned models 701A-B which may generate a binary output indicative of prediction of a future action (e.g., application usage) for a respective software application 245 of a plurality of software applications 245. For instance, the plurality of binary models 701A-B may be associated with a particular software application 245 of a plurality of software applications 245 available to the user 120 during a trip. Similar to the multi-class model 604, respective binary models 701 A-B may receive as input, user data 601 including contextual features 602 and a bag of apps 603. The user data 601 may be pre-processed to generate the bag of apps 603 and extract the contextual features 602 prior to being input into the respective binary models 701 A-B which allows for standardization / normalization to the user data 601. The respective binary models 701 A-B may process the contextual features 602 and the bag of apps 603 and generate a binary output indicative of whether the associated software application 245 has a threshold probability of usage during the trip. The output including the binary values may be received by a rules-based model 702 which may be configured to process and sort the binary output to determine a ranking (e.g., top 3, top 5, etc.) of the applications (e.g., software applications 245).

[00121] The binary models 701 A-B may be or may otherwise include various machine-learned models such as, for example, shallow learning models. Example shallow learning models may include linear models (e.g., SGDClassifier, Logistic Regression, Naive Bayes (Gaussian, Bernoli), etc.), tree-based models (e.g., decision trees and its variants, etc.), boosting / ensemble models (e.g., XGBoost, RandomForest, CatBoost, LightGBM, AdaBoost, etc.), or neural-based models (e.g., MLP (multi-layer perceptron), etc.). In an embodiment, the binary models 701 A-B may include regression networks, generative adversarial networks, neural networks (e.g., deep neural networks), support vector machines, k-nearest neighbors models, Bayesian networks, or other types of models. Example neural networks include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or other forms of neural networks. As another example, the binary models 701A-B may include generative models, such as stable diffusion models, generative adversarial networks (GAN), GPT models, etc.

[00122] As described herein the example architecture 700 may include N number of binary models 701A-B. For instance, the binary models 701A-B may be associated with respective software applications 245 available to the user 120. For instance, if there are 10 software applications 245 available to the user 120 during the trip, there may be 10 corresponding binary models 701 A-B configured to generate a binary output indicative of a probability of the corresponding software application 245 being used during the trip. For instance, as new software applications 245 are installed or downloaded (e.g., via OTA software updates, app stores, etc.), a corresponding binary model 701A-B may initialize to determine the probability of usage during future trips.

[00123] As described herein, the user data 601 may be pre-processed to generate a bag of apps 603 (e.g., sequence data) and extract the contextual features 602. The user actions (e.g., application usage) indicated by the user data 601 may be segmented prior to being input into the binary models 701A-B. For instance, the binary models 701A-B may only receive user data 601 associated with the corresponding software application 245. By way of example, binary model 701A may correspond to the comfort settings application and binary model 701B may correspond to the climate control application. As such binary model 701A may generate a binary output indicative of the probability of the comfort settings application being used during the trip and binary model 70IB may generate a binary output indicative of the probability of the climate control application being used during the trip.

[00124] The binary output may be represented as any numerical value. In an embodiment, the binary output may be represented as a “0” to indicate a probability of usage below a threshold or a “1” to indicate a probability of usage above a threshold. For instance, the binary models 701A-B may be trained to determine a probability score for the corresponding software applications 245. The probability score may be determined using the bag of app 603 and contextual features 602 which indicate the number of times and context by which the corresponding software application 245 was accessed during a trip. The probability score may be any numerical value on a scale which indicates a probability of usage. For instance, a probability score of 7 on a scale of 10 or .7 on a scale of 1.0 may indicate a 70% probability of usage. The probability score may be determined to consistently measure the probability of usage during a trip.

[00125] By way of example, binary model 70IB may correspond to the radio software application 245. Binary model 701B may receive a bag of apps 603 and contextual features 602 associated with the radio software application 245. For instance, the bag of apps 603 may include 5 bag of apps vectors indicating the radio software application 245 was used 3 times during each of 3 trips. Binary model 70IB may also determine that the radio software application 245 was used a few seconds after the navigation software application based on the bag of apps vectors and contextual features 602. For instance, the bag of apps 603 may include a first bags of app vector with the values [0, 1, 1, 0], a second bag of apps vector with values [0, 1, 1,0] and a third bag of apps vector with values [0, 1, 1, 0]. The first “1” may represent the navigation software application and the second “1” may represent the radio software application.

[00126] In addition to the bag of apps 603 indicating a pattern among a sequence of user behavior (e.g.. using the navigation and subsequently the radio), the binary model 70IB may also correlate contextual features 602 indicating the context of the sequence of actions. For instance, binary model 70 IB may determine the sequence of actions occurred around 8 AM, at a “home” location (e.g., semantic location), during a weekday. Based on the bag of apps 603 and contextual features 602, the binary model 701B may predict a binary output of a “1” based on similar context (e.g., around 8 AM, “home” location, during the week). The binary output may be associated with a probability score indicating the probability of the binary output occurring during the trip.

[00127] For instance, the plurality of binary models 701A-B may each output a binary output based on determining a probability of usage and a rules-based model 702 may receive the output to determine a ranking of software applications 245 by processing and sorting the binary output. The rules-based model 702 can include a heuristic set of rules. The set of rules can be developed to process the binary outputs and sort the output into a vector as described herein.

[00128] In some examples, the rules-based model 702 may include one or more machine-learned models. This can include one or more machine-learned models trained to determine a ranking of software applications given a plurality of binary inputs. The rules-based model 702 can be or can otherwise include various machine-learned models such as, for example, regression networks, generative adversarial networks, neural networks (e.g., deep neural networks), support vector machines, decision trees, ensemble models, k-nearest neighbors models, Bayesian networks, or other types of models including linear models or non-linear models. Example neural networks include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or other forms of neural networks. The one or more models can be trained through the use of one or more model trainers and training data. The model trainers can be trained using one or more training or learning algorithms to train the models to determine a ranking of software applications 245 based on binary inputs.

[00129] In an embodiment, the output of the binary model 701B may be associated with a threshold probability (e.g., likelihood). A threshold probability may include a numerical representation of the probability of the output occurring during a trip. For instance, the binary models 701A-B may generate output for the radio software application as 0.45 and 0.60 for the navigation software application. As such the rules-based model 702 may receive the binary output and determine a ranking of [navigation, radio, etc.]. The ranking may indicate that the navigation software application is most likely to be used first during the trip and the radio software application is most likely to be used subsequent to the navigation software application during the trip. In an embodiment, the ranking may be indicative of the top 2, 3, 5, etc. software applications 245 which are associated with a binary output that satisfies a probability threshold.

[00130] The rules-based model 702 may be configured to compare the binary output to the probability threshold to determine the ranking. By way of example, the rules-based model 702 may determine a predetermined or dynamic probability threshold of 0.4. Software applications 245 which are associated with a binary output below 0.4 may be removed or filtered from the ranking. The rules-based model 702 may be configured to rank the remaining software applications 245 which satisfy the probability threshold based on their numerical values. For instance, the comfort settings software application may be associated with a binary output of 0.2. If the probability threshold is 0.4, the comfort settings software application may be removed from the ranking based on being below the probability threshold. In another example, the comfort settings software application is associated with a binary output of 0.8. The rules-based model 702 may determine a ranking (e.g., recommended top-3, etc.) of [comfort settings, navigation, radio] based on the navigation software application being associated with a binary output of 0.6 and the radio software application being associated with a binary output of 0.45. The rules-based model 702 may output data indicative of the recommended applications 605 to generate or update a display (e.g., display device 345) within the vehicle 105.

[00131] For instance, FIG. 8 illustrates an example user interface according to an embodiment hereof. The example user interface 800 depicts user interface elements 801A-C associated with recommended applications 306 displayed on the display device 345 within the vehicle 105. While examples here describe the user interface elements 801A-C associated with recommended applications 306 as displayed on the display device 345, the present disclosure is not limited to such embodiment. The user interface elements 801A-C associated with recommended applications 306 may be displayed on other devices such as the user device 115, head unit 347, etc.

[00132] As described herein, the HMI system 340 may receive the output indicative of the recommended applications 306 and generate content 335 to generate or update a display of the display device 345. For instance, the HMI system may generate one or more command instructions to generate content 335. The content 335 may be associated with the recommended applications 605 and include a particular arrangement of software applications 245 displayed via the display device 345. For instance, the content 335 including the recommended applications 605 may improve the functioning of the display device 345 by displaying the recommended applications 605 in a more prominent position (e.g., larger than the other applications, at the top, in front of the other applications, etc.). This arrangement may allow the user 120 to access the recommended applications 605 more easily than the other software applications 245 without having to navigate to other screens. As such the recommended applications 605 may be displayed (e.g., arranged) in a manner that allows more content 335 to be displayed via the display device 345 during the trip.

[00133] As the user 120 begins the trip and performs one or more user actions (e.g., interacting with one or more software applications 245), the vehicle computing system 200 may capture user feedback data. The user feedback data may indicate a sequence of actions taken by the user 120 during the trip and the context by which the sequence of actions were taken. In an embodiment, the user feedback data may be used as training data to further train machine-learned models to anticipate the user’s behavior on future trips.

[00134] For instance, FIG. 9 illustrates an example dataflow pipeline for generating training data according to an embodiment hereof. The dataflow pipeline 900 may be performed by a computing system described with reference to the other figures. In an embodiment, the dataflow pipeline 900 may be performed by the control circuit of a vehicle computing system 200 of FIG. 1. One or more portions of the dataflow pipeline 900 may be implemented as an algorithm on the hardware components of the devices described herein. For example, the dataflow pipeline 900 may be implemented as operations / instructions that are executable by computing hardware.

[00135] FIG. 9 illustrates elements performed in a particular order for purposes of illustration and discussion. Those of ordinary skill in the art, using the disclosures provided herein, will understand that the elements of any of the dataflow pipeline discussed herein may be adapted, rearranged, expanded, omitted, combined, or modified in various ways without deviating from the scope of the present disclosure. FIG. 9 is described with reference to elements / terms described with respect to other systems and figures, for example illustrated purposes and is not meant to be limiting. One or more portions of dataflow pipeline 900 may be performed additionally, or alternatively, by other systems. For example, dataflow pipeline 900 may be performed by the remote computing platform 110, third-party computing platform 125, etc.

[00136] In the example dataflow pipeline 900, the recommended applications 605 determined by the one or more machine-learned models (e.g., multi-class model 604, binary models 701A-B, etc.) may be presented to the user 120 via the display device 345 during the trip. The user's behavior during the trip may be captured as user feedback data 901. User feedback data 901 may include implicit or explicit actions taken by the user 120 during the trip. In an embodiment, user feedback data 901 may include raw sensor data or log data generated by the user 120 during a trip. For instance, the user 120 may decide to interact with the recommended applications 605 suggested to the user 120 in order of ranking or out of order, interact with other software applications 245 not suggested (e.g., applications other than the recommended applications 605), or take no action at all.

[00137] Based on the user’s behavior during the trip, user feedback data 901 may be quantified as negative feedback data 902 or positive feedback data 903. By quantifying all implicit (e.g., the user takes no action, navigates to another application, etc.) and explicit (e.g., engage action, etc.) user behavior, user feedback data 901 may holistically be used to further train the machine-learned models which generated the recommended applications 605.

[00138] For instance, negative feedback data 902 may include neutral feedback 902A and dismissive feedback 902B. Neutral feedback 902A may be implicit user behavior where the user 120, despite being suggested or otherwise presented recommended applications 605, did not use the suggested application during the trip. For instance, neutral feedback 902A may indicate that the suggestion of the recommended application 605 was neither a correct nor an incorrect anticipation. However, the neutral feedback 902A may nevertheless be considered negative feedback data 902 because the user 120 did not use the suggested application during the trip. Dismissive feedback 902B may be explicit user behavior where the user 120, once presented the recommended applications 605 takes engage action to remove one or more of the suggested applications from view . For instance, the user 120 may “swipe-away” one or more suggested applications from view. In an embodiment, the user 120 may take other actions such as deleting the application, using voice commands, hand gestures, or any other action to indicate the intent to remove the suggested application from view or otherwise disregard the application.

[00139] Negative feedback data 902 may be labeled and weighted to indicate the user feedback data 901 is negative. For instance, negative feedback data 902 may be labeled as a “0” or other indicators to consistently categorize user feedback data 901 as negative feedback data 902. Negative feedback data 902 may also be weighted based on being neutral feedback 902A or dismissive feedback 902B. For instance, a weighted value such as (X) or any other value may be applied to neutral feedback 902A and dismissive feedback 902B. The weight (e.g., X, etc.) may be associated with each type of negative or neutral user action (e.g., negative user feedback)

[00140] By way of example, neutral feedback 902A may be assigned a weighted value of XI to indicate a neutral or “no change” effect. For instance, since the user 120 did not engage, use, or reject the recommended application 605 during the trip, this type of implicit feedback may be weighted such that no change in training parameters may be needed. In contrast, dismissive feedback 902B may be assigned a weighted value of X2 to indicate stronger (e.g., doubling effect, etc.) or higher negative feedback data 902. For instance, a dismissal of a recommended application 605 may be weighted more heavily due to the clear intentions of the user 120. The X2 value may be weighted such that one or more parameters may be updated to correct the determination of anticipated user behavior.

[00141] Positive feedback data 903 may also include implicit and explicit user behaviors. For instance, positive feedback data 903 may include engage feedback 903A and long way feedback 903B. Engage feedback 903A may include explicit user behavior where the user 120 click on the recommended applications 605 presented during the trip. Engage feedback 903A may include any instances where the user 120 confirms the suggested application through its usage during a trip. For example, the recommended applications 605 may be suggested to the user 120 via voice prompts, augmented reality displays, etc. and the user 120 may respond using voice commands or interacting with augmented reality icons.

[00142] Long way feedback 903B may include user behavior where the user 120 put for additional effort to use an available application or a recommended application 605. For instance, the user 120 may access a software application using a longer, more tedious way. By of example, the user 120 may search (e.g., via voice commands, keyboard, etc.) for a particular application which was prominently displayed as a recommended application 605. In another example, the user interface of the display device 345 may include a pathway to access a particular application and the user 120 may nonetheless take a longer way to access the particular application.

[00143] Positive feedback data 903, in contrast to negative feedback data 902, may be labeled as a "I ” or other indicators to consistently categorize user feedback data 901 as positive feedback data 903. Positive feedback data 903 may be weighted based on being engage feedback 903A or long way feedback 903B. For instance, a value such as (a) or any other values may be applied to engage feedback 903A and long way feedback 903B. The value (a) may indicate a positive weight which may be assigned to user behaviors which indicate the strength or level of the user’s engagement during a trip. For instance, engage feedback 903A may be assigned a weight of al to indicate a positive level of engagement and long way feedback 903B may be assigned a weight of a2 to indicate a higher or stronger level of user engagement based on the user 120 putting forth additional effort to access and use the software application 245 (e.g., the long way) during the trip. In an embodiment, the a2 weight may have a doubling weighted effect compared to the engage feedback 903A.

[00144] Once negative feedback data 902 and positive feedback data 903 have been processed (e.g., labeled, weighted, etc.), the user feedback data 901 may be processed further to generate training data labels. For instance, a label generator 908 may be used to generate training labels for the user feedback data 901 once it has been properly classified (e.g., labeled) and weighted. Training labels may include a description that informs the machine-learned models (e.g., multiclass model 604, binary models 701A-B, etc.) what a particular data (e.g., negative feedback data 902, positive feedback data 903, etc.) represents so that it may learn from the example.

[00145] The label generator 908 may annotate the negative feedback data 902 and positive feedback data and generate training labels 906. In an embodiment, label generator 908 may annotate the negative feedback data 902 and positive feedback data 903 based on weights applied to the negative feedback data 902 and positive feedback data 903 respectively. For instance, training labels 906 may be generated based on user feedback data 901 that is stronger (e.g., long way feedback 903B, dismissive feedback 902B, etc.). For example, generating training labels 906 based on the strength (e.g., weights) may provide a higher impact during training. Additionally or alternatively, generating a higher number of training data samples having respective training labels 906 for instances when user feedback data 901 is stronger (e.g., long way feedback 903B, dismissive feedback 902B, etc.) may provide a higher impact during training.

[00146] The training labels 906 may represent the ground truth against which the machine-learned model’s output data (e.g., recommended applications 605) is compared. For example, a training sample 907 including a training context and bag of apps 905 along with the training labels 906 may be used to train the machine-learned models. The training context and bag of apps 905 may include the contextual features and bag of apps associated with the user feedback data 901. In an embodiment, the training sample 907 may be input into the machine-learned models (e.g., multi-class model 604, binary models 701A-B) as part of training using a training system. An example of a training system is further described with reference to FIG. 11.

[00147] In an embodiment, the machine-learned models (e.g., multi-class model 604, binary models 701A-B) may be training in between trips. By way of example, the machine-learned models may determine user preference data indicative of recommended applications 605 on a first trip and receive user feedback data 901 during the first trip. When the first trip ends, the training sample 907 may be used to train the machine-learned models prior to the start of the second trip. As such the machine-learned models may generate updated user preference data including updated recommended applications based on the user feedback data 901 providing a ground truth sample (e.g., training labels, 906, etc.).

[00148] In some embodiments, training may occur during a trip. For instance, the user 120 may provide user feedback during a first portion of a trip and the machine-learned models may generate a training sample 907 during the trip and generate an updated user preference data including updated recommended applications. In other embodiments, training may be performed offline (e.g., with an updated model being provided during OTA software updates, etc.).

[00149] FIG. 10 illustrates a flowchart diagram of an example method according to an embodiment hereof. The method 1000 may be performed by a computing system described with reference to the other figures. In an embodiment, the method 1000 may be performed by the control circuit of a vehicle computing system 200 of FIG. 1. One or more portions of the method 1000 may be implemented as an algorithm on the hardware components of the devices described herein. For example, the steps of method 1000 may be implemented as operations / instructions that are executable by computing hardware.

[00150] FIG. 10 illustrates elements performed in a particular order for purposes of illustration and discussion. Those of ordinary skill in the art, using the disclosures provided herein, will understand that the elements of any of the methods discussed herein may be adapted, rearranged, expanded, omitted, combined, or modified in various ways without deviating from the scope of the present disclosure. FIG. 10 is described with reference to elements / terms described with respect to other systems and figures, for example illustrated purposes and is not meant to be limiting. One or more portions of method 1000 may be performed additionally, or alternatively, by other systems. For example, method 1000 may be performed by the remote computing platform 110, third-party computing platform 125, etc.

[00151] In an embodiment, the method 1000 includes a step 1002, or otherwise begin by accessing sequence data indicative of a plurality of actions previously taken in sequence by a user within a vehicle. For instance, user data 601 may be processed to generate a bag of apps 603. The bag of apps 603 may include sequence data indicative of the sequence of actions (e.g., adjusting the AC, activation GPS, adjusting the seat, etc.) taken by the user 120 using binary values. For instance, the bag of apps 603 may include a bag of apps vector with a vector size equal to the number of software applications 245 available to the user 120 during a trip. The bag of apps vector may be populated with a value of “1” if a respective software application 245 was used during the trip by the user 120 and a value of “0” if there was no usage. The bag of apps 603 may include a plurality of bag of apps vectors which represent the sequential usage of software applications during the trip.

[00152] In an embodiment, the method 1000 may include a step 1004, or otherwise continue by determining, based on the sequence data, context data associated with the plurality of actions, wherein the context data is indicative of at least a time or a location of respective actions of the plurality of actions. The user data 601 may be analyzed (e.g., pre-processed) to extract contextual features 602 (e.g., the inside and outside temperature, latitude / longitude, hour of the day / day of week, time since start of trip, etc.) from the user data 601. The contextual features 602 may indicate the context (e.g., conditions, setting, etc.) in which the respective actions of the sequence of actions were taken by the user 120. For instance, the contextual features 602 may include context associated with the trip. Example context associated with the trip may include the location of the vehicle 105 (e.g., located at work, home, gym, etc.), the temperature observed during the trip, the day / time stamp (e.g., early morning, late afternoon on a weekday or weekend), etc. The context associated with the trip may provide additional information to better understand and anticipate user behavior.

[00153] In an embodiment, the method 1000 may include a step 1006, or otherwise continue by determining, using one or more machine-learned models and based on the sequence data and the context data, user preference data indicative of a ranking of computing programs associated with the vehicle, wherein the ranking is based on a probability of user interaction. For instance, the multi-class model 604 may determine, based on the sequence data (e.g., bag of apps 603) and the context data (e.g., contextual features 602), user preference data (e.g., future user behavior). The user preference data may be indicative of a ranking of software applications 245 which facilitate an anticipated future behavior of the user 120. The ranking may be based on a probability of the user 120 performing a future action (e.g., via a software application 245) during the trip.

[00154] By way of example, the multi-class model 604 may associate the software applications 245 and contextual features 602 with different weights which can be used in determining the association score. By way of example, the multi-class model 604 may assign a greater weight to the climate control application using the temperature feature rather than the location and day / time features. For instance, the multi-class model 604 may determine the probability of the climate control application usage may be higher when the climate application was used under the same (e.g., or substantially the same) temperature as one or more previous trips. For example, the location and day / time features may vary greatly among previous trips, but the temperature may be relatively consistent among previous trips. As such the multi-class model 604 may associate a higher association score with the temperature features for the climate control application to more accurately anticipate the user 120 using the climate control application.

[00155] In an embodiment, the multi-class model 604 may determine probability scores for each of the available software applications 245 during the trip using the association score. The probability score may be determined using the bag of apps 603 and contextual features 602 which indicate the number of times and context by which the corresponding software application 245 was accessed during a trip. The probability score may be any numerical value on a scale which indicates a probability of usage. For instance, a probability score of 7 on a scale of 10 or .7 on a scale of 1.0 may indicate a 70% probability of usage. The probability score may be determined to consistently measure the probability of application usage during a trip.

[00156] For example, the multi-class-model 604 may analyze the user data 601 and determine user preference data. The user preference data may be indicative of probability scores for each of the available software applications 245. By way of example, the multi-class model 604 may determine user preference data indicative of probability scores of 0.5, 0.2, and 0.9 for the comfort control software application, navigation software application, and climate control software application respectively. The user preference data may be translated as a ranking of available software applications 245 where the top ranked software applications 245 (e.g., top 2, top 3, top 7, etc.) are recommended applications 605.

[00157] For instance, the output of the multi-class model 604 may be associated with a probability threshold. The probability threshold may be used to filter and rank the available software applications 245. By way of example, the probability threshold may be predetermined or dynamically configured as 0.4. Based on probability scores of 0.5, 0.2, and 0.9 for the comfort control software application, navigation software application, and climate control software application respectively being determined by the multi-class model 604, the multi-class model 604 may determine a top 2 ranking of climate control and comfort control based on the navigation software application having a probability score below the probability threshold. For instance, the probability score of 0.9 associated with the climate control software application and the 0.5 probability score associated with the comfort control software application may indicate that the user 120 is more likely to interact with the climate controls prior to the comfort controls during the trip.

[00158] In an embodiment, the method 1000 may include a step 1008, or otherwise continue by outputting one or more signals to initiate a display of content indicative of the user preference data. For instance, the multi-class model 604 may determine the probability of application usage across a plurality of software applications 245 based on the contextual features 602 and bag of apps 603 user preference data. The multi-class model 604 may output user preference data which may be translated as the ranking (e.g., recommended applications 605) of available software applications 245 where the top ranked software applications 245 (e.g., top 2, top 3, top 7, etc.) are used to generate or update a user interface display.

[00159] By way of example, the multi-class model 604 may output user preference data indicative of the recommended applications 605. The recommended applications 605 may be received by the HMI system 340 as used to generate content 335. The content 335 may include one or more command instructions which generate or update a user interface (e.g., display device 345, etc.) to display the anticipated applications (e.g. recommended applications 605) to be used by the user 120 during the trip.

[00160] In an embodiment, the method 1000 may include a step 1010, or otherwise continue by receiving ground truth data, the ground truth data indicative of user feedback associated with the ranking of the computer programs. For instance, the multi-class model 604 may utilize one or more training samples 907 to be further trained. Training samples may include training labels 906 that represent the ground truth associated with output (e.g., user preference output, recommended applications 605, etc.) of the multi-class model 604. For instance, the training samples may include a comparison of the ground truth (e.g., actual usage) to the predicted output of multi-class model 604.

[00161] By way of example, a training sample 907 including a training context and bag of apps 905 along with the training labels 906 may be used to train the multi-class model 604. The training context and bag of apps 905 may include the contextual features and bag of apps associated with the user feedback data 901. In an embodiment, the training sample 907 may be input into the multi-class model 604 and trained using a training system.

[00162] In an embodiment, the method 1000 may include a step 1012, or otherwise continue by, based on the ground truth data, sequence data, and context data, determining using the one or more machine-learned models updated user preference data. For instance, the multi-class model 604 may be trained using a training system. By way of example, the training sample, 907 may be input into the multi-class model 604 by a training system configured to train the multi-class model 604. The training system may iteratively evaluate the output of the multi-class model 604 using various training techniques such as backwards propagation of errors. The training sample 907 may include a plurality of labels which have a higher impact on training.

[00163] For instance, training samples 907 may include weighted user feedback data 901 which facilitates an increase in labels within the training sample 907. The training system may iteratively train the multi-class model 604 based on the additional training labels. In an embodiment, the multi-class model 604 may be trained in between trips, on a scheduled cadence, or on-demand. In other embodiments, training may be performed offline (e.g., with an updated model being provided during OTA software updates, etc.)

[00164] FIG. 11 illustrates a block diagram of an example computing system 7000 according to an embodiment hereof. The system 7000 includes a computing system 6005, a remote computing system 7005, a user device 9005, and a training computing system 8005 that are communicatively coupled over one or more networks 9050.

[00165] The computing system 6005 may include one or more computing devices 6010 or circuitry. For instance, the computing system 6005 may include a control circuit 6015 and a non-transitory computer-readable medium 6020, also referred to herein as memory. In an embodiment, the control circuit 6015 may include one or more processors (e.g., microprocessors), one or more processing cores, a programmable logic circuit (PLC) or a programmable logic / gate array (PLA / PGA), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or any other control circuit. In some implementations, the control circuit 6015 may be part of, or may form, a vehicle control unit (also referred to as a vehicle controller) that is embedded or otherwise disposed in a vehicle (e.g., a Mercedes-Benz® car or van). For example, the vehicle controller may be or may include an infotainment system controller (e.g., an infotainment head-unit), a telematics control unit (TCU), an electronic control unit (ECU), a central powertrain controller (CPC), a charging controller, a central exterior &interior controller (CEIC), a zone controller, or any other controller. In an embodiment, the control circuit 6015 may be programmed by one or more computer-readable or computer-executable instructions stored on the non-transitory computer-readable medium 6020.

[00166] In an embodiment, the non-transitory computer-readable medium 6020 may be a memory device, also referred to as a data storage device, which may include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. The non-transitory computer-readable medium 6020 may form, e.g., a hard disk drive (HDD), a solid state drive (SDD) or solid state integrated memory, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), dynamic random access memory (DRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), and / or a memory stick.

[00167] The non-transitory computer-readable medium 6020 may store information that may be accessed by the control circuit 6015. For instance, the non-transitory computer-readable medium 6020 (e.g., memory devices) may store data 6025 that may be obtained, received, accessed, written, manipulated, created, and / or stored. The data 6025 may include, for instance, any of the data or information described herein. In some implementations, the computing system 6005 may obtain data from one or more memories that are remote from the computing system 6005.

[00168] The non-transitory computer-readable medium 6020 may also store computer-readable instructions 6030 that may be executed by the control circuit 6015. The instructions 6030 may be software written in any suitable programming language or may be implemented in hardware. The instructions may include computer-readable instructions, computer-executable instructions, etc. As described herein, in various embodiments, the terms “computer-readable instructions” and “computer-executable instructions” are used to describe software instructions or computer code configured to carry out various tasks and operations. In various embodiments, if the computer-readable or computer-executable instructions form modules, the term “module” refers broadly to a collection of software instructions or code configured to cause the control circuit 6015 to perform one or more functional tasks. The modules and computer-readable / executable instructions may be described as performing various operations or tasks when the control circuit 6015 or other hardware component is executing the modules or computer-readable instructions.

[00169] The instructions 6030 may be executed in logically and / or virtually separate threads on the control circuit 6015. For example, the non-transitory computer-readable medium 6020 may store instructions 6030 that when executed by the control circuit 6015 cause the control circuit 6015 to perform any of the operations, methods and / or processes described herein. In some cases, the non-transitory computer-readable medium 6020 may store computer-executable instructions or computer-readable instructions, such as instructions to perform at least a portion of the methods of FIG. 10.

[00170] In an embodiment, the computing system 6005 may store or include one or more machine-learned models 6035. For example, the machine-learned models 6035 may be or may otherwise include various machine-learned models, including machine-learned generative models (e.g., multi-class model 604, binary models 701A-B, etc.). In an embodiment, the machine-learned models 6035 may include shallow learning models. Shallow learning models may include linear models (e.g., SGDClassifier, Logistic Regression, Naive Bayes (Gaussian, Bernoli), etc.), tree-based models (e.g., decision trees and its variants, etc.), boosting / ensemble models (e.g., XGBoost, RandomForest, CatBoost, LightGBM, AdaBoost, etc.), or neural-based models (e.g., MLP (multi-layer perceptron), etc.). In an embodiment, the machine-learned models 6035 may include neural networks (e.g., deep neural networks) or other types of machine-learned models, including non-linear models and / or linear models. Neural networks may include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks or other forms of neural networks. Some example machine-learned models may leverage an attention mechanism such as self-attention. For example, some example machine-learned models may include multi-headed self-attention models (e.g., transformer models). As another example, the machine-learned models 6035 can include generative models, such as stable diffusion models, generative adversarial networks (GAN), GPT models, and other suitable models.

[00171] In an aspect of the present disclosure, the models 6035 may be used to predict a sequence of future actions of a user 120. For example, the machine-learned models 6035 can, in response to contextual features 602 and a sequential aspect (e.g., bag of apps 603), the models 6035 may be trained to predict a sequence of future actions (e.g., associated with one or more software applications 245). The models 6035 may determine based on similar conditions (e.g., context), a predicted sequence of actions the user 120 will likely take and generate output to display software applications 245 associated with the predicted sequence of actions.

[00172] In an embodiment, the one or more machine-learned models 6035 may be received from the remote computing system 7005 over networks 9050, stored in the computing system 6005 (e.g., non-transitory computer-readable medium 6020), and then used or otherwise implemented by the control circuit 6015. In an embodiment, the computing system 6005 may implement multiple parallel instances of a single model.

[00173] Additionally, or alternatively, one or more machine-learned models 6035 may be included in or otherwise stored and implemented by the remote computing system 7005 that communicates with the computing system 6005 according to a client-server relationship. For example, the machine-learned models 6035 may be implemented by the remote computing system 7005 as a portion of a web service. Thus, one or more models 6035 may be stored and / or implemented (e.g., as models 7035) at the computing system 6005 and / or one or more models 6035 may be stored and implemented at the remote computing system 7005.

[00174] The computing system 6005 may include one or more communication interfaces 6040. The communication interfaces 6040 may be used to communicate with one or more other systems. The communication interfaces 6040 may include any circuits, components, software, etc. for communicating via one or more networks (e.g., networks 9050). In some implementations, the communication interfaces 6040 may include for example, one or more of a communications controller, receiver, transceiver, transmitter, port, conductors, software and / or hardware for communicating data / information.

[00175] The computing system 6005 may also include one or more user input components 6045 that receives user input. For example, the user input component 6045 may be a touch-sensitive component (e.g., a touch-sensitive display screen or a touch pad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component may serve to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, cursor-device, joystick, or other devices by which a user may provide user input.

[00176] The computing system 6005 may include one or more output components 6050. The output components 6050 may include hardware and / or software for audibly or visually producing content. For instance, the output components 6050 may include one or more speakers, earpieces, headsets, handsets, etc. The output components 6050 may include a display device, which may include hardware for displaying a user interface and / or messages for a user. By way of example, the output component 6050 may include a display screen, CRT, LCD, plasma screen, touch screen, TV, projector, tablet, and / or other suitable display components.

[00177] The remote computing system 7005 may include one or more computing devices 7010. In an embodiment, the remote computing system 7005 may include or is otherwise implemented by one or more computing devices onboard an autonomous drone. In instances in which the remote computing system 7005 includes computing devices onboard an autonomous drone, such drone computing devices may operate according to sequential computing architectures, parallel computing architectures, or some combination thereof.

[00178] The remote computing system 7005 may include a control circuit 7015 and a non-transitory computer-readable medium 7020, also referred to herein as memory 7020. In an embodiment, the control circuit 7015 may include one or more processors (e.g., microprocessors), one or more processing cores, a programmable logic circuit (PLC) or a programmable logic / gate array (PLA / PGA), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or any other control circuit. In an embodiment, the control circuit 7015 may be programmed by one or more computer-readable or computerexecutable instructions stored on the non-transitory computer-readable medium 7020.

[00179] In an embodiment, the non-transitory computer-readable medium 7020 may be a memory device, also referred to as a data storage device, which may include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. The non-transitory computer-readable medium may form, e.g., a hard disk drive (HDD), a solid state drive (SDD) or solid state integrated memory, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), dynamic random access memory (DRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), and / or a memory stick.

[00180] The non-transitory computer-readable medium 7020 may store information that may be accessed by the control circuit 7015. For instance, the non-transitory computer-readable medium 7020 (e.g., memory devices) may store data 7025 that may be obtained, received, accessed, written, manipulated, created, and / or stored. The data 7025 may include, for instance, any of the data or information described herein. In some implementations, the remote computing system 7005 may obtain data from one or more memories that are remote from the remote computing system 7005.

[00181] The non-transitory computer-readable medium 7020 may also store computer-readable instructions 7030 that may be executed by the control circuit 7015. The instructions 7030 may be software written in any suitable programming language or may be implemented in hardware. The instructions may include computer-readable instructions, computer-executable instructions, etc. As described herein, in various embodiments, the terms “computer-readable instructions” and “computer-executable instructions” are used to describe software instructions or computer code configured to carry out various tasks and operations. In various embodiments, if the computer-readable or computer-executable instructions form modules, the term “module” refers broadly to a collection of software instructions or code configured to cause the control circuit 7015 to perform one or more functional tasks. The modules and computer-readable / executable instructions may be described as performing various operations or tasks when the control circuit 7015 or other hardware component is executing the modules or computer-readable instructions.

[00182] The instructions 7030 may be executed in logically and / or virtually separate threads on the control circuit 7015. For example, the non-transitory computer-readable medium 7020 may store instructions 7030 that when executed by the control circuit 7015 cause the control circuit 7015 to perform any of the operations, methods and / or processes described herein. In some cases, the non-transitory computer-readable medium 7020 may store computer-executable instructions or computer-readable instructions, such as instructions to perform at least a portion of the method of FIG. 10.

[00183] The remote computing system 7005 may include one or more communication interfaces 7040. The communication interfaces 7040 may be used to communicate with one or more other systems. The communication interfaces 7040 may include any circuits, components, software, etc. for communicating via one or more networks (e.g., networks 9050). In some implementations, the communication interfaces 7040 may include for example, one or more of a communications controller, receiver, transceiver, transmitter, port, conductors, software and / or hardware for communicating data / information.

[00184] The computing system 6005 and / or the remote computing system 7005 may train the models 6035, 7035 via interaction with the training computing system 8005 that is communicatively coupled over the networks 9050. The training computing system 8005 may be separate from the remote computing system 7005 or may be a portion of the remote computing system 7005.

[00185] The training computing system 8005 may include one or more computing devices 8010. In an embodiment, the training computing system 8005 may include or is otherwise implemented by one or more server computing devices. In instances in which the training computing system 8005 includes plural server computing devices, such server computing devices may operate according to sequential computing architectures, parallel computing architectures, or some combination thereof.

[00186] The training computing system 8005 may include a control circuit 8015 and a non-transitory computer-readable medium 8020, also referred to herein as memory 8020. In an embodiment, the control circuit 8015 may include one or more processors (e.g., microprocessors), one or more processing cores, a programmable logic circuit (PLC) or a programmable logic / gate array (PLA / PGA), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or any other control circuit. In an embodiment, the control circuit 8015 may be programmed by one or more computer-readable or computerexecutable instructions stored on the non-transitory computer-readable medium 8020.

[00187] In an embodiment, the non-transitory computer-readable medium 8020 may be a memory device, also referred to as a data storage device, which may include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. The non-transitory computer-readable medium may form, e.g., a hard disk drive (HDD), a solid state drive (SDD) or solid state integrated memory, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), dynamic random access memory (DRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), and / or a memory stick.

[00188] The non-transitory computer-readable medium 8020 may store information that may be accessed by the control circuit 8015. For instance, the non-transitory computer-readable medium 8020 (e.g., memory devices) may store data 8025 that may be obtained, received, accessed, written, manipulated, created, and / or stored. The data 8025 may include, for instance, any of the data or information described herein. In some implementations, the training computing system 8005 may obtain data from one or more memories that are remote from the training computing system 8005.

[00189] The non-transitory computer-readable medium 8020 may also store computer-readable instructions 8030 that may be executed by the control circuit 8015. The instructions 8030 may be software written in any suitable programming language or may be implemented in hardware. The instructions may include computer-readable instructions, computer-executable instructions, etc. As described herein, in various embodiments, the terms “computer-readable instructions” and “computer-executable instructions” are used to describe software instructions or computer code configured to carry out various tasks and operations. In various embodiments, if the computer-readable or computer-executable instructions form modules, the term “module” refers broadly to a collection of software instructions or code configured to cause the control circuit 8015 to perform one or more functional tasks. The modules and computer-readable / executable instructions may be described as performing various operations or tasks when the control circuit 8015 or other hardware component is executing the modules or computer-readable instructions.

[00190] The instructions 8030 may be executed in logically or virtually separate threads on the control circuit 8015. For example, the non-transitory computer-readable medium 8020 may store instructions 8030 that when executed by the control circuit 8015 cause the control circuit 8015 to perform any of the operations, methods and / or processes described herein. In some cases, the non-transitory computer-readable medium 8020 may store computer-executable instructions or computer-readable instructions, such as instructions to perform at least a portion of the methods of FIG. 10.

[00191] The training computing system 8005 may include a model trainer 8035 that trains the machine-learned models 6035, 7035 stored at the computing system 6005 and / or the remote computing system 7005 using various training or learning techniques. For example, the models 6035, 7035 (e.g., a machine-learned model) may be trained using a loss function that evaluates quality of generated samples over various characteristics, such as similarity to the training data.

[00192] The training computing system 8005 may modify parameters of the models 6035, 7035 (e.g., the machine-learned model) based on the loss function (e.g., generative loss function) such that the models 6035, 7035 may be effectively trained for specific applications in a supervised manner using labeled data and / or in an unsupervised manner.

[00193] In an example, the model trainer 8035 may backpropagate the loss function through the models 6035, 7035 (e.g., the machine-learned model) to modify the parameters (e.g., weights) of the models 6035, 7035. The model trainer 8035 may continue to backpropagate the loss function through the machine-learned model, with or without modification of the parameters (e.g., weights) of the model. For instance, the model trainer 8035 may perform a gradient descent technique in which parameters of the machine-learned model may be modified in the direction of a negative gradient of the loss function. Thus, in an embodiment, the model trainer 8035 may modify parameters of the machine-learned model based on the loss function.

[00194] The model trainer 8035 may utilize training techniques, such as backwards propagation of errors. For example, a loss function may be backpropagated through a model to update one or more parameters of the models (e.g., based on a gradient of the loss function). Various loss functions may be used such as mean squared error, likelihood loss, cross entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques may be used to iteratively update the parameters over a number of training iterations.

[00195] In an embodiment, performing backwards propagation of errors may include performing truncated backpropagation through time. The model trainer 8035 may perform a number of generalization techniques (e.g., weight decays, dropouts, etc.) to improve the generalization capability of a model being trained. In particular, the model trainer 8035 may train the machine-learned models 6035, 7035 based on a set of training data 8040.

[00196] The training data 8040 may include unlabeled training data for training in an unsupervised fashion. Furthermore, in some implementations, the training data 8040 can include labeled training data for training in a supervised fashion. For example, the training data 8040 can be or can include the training data such as the user feedback dataset (e.g., training sample 807) of FIG. 8.

[00197] In an embodiment, if the user has provided consent / authorization, training examples may be provided by the computing system 6005 (e.g., of the user's vehicle). Thus, in such implementations, a model 6035 provided to the computing system 6005 may be trained by the training computing system 8005 in a manner to personalize the model 6035.

[00198] The model trainer 8035 may include computer logic utilized to provide desired functionality. The model trainer 8035 may be implemented in hardware, firmware, and / or software controlling a general-purpose processor. For example, in an embodiment, the model trainer 8035 may include program files stored on a storage device, loaded into a memory and executed by one or more processors. In other implementations, the model trainer 8035 may include one or more sets of computer-executable instructions that are stored in a tangible computer-readable storage medium such as RAM, hard disk, or optical or magnetic media.

[00199] The training computing system 8005 may include one or more communication interfaces 8045. The communication interfaces 8045 may be used to communicate with one or more other systems. The communication interfaces 8045 may include any circuits, components, software, etc. for communicating via one or more networks (e.g., networks 9050). In some implementations, the communication interfaces 8045 may include for example, one or more of a communications controller, receiver, transceiver, transmitter, port, conductors, software and / or hardware for communicating data / information.

[00200] The computing system 6005, the remote computing system 7005, and / or the training computing system 8005 may also be in communication with a user device 9005 that is communicatively coupled over the networks 9050.

[00201] The user device 9005 may include various types of user devices. This may include wearable devices (e.g., glasses, watches, etc.), handheld devices, tablets, or other types of devices.

[00202] The user device 9005 may include one or more computing devices 9010. The user device 9005 may include a control circuit 9015 and a non-transitory computer-readable medium 9020, also referred to herein as memory 9020. In an embodiment, the control circuit 9015 may include one or more processors (e.g., microprocessors), one or more processing cores, a programmable logic circuit (PLC) or a programmable logic / gate array (PLA / PGA), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or any other control circuit. In an embodiment, the control circuit 9015 may be programmed by one or more computer-readable or computer-executable instructions stored on the non-transitory computer-readable medium 9020.

[00203] In an embodiment, the non-transitory computer-readable medium 9020 may be a memory device, also referred to as a data storage device, which may include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. The non-transitory computer-readable medium may form, e.g., a hard disk drive (HDD), a solid state drive (SDD) or solid state integrated memory, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), dynamic random access memory (DRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), and / or a memory stick.

[00204] The non-transitory computer-readable medium 9020 may store information that may be accessed by the control circuit 9015. For instance, the non-transitory computer-readable medium 9020 (e.g., memory devices) may store data 9025 that may be obtained, received, accessed, written, manipulated, created, and / or stored. The data 9025 may include, for instance, any of the data or information described herein. In some implementations, the user device 9005 may obtain data from one or more memories that are remote from the user device 9005.

[00205] The non-transitory computer-readable medium 9020 may also store computer-readable instructions 9030 that may be executed by the control circuit 9015. The instructions 9030 may be software written in any suitable programming language or may be implemented in hardware. The instructions may include computer-readable instructions, computer-executable instructions, etc. As described herein, in various embodiments, the terms “computer-readable instructions” and “computer-executable instructions” are used to describe software instructions or computer code configured to carry out various tasks and operations. In various embodiments, if the computer-readable or computer-executable instructions form modules, the term “module” refers broadly to a collection of software instructions or code configured to cause the control circuit 9015 to perform one or more functional tasks. The modules and computer-readable / executable instructions may be described as performing various operations or tasks when the control circuit 9015 or other hardware component is executing the modules or computer-readable instructions.

[00206] The instructions 9030 may be executed in logically or virtually separate threads on the control circuit 9015. For example, the non-transitory computer-readable medium 9020 may store instructions 9030 that when executed by the control circuit 9015 cause the control circuit 9015 to perform any of the operations, methods and / or processes described herein. In some cases, the non-transitory computer-readable medium 9020 may store computer-executable instructions or computer-readable instructions, such as instructions to perform at least a portion of the methods of FIG. 10.

[00207] The user device 9005 may include one or more communication interfaces 9035. The communication interfaces 9035 may be used to communicate with one or more other systems. The communication interfaces 9035 may include any circuits, components, software, etc. for communicating via one or more networks (e.g., networks 7050). In some implementations, the communication interfaces 9035 may include for example, one or more of a communications controller, receiver, transceiver, transmitter, port, conductors, software and / or hardware for communicating data / information.

[00208] The user device 9005 may also include one or more user input components 9040 that receives user input. For example, the user input component 9040 may be a touch-sensitive component (e.g., a touch-sensitive display screen or a touch pad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component may serve to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, cursor-device, joystick, or other devices by which a user may provide user input.

[00209] The user device 9005 may include one or more output components 9045. The output components 9045 may include hardware and / or software for audibly or visually producing content. For instance, the output components 9045 may include one or more speakers, earpieces, headsets, handsets, etc. The output components 9045 may include a display device, which may include hardware for displaying a user interface and / or messages for a user. By way of example, the output component 9045 may include a display screen, CRT, LCD, plasma screen, touch screen, TV, projector, tablet, and / or other suitable display components. As described herein, the output components 9045 may include a form factor such as lens of glasses. This can be used for an AR interface displayed via the user device 9005, while it is worn by a user.

[00210] The one or more networks 9050 may be any type of communications network, such as a local area network (e.g., intranet), wide area network (e.g., Internet), or some combination thereof and may include any number of wired or wireless links. In general, communication over a network 9050 may be carried via any type of wired and / or wireless connection, using a wide variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL). Additional Discussion of Various Embodiments

[00211] Embodiment 1 relates to a computing system of a vehicle. The computing system may include one or more processors and one or more tangible, non-transitory, computer readable media storing instructions that when executed by the one or more processors cause the computing system to perform operations. The one or more processors may cause the computing system to access sequence data indicative of a plurality of actions previously taken in sequence by a user within a vehicle. The one or more processors may cause the computing system to determine, based on the sequence data, context data associated with the plurality of actions, wherein the context data is indicative of at least a time or a location of respective actions of the plurality of actions. The one or more processors may cause the computing system to determine, using one or more machine-learned models and based on the sequence data and the context data, user preference data indicative of a ranking of computing programs associated with the vehicle, wherein the ranking is based on a probability of user interaction. The one or more processors may cause the computing system to output one or more signals to initiate a display of content indicative of the user preference data.

[00212] Embodiment 2 includes the computing system of Embodiment 1. In this embodiment, the one or more processors may cause the computing system to associate, based on the sequence data and the context data, the plurality of actions with a respective computing program of a plurality of computing programs.

[00213] Embodiment 3 includes the computing system of any of Embodiments 1 or 2. In this embodiment, the one or more processors may cause the computing system to receive ground truth data, the ground truth data indicative of user feedback associated with the ranking of the computer programs. In this embodiment, the one or more processors may cause the computing system to, based on the ground truth data, sequence data, and context data, determine using the one or more machine-learned models updated user preference data.

[00214] Embodiment 4 includes the computing system of any of Embodiments 1 or 3. In this embodiment, the user feedback is indicative of at least one of: (i) a dismiss action, (ii) a nonaction, (iii) a long way action, or (iv) an engage action.

[00215] Embodiment 5 includes the computing system of any of Embodiments 1 or 3. In this embodiment, the user feedback is indicative of at least one of: (i) positive feedback, (ii) neutral feedback, or (iii) negative feedback.

[00216] Embodiment 6 includes the computing system of any of Embodiments 1 or 3. In this embodiment, one or more weights are applied to the user feedback, the one or more weights associated with a level of interaction of the user.

[00217] Embodiment 7 includes the computing system of any of Embodiments 1 to 6. In this embodiment, the sequence is indicative of the plurality of actions taken during a respective trip.

[00218] Embodiment 8 includes the computing system of any of Embodiments 1 or 7. In this embodiment, the respective trip is based on a duration of time or distance between a point of origin and a destination.

[00219] Embodiment 9 includes the computing system of any of Embodiments 1 to 8. In this embodiment, the ranking includes at least three computing programs associated with a highest probability of user interaction.

[00220] Embodiment 10 includes the computing system of any of Embodiments 1 to 9. In this embodiment, the one or more processors may cause the computing system to determine, using the one or more machine-learned models, a binary value assigned to respective computing programs. [00221 ] Embodiment 11 relates to a computer-implemented method. In this embodiment, the method can include accessing sequence data indicative of a plurality of actions previously taken in sequence by a user within a vehicle. In this embodiment, the method can include determining, based on the sequence data, context data associated with the plurality of actions, wherein the context data is indicative of at least a time or a location of respective actions of the plurality of actions. In this embodiment, the method can include determining, using one or more machine-learned models and based on the sequence data and the context data, user preference data indicative of a ranking of computing programs associated with the vehicle, wherein the ranking is based on a probability of user interaction. In this embodiment, the method can include outputting one or more signals to initiate a display of content indicative of the user preference data.

[00222] Embodiment 12 includes the method of Embodiment 11. In this embodiment, the method can include associating, based on the sequence data and the context data, the plurality of actions with a respective computing program of a plurality of computing programs.

[00223] Embodiment 13 includes the method of Embodiment 11 to 12. In this embodiment, the method can include receiving ground truth data, the ground truth data indicative of user feedback associated with the ranking of the computer programs. In this embodiment, the method can include based on the ground truth data, sequence data, and context data, determining using the one or more machine-learned models updated user preference data.

[00224] Embodiment 14 includes the method of Embodiment 11 or 13. In this embodiment, the user feedback is indicative of at least one of: (i) a dismiss action, (ii) a non-action, (iii) a long way action, or (iv) an engage action.

[00225] Embodiment 15 includes the method of Embodiment 11 or 13. In this embodiment, the user feedback is indicative of at least one of: (i) positive feedback, (ii) neutral feedback, or (iii) negative feedback.

[00226] Embodiment 16 includes the method of Embodiment 11 or 13. In this embodiment, one or more weights are applied to the user feedback, the one or more weights associated with a level of interaction of the user.

[00227] Embodiment 17 includes the method of Embodiment 11 to 16. In this embodiment, the sequence is indicative of the plurality of actions taken during a respective trip.

[00228] Embodiment 18 includes the method of Embodiment 11 or 17. In this embodiment, the respective trip is based on a duration of time or distance between a point of origin and a destination.

[00229] Embodiment 19 includes the method of Embodiment 11 to 18. In this embodiment, the ranking comprises at least three computing programs associated with a highest probability of user interaction.

[00230] Embodiment 20 is directed to one or more non-transitory computer-readable media. The one or more non-transitory computer readable media can store instructions that are executable by a control circuit. In this Embodiment, the control circuit executing the instructions can access sequence data indicative of a plurality of actions previously taken in sequence by a user within a vehicle. In this Embodiment, the control circuit executing the instructions can determine, based on the sequence data, context data associated with the plurality of actions, wherein the context data is indicative of at least a time or a location of respective actions of the plurality of actions. In this Embodiment, the control circuit executing the instructions can determine, using one or more machine-learned models and based on the sequence data and the context data, user preference data indicative of a ranking of computing programs associated with the vehicle, wherein the ranking is based on a probability of user interaction. In this Embodiment, the control circuit executing the instructions can output one or more signals to initiate a display of content indicative of the user preference data. Additional Disclosure

[00231] As used herein, adjectives and their possessive forms are intended to be used interchangeably unless apparent otherwise from the context and / or expressly indicated. For instance, “component of a / the vehicle” may be used interchangeably with “vehicle component” where appropriate. Similarly, words, phrases, and other disclosure herein is intended to cover obvious variants and synonyms even if such variants and synonyms are not explicitly listed.

[00232] The technology discussed herein makes reference to servers, databases, software applications, and other computer-based systems, as well as actions taken, and information sent to and from such systems. The inherent flexibility of computer-based systems allows for a great variety of possible configurations, combinations, and divisions of tasks and functionality between and among components. For instance, processes discussed herein may be implemented using a single device or component or multiple devices or components working in combination. Databases and applications may be implemented on a single system or distributed across multiple systems. Distributed components may operate sequentially or in parallel.

[00233] While the present subject matter has been described in detail with respect to various specific example embodiments thereof, each example is provided by way of explanation, not limitation of the disclosure. Those skilled in the art, upon attaining an understanding of the foregoing, may readily produce alterations to, variations of, and equivalents to such embodiments. Accordingly, the subject disclosure does not preclude inclusion of such modifications, variations and / or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. For instance, features illustrated or described as part of one embodiment may be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present disclosure cover such alterations, variations, and equivalents.

[00234] Aspects of the disclosure have been described in terms of illustrative implementations thereof. Numerous other implementations, modifications, or variations within the scope and spirit of the appended claims may occur to persons of ordinary skill in the art from a review of this disclosure. Any and all features in the following claims may be combined or rearranged in any way possible. Accordingly, the scope of the present disclosure is by way of example rather than by way of limitation, and the subject disclosure does not preclude inclusion of such modifications, variations or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. Moreover, terms are described herein using lists of example elements joined by conjunctions such as “and,” “or,” “but,” etc. It should be understood that such conjunctions are provided for explanatory purposes only. The term “or” and “and / or” may be used interchangeably herein. Lists joined by a particular conjunction such as “or,” for example, may refer to “at least one of’ or “any combination of’ example elements listed therein, with “or” being understood as “and / or” unless otherwise indicated. Also, terms such as “based on” should be understood as “based at least in part on.” [00235J Those of ordinary skill in the art, using the disclosures provided herein, will understand that the elements of any of the claims, operations, or processes discussed herein may be adapted, rearranged, expanded, omitted, combined, or modified in various ways without deviating from the scope of the present disclosure. At times, elements may be listed in the specification or claims using a letter reference for exemplary illustrated purposes and is not meant to be limiting. Letter references, if used, do not imply a particular order of operations or a particular importance of the listed elements. For instance, letter identifiers such as (a), (b), (c),. . . , (i), (ii), (iii), ..., etc. may be used to illustrate operations or different elements in a list. Such identifiers are provided for the ease of the reader and do not denote a particular order, importance, or priority of steps, operations, or elements. For instance, an operation illustrated by a list identifier of (a), (i), etc. may be performed before, after, or in parallel with another operation illustrated by a list identifier of (b), (ii), etc.

Claims

I. A computing system (200) comprising:one or more processors; andone or more tangible, non-transitory, computer readable media storing instructions that when executed by the one or more processors cause the computing system to perform operations comprising:accessing sequence data indicative of a plurality of actions previously taken in sequence by a user within a vehicle (105);determining, based on the sequence data, context data associated with the plurality of actions, wherein the context data is indicative of at least a time or a location of respective actions of the plurality of actions;determining, using one or more machine-learned models and based on the sequence data and the context data, user preference data indicative of a ranking of computing programs associated with the vehicle (105), wherein the ranking is based on a probability of user interaction; andoutputting one or more signals to initiate a display of content indicative of the user preference data.

2. The computing system (100) of claim 1, wherein the operations comprise associating, based on the sequence data and the context data, the plurality of actions with a respective computing program of a plurality of computing programs.

3. The computing system (100) of claim 1, wherein the operations further comprise: receiving ground truth data, the ground truth data indicative of user feedback associated with the ranking of the computer programs; andbased on the ground truth data, sequence data, and context data, determining using the one or more machine-learned models updated user preference data.

4. The computing system (100) of claim 3, wherein the user feedback is indicative of at least one of: (i) a dismiss action, (ii) a non-action, (iii) a long way action, or (iv) an engage action.

5. The computing system (100) of claim 3, wherein the user feedback is indicative of at least one of: (i) positive feedback, (ii) neutral feedback, or (iii) negative feedback.

6. The computing system (100) of claim 3, wherein one or more weights are applied to the user feedback, the one or more weights associated with a level of interaction of the user.

7. The computing system (100) of claim 1, wherein the sequence is indicative of theplurality of actions taken during a respective trip.

8. The computing system (100) of claim 7, wherein the respective trip is based on a duration of time or distance between a point of origin and a destination.

9. The computing system (100) of claim 1, wherein the ranking comprises at least three computing programs associated with a highest probability of user interaction.

10. The computing system (100) of claim 1, wherein to the operations further comprise:determining, using the one or more machine-learned models, a binary value assigned to respective computing programs.

11. A computer-implemented method comprising:accessing sequence data indicative of a plurality of actions previously taken in sequence by a user within a vehicle (105);determining, based on the sequence data, context data associated with the plurality of actions, wherein the context data is indicative of at least a time or a location of respective actions of the plurality of actions;determining, using one or more machine-learned models and based on the sequence data and the context data, user preference data indicative of a ranking of computing programs associated with the vehicle (105), wherein the ranking is based on a probability of user interaction; andoutputting one or more signals to initiate a display of content indicative of the user preference data.

12. The computer-implemented method of claim 11, further comprising associating, based on the sequence data and the context data, the plurality of actions with a respective computing program of a plurality of computing programs.

13. The computer-implemented method of claim 11, further comprising:receiving ground truth data, the ground truth data indicative of user feedback associated with the ranking of the computer programs; andbased on the ground truth data, sequence data, and context data, determining using the one or more machine-learned models updated user preference data.

14. The computer-implemented method of claim 13, wherein the user feedback is indicative of at least one of: (i) a dismiss action, (ii) a non-action, (ii i) a long way action, or (iv) an engage action.

15. The computer-implemented method of claim 13, wherein the user feedback is indicative of at least one of: (i) positive feedback, (ii) neutral feedback, or (iii) negative feedback.

16. The computer-implemented method of claim 13, wherein one or more weights are applied to the user feedback, the one or more weights associated with a level of interaction of the user.

17. The computer-implemented method of claim 11, wherein the sequence is indicative of the plurality of actions taken during a respective trip.

18. The computer-implemented method of claim 17, wherein the respective trip is based on a duration of time or distance between a point of origin and a destination.

19. The computer-implemented method of claim 11, wherein the ranking comprises at least three computing programs associated with a highest probability of user interaction.

20. One or more processors; andone or more tangible, non-transitory, computer readable media storing instructions that when executed by the one or more processors cause a computing system to perform operations comprising:accessing sequence data indicative of a plurality of actions previously taken in sequence by a user within a vehicle (105);determining, based on the sequence data, context data associated with the plurality of actions, wherein the context data is indicative of at least a time or a location of respective actions of the plurality of actions;determining, using one or more machine-learned models and based on the sequence data and the context data, user preference data indicative of a ranking of computing programs associated with the vehicle (105), wherein the ranking is based on a probability of user interaction; andoutputting one or more signals to initiate a display of content indicative of the user preference data.

Images