Intelligent pre-emptive resource allocation to achieve high performance wearable devices

Abstract

A wearable computing device retrieves context information indicative of a current state of the wearable computing device and a computing device resource allocation profile. The computing device resource allocation profile can be indicative of a current allocation of computing device resources to tasks made based on respective priority levels assigned to each task. The wearable computing device determines a predicted state of the wearable computing device by applying a machine learning model to the context information and preemptively reallocates computing device resources to at least one task associated with the predicted state by sending, to a companion device, instructions for performing a first task included in the at least one task based on the computing device resource allocation profile and the predicted state. The wearable computing device performs a second task included in the at least one task using computing device resources preemptively reallocated to the second task.

Description

Background Technology

[0001] Wearable devices can include a variety of applications and services. Typically, due to hardware and battery constraints, wearable devices may have fewer computing resources to run such applications and services. Therefore, when multiple applications and services run concurrently, wearable devices can experience performance degradation (e.g., slower speeds and lag), as well as battery drain due to unnecessary background processes. Furthermore, due to computing resource constraints, users of wearable devices may experience latency when accessing certain features of their wearable devices. Summary of the Invention

[0002] Generally, the technology disclosed herein relates to allocating device resources based on context information and predicted actions. With the explicit consent of the user, a computing device (e.g., a wearable computing device, such as a smartwatch) can retrieve context information indicating the current state of the wearable computing device, as well as information indicating a resource allocation profile for the wearable computing device. For example, context information may indicate the current location of the wearable computing device, historical application usage data, the battery level of the wearable computing device, the user's calendar events, the time of day, the network signal strength of the wearable computing device, and the user's biometric sensor data; and the computing device resource allocation profile may indicate how the memory of the wearable computing device is currently allocated to perform various tasks. That is, the computing device resource allocation profile can be a dynamic, real-time representation of resource usage, reflecting the current state of the computing device (e.g., memory pressure, CPU load). A context engine can determine the predicted state of the wearable computing device by applying at least a machine learning model to the context information. In other words, the context engine can predict the possible state or intention of the user operating the wearable computing device (e.g., "about to pay," "start exercising," "commuting," "low battery level and away from the charger") based on a holistic fusion of the retrieved context information. The resource manager can preemptively reallocate computing device resources to at least one task associated with a predicted state based on information indicating a computing device resource allocation profile. That is, the wearable computing device can continuously assess the current context and device metrics (e.g., memory stress, battery level, thermal state, etc.) to preemptively adjust resource allocation. In some examples, preemptively reallocating computing device resources may involve reserving memory for applications associated with the predicted state, prefetching data for applications associated with the predicted state, adjusting the appropriate priority level assigned to currently executing tasks, sending instructions to the companion device to execute tasks associated with the predicted state, and suspending one or more currently executing tasks.

[0003] In some examples, to preemptively reallocate computing resources, the resource manager can offload at least a first task to the companion device, allowing the wearable computing device to utilize the companion device's resources to complete tasks associated with the predicted state. For example, the wearable computing device can send instructions to the companion device to perform a first task—such as prefetching data for an application associated with the predicted state. The wearable computing device can then receive the prefetched data from the companion device and use it to perform a second task. In this way, the wearable computing device can reallocate fewer computing resources to complete tasks associated with the predicted state. The wearable computing device can update its computing resource allocation profile to reflect the reallocation of computing resources to tasks associated with the predicted state.

[0004] In one example, this disclosure relates to a method comprising retrieving, by a wearable computing device, contextual information indicating the current state of the wearable computing device and information indicating a computing device resource allocation profile of the wearable computing device, wherein the computing device resource allocation profile indicates a current allocation of multiple computing device resources to multiple tasks based on a corresponding priority level assigned to each task. The method further comprises determining at least one predicted state of the wearable computing device by the wearable computing device applying at least a machine learning model to the contextual information. The method further comprises: preemptively reallocating one or more computing device resources from the multiple computing device resources to the at least one task by the wearable computing device, based on the information indicating the computing device resource allocation profile and the at least one predicted state, by sending an instruction from at least the wearable computing device to an accompanying device for executing a first task included in at least one task associated with the at least one predicted state. The method further comprises executing the second task by the wearable computing device using one or more computing device resources that were preemptively reallocated to the second task included in the at least one task.

[0005] In another example, this disclosure relates to a wearable computing device including one or more processors and one or more storage devices storing instructions. When executed by the one or more processors, the instructions cause the one or more processors to retrieve context information indicating the current state of the wearable computing device and information indicating a computing device resource allocation profile of the wearable computing device, wherein the computing device resource allocation profile indicates a current allocation of multiple computing device resources to multiple tasks based on a corresponding priority level assigned to each task. The instructions further cause the one or more processors to determine at least one predicted state of the wearable computing device by applying at least a machine learning model to the context information. The instructions further cause the one or more processors to preemptively reallocate one or more computing device resources from the multiple computing device resources to the at least one task based on the information indicating the computing device resource allocation profile and the at least one predicted state by sending instructions from at least the wearable computing device to an accompanying device for executing a first task included in at least one task associated with the at least one predicted state. The instructions further cause the one or more processors to execute the second task by using one or more computing device resources that have been preemptively reallocated to the second task included in the at least one task.

[0006] In another example, this disclosure relates to a non-transitory computer-readable storage medium encoded with instructions that, when executed by one or more processors, cause the one or more processors to retrieve context information indicating the current state of a wearable computing device and information indicating a computing device resource allocation profile of the wearable computing device, wherein the computing device resource allocation profile indicates a current allocation of multiple computing device resources to multiple tasks based on a corresponding priority level assigned to each task. The instructions further cause the one or more processors to determine at least one predicted state of the wearable computing device by applying at least a machine learning model to the context information. The instructions further cause the one or more processors to preemptively reallocate one or more computing device resources from the multiple computing device resources to the at least one task, based on the information indicating the computing device resource allocation profile and the at least one predicted state, by sending instructions from at least the wearable computing device to an accompanying device for executing a first task included in at least one task associated with the at least one predicted state. The instructions further cause the one or more processors to execute the second task by using one or more computing device resources that have been preemptively reallocated to the second task included in the at least one task.

[0007] Details of one or more examples of this disclosure are set forth in the accompanying drawings and the following description. Other features, objectives, and advantages will be apparent from the specification, drawings, and claims. Attached Figure Description

[0008] Figure 1 This is an example computing device configured to predict future user actions based on context information according to one or more techniques disclosed herein.

[0009] Figure 2 This is a block diagram of an example computing device configured to predict future user actions based on context information, according to one or more techniques of this disclosure.

[0010] Figure 3A This is a conceptual diagram illustrating an example training process for a machine learning module according to one or more techniques disclosed herein.

[0011] Figure 3B This is a conceptual diagram illustrating an example trained machine learning module according to one or more techniques of this disclosure.

[0012] Figure 4 This is a block diagram of an example wearable computing device that communicates with an accessory device configured to handle tasks based on predicted user actions, according to one or more technologies of this disclosure.

[0013] Figure 5 This is a flowchart illustrating an example operation of a computing device that predicts future user actions based on context information according to one or more techniques of this disclosure. Detailed Implementation

[0014] Figure 1 This describes an example computing device configured to predict future user actions based on context information, according to one or more technologies disclosed herein. Specifically, the technologies of this disclosure enable computing devices, such as wearable computing device 102, to allocate computing device resources based on context information and the user's predicted actions. Example computing devices implementing the technologies of this disclosure may include, but are not limited to, portable, mobile, or other devices, such as mobile phones (including smartphones), wearable computing devices (e.g., smartwatches, smart glasses, digital wristbands, augmented reality (AR) or virtual reality (VR) headsets and glasses, etc.), laptop computers, desktop computers, tablet computers, smart TV platforms, server computers, mainframes, infotainment systems (e.g., vehicle head units), etc. Figure 1 As shown in the example, wearable computing device 102 includes one or more user interface (UI) components 132, which include one or more sensors 104 and at least one user interface (UI) module 106. While the user interface module 106 and the resource manager module 118 are... Figure 1In the example shown, it is located within the wearable computing device 102, but in other examples, all or part of the functionality provided by the user interface module 106 and the resource manager module 118 (and / or other modules in other figures described herein) may be delegated to a cloud computing system and / or supporting devices. Example computing devices implementing the techniques of this disclosure may include... Figure 1 Components and / or modules not shown.

[0015] like Figure 1 As further shown in the example, computing device 102 includes one or more user interface components 132 (“UI components 132”). The UI components 132 of wearable computing device 102 can be configured to serve as an input and / or output device for wearable computing device 102. UI components 132 can be implemented using various technologies. For example, UI components 132 can be configured to receive input from a user via haptic, audio, and / or video feedback. Examples of input devices include presence-sensitive displays, presence-sensitive or touch-sensitive input devices, voice response systems, video cameras, microphones, or any other type of device for detecting input from a user. In some examples, presence-sensitive displays include touch-sensitive or presence-sensitive input screens, such as resistive touchscreens, surface acoustic wave touchscreens, capacitive touchscreens, projected capacitive touchscreens, pressure-sensitive screens, acoustic pulse recognition touchscreens, or other presence-sensitive technologies. That is, the UI components 132 of wearable computing device 102 can include presence-sensitive devices that can receive haptic input from a user. UI component 132 can receive instructions for haptic input by detecting one or more gestures from the user (e.g., when the user touches or points at one or more locations of UI component 132 with a finger or stylus).

[0016] UI component 132 may additionally or alternatively be configured to serve as an output device by providing output to a user through tactile, audio, or video stimuli. Examples of output devices include a sound card, a video graphics adapter card, or any of one or more display devices, such as a liquid crystal display (LCD), a dot matrix display, a light-emitting diode (LED) display, a microLED, a miniLED, an organic light-emitting diode (OLED) display, electronic ink, or a similar monochrome or color display capable of outputting visual information to a user. Additional examples of output devices include speakers, tactile devices, or other devices that can generate understandable output for a user. For example, UI component 132 may present the output to the user as a graphical user interface, which may be associated with functionality provided by wearable computing device 102. In this way, UI component 132 may present various user interfaces of applications (e.g., email messaging applications, internet browser applications, etc.) that are executed at or accessible by wearable computing device 102. Users can interact with the corresponding user interface of an application to cause wearable computing device 102 to perform function-related operations.

[0017] In some examples, the UI component 132 of the wearable computing device 102 can detect two-dimensional and / or three-dimensional gestures as input from the user. For example... Figure 1 As shown in the example, UI component 132 includes one or more sensors 104. Sensors 104 can be configured to detect one or more physiological characteristics of a user with the user's explicit consent. For example, sensor 104 can detect movement of the user within a threshold distance of sensor 104 (e.g., gait or movement of the hand, arm, face, eyes, etc.). Sensor 104 can determine a two-dimensional or three-dimensional vector representation of the movement and associate that vector representation with a gesture input having multiple dimensions (e.g., waving, facial expression, etc.). In other words, in some examples, sensor 104 can detect multidimensional gestures without the user making a gesture on or near the screen or surface for which UI component 132 outputs information for display. Instead, sensor 104 can detect multidimensional gestures performed at or near sensor 104, which may or may not be located near the screen or surface for which UI component 132 outputs information for display. In some examples, sensor 104 is configured to detect one or more physiological characteristics of a user, including but not limited to baseline heart rate, skin color, blood pressure reading, skin temperature, electrocardiogram reading, voice, etc.

[0018] In some examples, sensor 104 is a photoplethysmography (PPG) sensor, which measures changes in blood volume in a user's skin using a light sensor. PPG sensors are commonly used in smartwatches to monitor heart rate and can also be used to detect unique blood flow patterns in a user's wrist. In another example, sensor 104 is an electrocardiogram (ECG) sensor. An ECG sensor detects electrical activity in the heart of a user of the wearable computing device 102. In other examples, sensor 104 can be any other biometric sensor used to collect biometric data from a user, such as a facial recognition or fingerprint sensor. In some examples, sensor 104 can include motion sensors (e.g., accelerometers, gyroscopes, compasses, etc.), audio and / or visual sensors (e.g., microphones, still and / or video cameras, etc.) or other types of sensors (e.g., pressure sensors, light sensors, proximity sensors, ultrasonic sensors, GPS sensors, etc.).

[0019] exist Figure 1 In the example, the wearable computing device 102 further includes a user interface (UI) module 106. The UI module 106 may perform the operations described herein using hardware, software, firmware, or a combination thereof residing in and / or executing on the wearable computing device 102. The wearable computing device 102 may utilize one or more processors to execute module 106. In some examples, the wearable computing device 102 may execute the UI module 106 as a virtual machine executing on the underlying hardware. The UI module 106 may execute as one or more services of an operating system or computing platform, or as one or more executable programs at the application layer of the computing platform.

[0020] like Figure 1 The UI module 106 shown in the example can be operated by the wearable computing device 102 to perform one or more functions, such as receiving input and sending indications of such input to other components associated with the wearable computing device 102. The UI module 106 can also receive data from components associated with the wearable computing device 102. Using the received data, the UI module 106 can cause other components associated with the wearable computing device 102 (such as UI component 132) to provide output based on that data.

[0021] The techniques described herein may relate to the collection and analysis of user information. Generally, providing users with the opportunity to provide input to control whether the procedures or features of the wearable computing device 102 can collect and utilize user information (e.g., user biometric data, information about the user's current location, current speed, movement, location history, etc.) or specifying whether and / or how the wearable computing device 102 can receive user-related content. That is, such information, including but not limited to location history, application usage patterns, calendar events, and biometric information, is only collected after receiving explicit and affirmative user consent. Furthermore, specific data may be processed in one or more ways before being stored or used by the wearable computing device 102, such that personally identifiable information is removed. For example, user identity may be processed to make it impossible to determine personally identifiable information about the user, or the user's geographic location may be generalized (e.g., to a city, zip code, or state level) if location information is available, making it impossible to determine the user's specific location. Therefore, users control how the wearable computing device 102 collects and uses information about them, for example, by providing users with clear, accessible controls to manage their data, including the ability to view, revoke, and delete their data at any time. All collected data is handled in accordance with strict privacy protocols, and where possible, data is anonymized, aggregated, or processed on the device to protect user privacy.

[0022] According to the technology disclosed herein, wearable computing device 102 may include a resource manager module 118 that can preemptively allocate computing device resources based on context information and predicted user actions. With the user's explicit consent, wearable computing device 102 can retrieve context information indicating the current state of wearable computing device 102 and information indicating a computing device resource allocation profile for wearable computing device 102. For example, context information may indicate the current location of wearable computing device 102, historical application usage data, battery level of wearable computing device 102, user's calendar events, daytime, network signal strength of wearable computing device 102, and user's biometric sensor data. Generally, the computing device resource allocation profile may indicate how the memory of wearable computing device 102 is currently allocated to perform various tasks.

[0023] Generally, the context engine included in the wearable computing device 102 can determine the predicted state of the wearable computing device 102 by applying at least a machine learning model to context information. That is, the context engine can predict the possible state or intention of a user operating the wearable computing device 102 (e.g., "about to pay," "start exercising," "commuting," "low battery level and away from charger") based on a holistic fusion of retrieved context information. Generally, the resource manager module 118 can preemptively reallocate computing device resources to at least one task associated with the predicted state based on information indicating a computing device resource allocation profile. Generally, the computing device resource allocation profile can be a dynamic, real-time representation of resource usage, rather than a static configuration. That is, the computing device resource allocation profile can reflect the current state of the wearable computing device 102 (e.g., memory pressure, CPU load). In some examples, the resource manager module 118 can continuously evaluate the computing device resource allocation profile, allowing the wearable computing device 102 to proactively and / or preemptively adjust resource allocation. The resource manager module 118 can continuously assess the current context and device metrics of the wearable computing device 102 (e.g., memory pressure, battery level, thermal state, etc.) to preemptively adjust resource allocation. In some examples, to preemptively reallocate computing device resources, the resource manager module 118 can cause the wearable computing device 102 to reserve memory for applications associated with a predicted state, prefetch data for applications associated with a predicted state, adjust the appropriate priority level assigned to currently executing tasks, send instructions to companion devices to execute tasks associated with a predicted state, and / or suspend one or more currently executing tasks. Generally, while the wearable computing device 102 can be described as being configured to "preemptively" adjust resource allocation, "preemptively" can be interchanged with "actively".

[0024] In this way, the techniques described herein can improve the lifespan and functionality of user devices. For example, as devices age, they may experience: performance degradation due to concurrent apps and services; battery depletion due to unnecessary background processes; limited functionality and slow response when calling resource-intensive features; and rapid hardware obsolescence due to updated applications exceeding the device's CPU and memory budget. Conventional power schedulers and / or task schedulers may only react after the user launches an app, so critical apps (such as a wallet at a subway gate) may still suffer from cold start latency, while low-priority tasks continue to consume power. Therefore, by leveraging contextual information and predicted user actions, the techniques described herein can provide a solution for proactively allocating device resources, which helps improve device operation and user experience.

[0025] Figure 2 This is a block diagram further illustrating an example wearable computing device configured to predict future user actions based on context information, according to one or more technologies of this disclosure. Wearable computing device 202 can be used with... Figure 1 The wearable computing device 102 is similar. Figure 2 As shown, the wearable computing device 202 includes a user interface component 232, which includes one or more sensors 204, a processor 224, a communication unit 228, and one or more storage devices 238. The user interface component 232 and the one or more sensors 204 may be similar to... Figure 1 The user interface component 132 and one or more sensors 104. The storage device 238 further includes a UI module 206 (this module can be connected to...). Figure 1 The UI module 106, if not fundamentally similar, is similar; the operating system 222 (“OS 222”); the scene manager module 212; and the resource manager module 218 (this module can be used with…). Figure 1 The resource manager module 118 (if not fundamentally similar, then similar), one or more applications 214, and machine learning module 210.

[0026] For example, one or more communication units 228 of the wearable computing device 202 can communicate with external devices by transmitting and / or receiving data at the wearable computing device 202 (such as sending and / or receiving data to or from a remote computer system or companion device). Example communication units 228 include network interface cards (e.g., Ethernet cards), optical transceivers, radio frequency transceivers, or any other type of device capable of sending and / or receiving information. Other examples of communication units 228 may be devices configured to transmit and receive Ultrawideband®, Bluetooth®, GPS, 3G, 4G, and Wi-Fi®, etc., that can be found in computing devices (such as mobile devices).

[0027] like Figure 2 As shown in the example, communication channel 230 can interconnect each of the components shown to enable inter-component communication (physical, communicative, and / or operational communication). In some examples, communication channel 230 may include a system bus, a network connection (e.g., to a wireless connection as described above), one or more inter-process communication data structures, or any other component for transferring data locally or remotely between hardware and / or software.

[0028] UI module 206, machine learning module 210, scene manager module 212, application lifecycle manager 213, one or more applications 214, and resource manager module 218 (hereinafter referred to as "modules 206 to 218") can perform the operations described herein using software, hardware, firmware, or a mixture of hardware, software, and firmware residing in and executing on wearable computing device 202 or one or more other remote computing devices (e.g., cloud-based applications—not shown) or associated devices. Wearable computing device 202 can execute one or more modules of modules 206 to 218 using one or more processors 224, or can execute any or part of one or more modules of modules 206 to 218 as a virtual machine executing on or within the underlying hardware. Each module of modules 206 to 218 can be implemented in various ways, such as being implemented as a downloadable or pre-installed application, remotely implemented as a cloud application, or implemented as part of the operating system of wearable computing device 202. Other examples of wearable computing devices 202 that implement the technology disclosed herein may include Figure 2 Additional components not shown.

[0029] exist Figure 2 In the example, one or more processors 224 may implement functionality and / or execute instructions within the wearable computing device 202. For example, one or more processors 224 may receive and execute instructions that provide functionality to UIC 232, communication unit 228, one or more storage devices 238, and OS 222 to perform one or more operations as described herein. For example, one or more processors 224 may receive and execute instructions that provide functionality to some or all of modules 206 to 218 to perform one or more operations or various functions described herein. One or more processors 224 include a central processing unit (CPU). Examples of CPUs include, but are not limited to: digital signal processors (DSPs), general-purpose microprocessors, tensor processing units (TPUs); neural processing units (NPUs); neural processing engines; the core of a CPU, VPU, GPU, TPU, NPU, or other processing device; application-specific integrated circuits (ASICs); field-programmable arrays (FPGAs); or other equivalent integrated or discrete logic circuit systems.

[0030] One or more storage devices 238 within the wearable computing device 202 may store information for processing during operation of the wearable computing device 202 (e.g., the wearable computing device 202 may store data accessible during execution by modules 206 to 218 at the wearable computing device 202). In some examples, the storage device 238 is temporary memory, meaning that its primary purpose is not long-term storage. The storage device 238 on the wearable computing device 202 may be configured to temporarily store information as volatile memory and therefore not retain the stored contents in the event of a power outage. Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and other forms of volatile memory known in the art.

[0031] In some examples, storage device 238 also includes one or more computer-readable storage media. Storage device 238 can be configured to store a larger amount of information than volatile memory. Storage device 238 can be further configured as non-volatile memory space for long-term storage of information and to retain information after power-on / power-off cycles. Examples of non-volatile memory include magnetic hard disks, optical disks, floppy disks, flash memory, or electrically programmable memory (EPROM) or electrically erasable programmable memory (EEPROM). Storage device 238 can store program instructions and / or information (e.g., data) associated with modules 206 to 218.

[0032] In some examples, the operating system 222 controls the operation of components of the wearable computing device 202. For example, in one example, the operating system 222 facilitates communication between modules 204 to 218 and processor 224, one or more UI components 232 (including one or more sensors 204), one or more communication units 228, and one or more communication channels 230. Modules 206 to 218 may each include program instructions and / or data executable by the wearable computing device 202 (e.g., one or more processors 224). For example, each module in 206 to 218 may include instructions that cause the wearable computing device 202 to perform one or more of the operations and actions described in this disclosure.

[0033] UI module 206 and UI component 232 can receive one or more input instructions from the user. UI module 206 and UI component 232 can interpret the input detected at UI component 232 (e.g., when the user provides one or more gestures at one or more locations of UI component 232 displaying the graphical user interface), and can forward information about the input detected at UI component 232 to one or more associated platforms, operating systems, applications and / or services executing at wearable computing device 202 to enable wearable computing device 202 to perform various functions.

[0034] UI module 206 can receive information and instructions from one or more associated platforms, operating systems, applications, and / or services executing at wearable computing device 202 to generate a graphical user interface. Additionally, UI module 206 can act as an intermediary between one or more associated platforms, operating systems, applications, and / or services executing at wearable computing device 202 and various output devices of wearable computing device 202 (e.g., speakers, LED indicators, audio or electrostatic haptic output devices, etc.) to utilize wearable computing device 202 to generate outputs (e.g., graphics, a beam of light, sound, haptic response, etc.).

[0035] OS 222 and application 214 may perform the operations described herein using software, hardware, firmware, or a mixture of hardware, software, and firmware residing in and / or executing on the wearable computing device 202. The wearable computing device 202 may utilize multiple processors or multiple devices to execute OS 222 and application 214 as a virtual machine executing on the underlying hardware, as one or more services of an operating system or computing platform, and / or as one or more executable programs at the application layer of the computing platform of the wearable computing device 202.

[0036] In some examples, OS 222 can provide an execution environment for one or more modules such as modules 206 to 218. OS 222 can represent a multi-threaded operating system or a single-threaded operating system, with which modules 206 to 218 can interface to access the hardware of the wearable computing device 202. OS 222 may include a kernel that facilitates access to the underlying hardware of the wearable computing device 202, wherein the kernel may present multiple different interfaces (e.g., application programmer interfaces (APIs)) that modules 206 to 218 can call to access the underlying hardware of the wearable computing device 202.

[0037] According to the technology described herein, in order to effectively and efficiently manage the allocation of computing device resources, wearable computing device 202 may include a context manager module 212, a resource manager module 218, and an application lifecycle manager module 213, which can work together to provide an intelligent preemptive resource allocation system for dynamically managing applications and services running on wearable computing device 202.

[0038] Generally, the context manager module 212 can aggregate and fuse contextual information, such as various signal arrays from the wearable device 202 (e.g., location, biometric data, thermal state, battery level, application data such as calendar events). In some examples, one or more signals may be received from the companion device. Generally, the context manager module 212 can retrieve contextual information via one or more application programming interfaces (APIs). Generally, the context manager module 212 can synthesize and analyze contextual information to predict the user's future intentions with high fidelity (e.g., by applying the machine learning module 210 to the contextual information). That is, the context manager module 212 can provide contextual information as input to the machine learning module 210, whereby the machine learning module 210 can predict user needs, such as predicted future states.

[0039] More specifically, the context manager module 212 can receive and process context information, including one or more of the following: location data (e.g., GPS, Wi-Fi, and / or cellular data), historical app usage data, battery level, system time data, on-device calendar data, biometric data (e.g., heart rate variability, activity level from accelerometer, sleep patterns), real-time internal resource metrics (e.g., memory pressure, CPU utilization, thermal state), and information related to the companion device (which can be obtained via low-latency inter-device protocols (e.g., Bluetooth LE, custom socket APIs) and may include companion device connectivity information (Wi-Fi, cellular type / strength), companion device battery information, companion device calendar events, historical companion device app usage, and data prefetched by the companion device's resource manager, etc.). This information can be provided as input data to the machine learning module 210. Therefore, the machine learning module 210 can determine the user's proximity to relevant services (e.g., payment terminals, transfer stations), the user's historical app usage and typical usage time, and the user's daily routines and schedules (e.g., sleep, work, exercise), etc. In other words, the context manager module 212 (which may be a “context engine”) can employ the machine learning module 210 to predict the user’s possible state or intention (e.g., “about to pay”, “start exercising”, “commuting”, “low battery and far from the charger”) based on the overall fusion of signals (location, time, history, calendar, sensor data such as heart rate trends, phone connection / status, battery level) received and processed by the context manager module 212.

[0040] Therefore, generally, the context manager module 212 can retrieve context information indicating the current state of the wearable computing device 202 and can employ the machine learning module 210 to determine at least one predicted state of the wearable computing device 202 by applying at least a machine learning model to the context information. In some examples, the context manager module 212 can intelligently combine rule-based predictions with predictions generated by the machine learning module 210. Thus, in some examples, the context manager module 212 can employ a hybrid model to determine the predicted state. More specifically, the context manager module 212 can use a rule-based system or a rule-based inference model to make deterministic predictions, such as making predictions in well-defined scenarios, such as predicting a "commuting" state based on a specific time of day and location near a transfer station. The context manager module 212 can additionally or alternatively use one or more trained machine learning models included in the machine learning module 210 to make more advanced context-aware predictions, such as for more complex or ambiguous scenarios.

[0041] In some examples, if rule-based predictions produce high-confidence matches, they may take precedence over machine learning-based predictions (e.g., the output of a rule-based model may be assigned a higher priority score than the output of machine learning module 210). In other examples, the highest-probability prediction output by machine learning module 210 may be used, provided that the prediction meets a predefined confidence threshold. In still other examples, rule-based inference may be combined with machine learning models, which, for example, can produce more nuanced and robust predictions for high-confidence scenarios.

[0042] In some examples, machine learning module 210 may determine at least one potential state of wearable computing device 202 based on contextual information and determine whether a confidence score associated with at least one potential state of wearable computing device 202 meets a confidence score threshold. In these examples, in response to determining that the confidence score meets the confidence score threshold, machine learning module 210 may determine that at least one potential state of wearable computing device 202 is at least one predicted state of wearable computing device 202.

[0043] Wearable computing device 202 may receive and / or store context information indicating a computing resource allocation profile for wearable computing device 202, wherein the computing resource allocation profile may indicate the current allocation of multiple computing resources to multiple tasks based on the corresponding priority level assigned to each task. That is, generally, resource manager module 218 may proactively reallocate computing resources using the computing resource allocation profile of wearable computing device 202 and the predicted state. In some examples, reallocating computing resources may involve adjusting CPU quotas, reserving contiguous blocks of random access memory (RAM), setting CPU affinity, and adjusting sensor polling rates. For example, resource manager module 218 may reserve specific contiguous blocks of RAM for applications that the context manager module 212 has predicted the user will use, thereby preventing other processes from causing memory fragmentation or consuming memory.

[0044] As another example, if a predicted state of wearable computing device 202 indicates that wearable computing device 202 will enter a location with low signal strength, resource manager module 218 can prioritize basic services and applications, and therefore can preemptively reserve and / or reallocate computing device resources for basic services and applications. That is, generally, resource manager module 218 can preemptively reallocate one or more computing device resources to at least one task associated with at least one predicted state, based on information indicating a computing device resource allocation profile and at least one predicted state.

[0045] Generally, to preemptively reallocate computing resources, resource manager module 218 sends instructions to application lifecycle manager 213, causing application lifecycle manager 213 to execute one or more processes. For example, one or more processes may include one or more of the following: reserving memory for at least one task associated with at least one predicted state; reserving memory for an application in application 214 associated with at least one predicted state; prefetching data for an application associated with at least one predicted state; adjusting the appropriate priority level for each of a plurality of tasks assigned; adjusting the polling frequency of sensors; and suspending one or more of a plurality of tasks. In some examples, resource manager module 218 may preemptively reallocate one or more computing resources to at least one task associated with at least one predicted state by outputting instructions that cause wearable computing device 102 to execute a first task included in the at least one task. In some other examples, resource manager module 218 may preemptively reallocate one or more computing resources to at least one task associated with at least one predicted state by sending instructions to an accompanying device for executing a first task included in the at least one task.

[0046] In other words, in some examples, wearable computing device 202 can determine that a companion device is near wearable computing device 202. Wearable computing device 202 can receive information indicating the state of the companion device and determine, based on the information indicating the state of the companion device, whether the companion device has sufficient available computing resources. In response to determining that the companion device has sufficient available computing resources, resource manager module 218 can send instructions to the companion device to perform a first subtask. That is, resource manager module 218 can "hand over" or offload one or more subtasks included in the task to the companion device, so that wearable computing device 202 can complete the task using fewer computing resources. In some examples, the first subtask may involve prefetching data for an application associated with at least one predicted state. In these examples, wearable computing device 202 can receive prefetched data from the companion device for an application associated with at least one predicted state. The application lifecycle manager module 213 can then use the prefetched data to perform one or more additional subtasks included in the task to complete the task. In some examples, the resource manager module 218 can update the computing device resource allocation profile based on computing device resources that have been preemptively reallocated to subtasks performed by the wearable computing device 202.

[0047] Therefore, generally, the context manager module 212 can retrieve and fuse various context information, and the machine learning module 210 can analyze this information to determine at least one predicted state of the wearable computing device 202. The resource manager module 218 can use context information (e.g., computing device resource allocation profile) and at least one predicted state to output a set of instructions for preemptively reallocating one or more computing device resources to tasks associated with at least one predicted state. Generally, the resource manager module 218 can continuously evaluate the current context and real-time device metrics (e.g., memory pressure, battery level, thermal state) of the wearable computing device 202 and / or its companion devices to adjust instructions. In some examples, the resource manager module 218 can leverage the machine learning module 210 to determine optimal reallocation of computing device resources, task priorities, etc.

[0048] In some examples, the resource manager module 218 sends instructions to the companion device that cause the companion device to execute one or more processes. That is, the companion device can use its own computing resources to execute one or more subtasks included in a task. The application lifecycle manager module 213 can also receive a set of instructions, wherein the application lifecycle manager module 213 can execute one or more processes, such as reserving memory for at least one task associated with at least one predicted state; reserving memory for an application from application 214 associated with at least one predicted state; prefetching data for an application associated with at least one predicted state; adjusting the appropriate priority level of each task assigned from multiple tasks; adjusting the polling frequency of sensors; and suspending one or more tasks from multiple tasks. In other words, generally, the resource manager module 212 can determine a set of actions to be performed for a task associated with at least one predicted state and can generate instructions for performing said actions. The application lifecycle manager module 213 can receive and execute instructions, thereby causing the wearable computing device 202 to perform actions. For example, the application lifecycle manager module 213 can orchestrate a specific set of low-level actions, such as loading, unloading, and / or preparing the target application from application 214 for a "warm start," thereby eliminating cold start latency. In this way, the wearable computing device 202 can preemptively reallocate its local on-device resources, allowing tasks to be executed more efficiently.

[0049] Figure 3A This is a conceptual diagram illustrating an example training process for a machine learning module according to one or more techniques of this disclosure. In some examples, Figure 1The wearable computing device 102 can store and implement the machine learning module 310 locally (i.e., on the device). Therefore, in some examples, the machine learning module 310 may be stored on an embedded device or user computing device (such as a mobile device) and / or implemented locally by that embedded device or user computing device. Output data obtained through the local implementation of the machine learning module 310 at the embedded device or user computing device can be used to improve the performance of the embedded device or user computing device (e.g., an application implemented by the embedded device or user computing device). The machine learning module 310 described herein can be trained at a training computing system and then provided for use on one or more computing devices (such as... Figure 1 The training process 340 is stored and / or implemented at the wearable computing device 102. Figure 1 The training process 340 is performed locally at the wearable computing device 102. However, in some examples, the training process 340 may be included in or separate from any computing system that implements the machine learning module 310.

[0050] Generally, the machine learning module 310 may be or include one or more inference models, i.e., one or more trained machine learning models that can be used to make predictions based on new, unseen data. The machine learning module 310 can "infer" a conclusion or output, which can be a prediction, classification, recommendation, or other type of decision-making. The machine learning module 310 can be trained according to one or more of a variety of different training types or techniques. For example, in some examples, the machine learning module 310 can be trained by... Figure 3A The training process is carried out in 340.

[0051] like Figure 3A As further shown in the examples, in some examples, the machine learning module 310 can be trained on training data 331, which may include input data 333 with labels 337. Figure 3A The training process shown is an example training process; other training processes can also be used. Generally, during training process 340, machine learning module 310 can learn patterns from training data 331, and training process 340 can optimize the parameters of machine learning module 310 to minimize prediction error.

[0052] After the user grants permission to use such data for training, training data 331 may include anonymized usage logs of shared streams, such as content items that have been shared together, bundled content fragments that have been identified as belonging together (e.g., entities from a knowledge graph), etc. In some examples, training data 331 may include examples of input data 333 that have been assigned labels 337 corresponding to output data 335.

[0053] In some examples, the machine learning module 310 can be trained by optimizing an objective function (such as objective function 339). For example, in some examples, objective function 339 can be or include a loss function that compares (e.g., determines the difference between) the output data generated by the model from the training data and the labels associated with the training data (e.g., the true labels). For example, the loss function can evaluate the sum or mean of the squared differences between the output data 335 and the labels. In some examples, objective function 339 can be or include a cost function that describes the cost of a certain outcome or output data. Other examples of objective function 339 may include margin-based techniques, such as, for example, triplet loss or maximum margin training.

[0054] One or more of various optimization techniques can be performed to optimize the objective function 339. For example, optimization techniques can minimize or maximize the objective function 339. Example optimization techniques include Hessian-based techniques and gradient-based techniques, such as coordinate descent; gradient descent (e.g., stochastic gradient descent); subgradient methods, etc. Other optimization techniques include black-box optimization techniques and heuristics.

[0055] In some examples, backpropagation can be combined with optimization techniques (e.g., gradient-based techniques) to train machine learning module 310 (e.g., when the machine learning model is a multi-layer model such as an artificial neural network). For example, iterative loops of propagation and model parameter (e.g., weight) updates can be performed to train machine learning module 310. Example backpropagation techniques include truncated time-through backpropagation, Levenberg-Marquardt backpropagation, etc.

[0056] In some examples, unsupervised learning techniques can be used to train the machine learning module 310 described herein. Unsupervised learning can include inferring functions from unlabeled data to describe hidden structures. For example, the data may not include classification or categorization. Unsupervised learning techniques can be used to generate machine learning models capable of performing clustering, anomaly detection, learning latent variable models, or other tasks.

[0057] The machine learning module 310 can be trained using semi-supervised techniques that combine aspects of supervised and unsupervised learning. The machine learning module 310 can be trained using evolutionary techniques or genetic algorithms, or generated in other ways. In some examples, reinforcement learning can be used to train the machine learning module 310 described herein. In reinforcement learning, an agent (e.g., a model) takes actions in the environment and learns to maximize rewards and / or minimize penalties resulting from such actions. Reinforcement learning differs from supervised learning problems in that it does not present correct input / output pairs or explicitly correct suboptimal actions.

[0058] In some examples, one or more generalization techniques may be performed during training to improve the generalization of the machine learning module 310. Generalization techniques can help reduce the overfitting of the machine learning module 310 to the training data. Example generalization techniques include dropout; weight decay; batch normalization; early stopping; subset selection; stepwise selection, etc.

[0059] In some examples, the machine learning module 310 described herein may include or be influenced by multiple hyperparameters (such as, for example, learning rate, number of layers, number of nodes per layer, number of leaves in the tree, number of clusters, etc.). Hyperparameters can affect model performance. Hyperparameters can be selected manually or automatically by applying techniques such as, for example, grid search, black-box optimization techniques (e.g., Bayesian optimization, random search, etc.), gradient-based optimization, etc. Example techniques and / or tools for performing automatic hyperparameter optimization include Hyperopt; Auto-WEKA; Spearmint; Metric Optimization Engine (MOE), etc.

[0060] In some examples, various techniques can be used to optimize and / or adapt the learning rate while training the model. Example techniques and / or tools for performing learning rate optimization or adaptation include Adagrad; Adaptive Moment Estimation (ADAM); Adadelta; RMSprop, etc.

[0061] In some examples, transfer learning techniques can be used to provide an initial model from which the machine learning module 310 described herein is trained. In some examples, transfer learning involves reusing a model and its parameters acquired while solving a problem and applying it to different but related problems. Models trained on very large datasets can be retrained or fine-tuned on additional data. Typically, all model design and their parameters are copied from the source model, except for the output layer. The output layer is often referred to as the head, while the other layers are often referred to as the foundation. The source parameters can be considered to include knowledge learned from the source dataset, and this knowledge can also be applied to the target dataset. Fine-tuning may include updating the head parameters, while the body parameters are fixed or updated in later steps.

[0062] In some examples, the machine learning module 310 can be trained offline or online. In offline training (also known as batch learning), the machine learning module 310 is trained on the entire static training dataset. In online learning, the machine learning module 310 is continuously trained (or retrained) as new training data becomes available (e.g., when the model is used to perform inference).

[0063] In some examples, the training process 340 may involve centralized training of the machine learning module 310 (e.g., based on a centrally stored dataset). In other implementations, distributed training techniques such as distributed training, federated learning, etc., can be used to train, update, or personalize the machine learning module 310.

[0064] The machine learning module 310 described herein can be trained according to one or more of various training types or techniques. For example, in some examples, the machine learning module 310 can be trained using supervised learning via a training process 340, wherein the machine learning module 310 is trained on a training dataset that includes instances or examples with labels. Labels can be applied manually by experts, generated through crowdsourcing, or provided through other techniques (e.g., through physical-based or complex mathematical models). In some examples, training examples can be provided by the user's computing device if the user has provided consent. In some examples, this process may be referred to as personalizing the model.

[0065] In some examples, the machine learning module 310 includes a language model that can be trained (e.g., pre-trained, fine-tuned, etc.) through a training process 340. For example, the training process 340 can pre-train the language model on a large and diverse corpus of text. Thus, in some examples, the training data 331 can include a dataset covering a wide range of topics and domains to ensure that the machine learning module 310 learns diverse language patterns and contextual relationships. The training process 340 can train the language model to optimize an objective function 339. The objective function 339 can be or include a loss function, such as cross-entropy loss, which compares the output data generated by the model from the training data 331 with the labels 337 (e.g., ground truth labels) associated with the training data 331 (e.g., determining the differences between them). For example, the objective function 339 of the language model could be to correctly predict the next word in a word sequence or to fill in missing words as correctly as possible.

[0066] In some examples, training procedure 340 can use techniques such as Low-Rank Adaptive (LoRA) to train or fine-tune the Language Model (LLM) implemented by machine learning module 310. Generally, LoRA reduces the number of trainable parameters by freezing the pre-trained weights of the LLM and injecting a small, trainable low-rank matrix to adapt the model to a specific task. LoRA can be useful when the model needs to adapt to multiple tasks using limited task-specific data. That is, training procedure 340 can use LoRA for task-specific fine-tuning. In some examples, training procedure 340 can use techniques such as Retrieval Augmented Generation (RAG), a hybrid framework combining information retrieval and text generation. RAG can be used to fine-tune the generative model implemented by machine learning module 310 by retrieving relevant information from external databases or datasets (e.g., large and diverse text corpora) and using that information to generate more accurate and informative outputs. RAG can help generate more factually accurate and context-relevant summaries and question responses.

[0067] In some examples, the training process 340 can continuously or periodically train the language model included in the machine learning module 310. In some examples, the training process 340 can fine-tune the language model using feedback during the training process. For example, Figure 2 The UI component 232 can receive user input via a computing device, selecting feedback (e.g., approval, disapproval, etc.) related to the generated application functionality and the associated GUI presented to the user on the computing device. In some examples, the feedback can indicate whether the generated application functionality and the associated GUI are accurate or inaccurate, correct or incorrect, high-quality or low-quality, etc. The UI module 206 can receive this feedback and can send it to the machine learning module 310 (specifically to the training process 340), where the training process 340 uses the feedback for training. For example, the training process 340 can convert the feedback into labeled data for supervised training. Alternatively or additionally, the training process 340 can fine-tune the language model by analyzing the relationship between the performance of the language model and user feedback, and iterate the fine-tuning process as needed (e.g., to receive more positive user feedback and less negative user feedback). In this way, the techniques of this disclosure can establish a feedback loop that continuously improves the quality of the language model's output data 335 (e.g., instruction files).

[0068] Figure 3B This is a conceptual diagram illustrating example trained machine learning modules according to one or more techniques of this disclosure. In some examples, Figure 1The wearable computing device 102 can store and implement the machine learning module 310 locally (i.e., on the device). Therefore, in some examples, the machine learning module 310 may be stored on an embedded device or user computing device (such as a mobile device) and / or implemented locally by that embedded device or user computing device. Output data obtained through the local implementation of the machine learning module 310 at the embedded device or user computing device can be used to improve the performance of the embedded device or user computing device (e.g., an application implemented by the embedded device or user computing device). Figure 3B The machine learning module 310 can be used in computing systems (such as...) Figure 1 The wearable computing device 102 is trained at a remote computing system (communicationd with a remote computing system) and then provided for use in Figure 1 The wearable computing device 102 stores and / or implements [the technology]. In some examples, the machine learning module 310 is [located in...]. Figure 1 The wearable computing device 102 performs the task locally. In some examples, the wearable computing device 102 can perform machine learning as a service.

[0069] like Figure 3B As shown, in some examples, the machine learning module 310 (e.g., via...) Figure 3A The training process 340) is trained to receive input data 333, which can be of one or more types, and in response, provides output data 335, which can be of one or more types. Therefore, Figure 3B A machine learning module 310 performing inference is shown, wherein the machine learning module 310 can use learned patterns to make predictions or decisions on new data (e.g., input data 333). The machine learning module 310 may include... Figure 3A The training process involves training one or more machine learning models.

[0070] Input data 333 may include one or more features associated with an instance or example. In some examples, the one or more features associated with an instance or example may be organized into a feature vector. In some examples, output data 335 may include one or more predictions. Prediction may also be referred to as inference. Therefore, given features associated with a particular instance, machine learning module 310 may output a prediction for such instances based on those features.

[0071] Machine learning module 310 can be or includes one or more machine learning models from various different types of machine learning models. Specifically, in some examples, machine learning module 310 can perform NLP tasks. Machine learning module 310 can summarize, translate, or organize input data 333. Machine learning module 310 can use recurrent neural networks (RNNs) and / or transformer models (self-attention models). Example models may include, but are not limited to, GPT-3, BERT, Gemini (e.g., Gemini Ultra, Gemini Pro, Gemini Flash, Gemini Nano), Android AICore, and T5. In some examples, machine learning module 310 can perform classification, summarization, name generation, regression, clustering, anomaly detection, recommendation generation, and / or other tasks.

[0072] In some examples, the machine learning module 310 can perform various types of classification based on the input data 333. For example, the machine learning module 310 can perform binary classification or multi-class classification. In binary classification, the output data 335 can include classifying the input data 333 into one of two distinct classes. In multi-class classification, the output data 335 can include classifying the input data 333 into one (or more) of more than two classes. The classification can be single-label or multi-label. The machine learning module 310 can perform discrete class classification, where the input data 333 is simply classified into one or more classes or categories.

[0073] In some examples, the machine learning module 310 can perform classification, where it provides a numerical value for each of one or more classes, describing the degree to which the input data 333 is believed to be classified into the corresponding class. In some instances, the numerical value provided by the machine learning module 310 may be referred to as a "confidence score," which indicates the corresponding confidence level associated with classifying the input into the appropriate class. In some examples, the confidence score may be compared to one or more thresholds to render a discrete class prediction. In some examples, only a certain number of classes (e.g., one class) with the highest relative confidence score may be selected to render the discrete class prediction.

[0074] Machine learning module 310 can output probabilistic classifications. For example, given a sample input, machine learning module 310 can predict a probability distribution over a set of classes. Therefore, machine learning module 310 can output the probability that the sample input belongs to each class, rather than simply outputting the most probable class to which the sample input should belong. In some examples, the sum of the probability distributions over all possible classes can be one. In some examples, a Softmax function or other types of functions or layers can be used to compress the set of real values ​​associated with each possible class into a set of real values ​​in the range (0, 1) that sum to 1.

[0075] In some examples, the probabilities provided by the probability distribution can be compared with one or more thresholds to render discrete class predictions. In other examples, only a certain number of classes (e.g., one class) with the highest relative predicted probability can be selected to render discrete class predictions.

[0076] When the machine learning module 310 performs classification, supervised learning techniques can be used to train the machine learning module 310. For example, the machine learning module 310 can be trained on a training dataset that includes training examples labeled as belonging to (or not belonging to) one or more classes.

[0077] In some examples, the machine learning module 310 can perform regression to provide output data in the form of continuous numerical values. Continuous numerical values ​​can correspond to any number of different measures or numerical representations, including, for example, monetary values, fractions, or other numerical representations. As an example, the machine learning module 310 can perform linear regression, multinomial regression, or nonlinear regression. As an example, the machine learning module 310 can perform simple regression or multiple regression. As described above, in some examples, the Softmax function or other functions or layers can be used to compress a set of real values, each associated with two or more possible classes, into a set of real values ​​in the range (0, 1) that sum to 1.

[0078] Machine learning module 310 can perform various types of clustering. For example, machine learning module 310 can identify one or more predefined clusters that most likely correspond to input data 333. Machine learning module 310 can identify one or more clusters within input data 333. That is, where input data 333 includes instances of multiple objects, documents, or other entities, machine learning module 310 can classify the multiple entities included in input data 333 into multiple clusters. In some examples where machine learning module 310 performs clustering, unsupervised learning techniques can be used to train machine learning module 310.

[0079] The machine learning module 310 can perform anomaly detection or outlier detection. For example, the machine learning module 310 can identify input data that does not conform to expected patterns or other characteristics (e.g., as previously observed from previous input data). As an example, anomaly detection can be used for fraud detection or system failure detection.

[0080] In some examples, the machine learning module 310 may provide output data in the form of one or more recommendations. For example, the machine learning module 310 may be included in a recommendation system or engine. As an example, given input data describing previous outcomes for certain entities (e.g., scores, rankings, or ratings indicating the amount of success or enjoyment), the machine learning module 310 may output suggestions or recommendations for one or more additional entities, which, based on previous outcomes, are expected to have the desired outcome (e.g., deriving scores, rankings, or ratings indicating success or enjoyment). As an example, given input data describing a computing device (such as...) Figure 1 Given input data from a wearable computing device 102, the recommendation system can output suggestions or recommendations for applications that the user may like or wish to download to the wearable computing device 102.

[0081] In some cases, the machine learning module 310 can act as an agent within the environment. For example, the machine learning module 310 can be trained using reinforcement learning, which will be discussed in further detail below.

[0082] In some examples, the machine learning module 310 can be a parametric model, while in other implementations it can be a non-parametric model. In some examples, the machine learning module 310 can be a linear model, while in other implementations it can be a non-linear model.

[0083] As described above, machine learning module 310 can be or includes one or more machine learning models of various different types. Examples of such different types of machine learning models are provided below for illustration. One or more of the example models described below can be used (e.g., combined) to provide output data 335 in response to input data 333. Additional models may be used in addition to the example models provided below.

[0084] In some examples, the machine learning module 310 may be or include one or more classifier models, such as, for example, a linear classification model; a quadratic classification model, etc. The machine learning module 310 may be or include one or more regression models, such as, for example, a simple linear regression model; a multiple linear regression model; a logistic regression model; a stepwise regression model; a multiple adaptive regression spline; a locally estimated scatter plot smoothing model, etc.

[0085] In some examples, the machine learning module 310 may be or include one or more decision tree-based models, such as, for example, classification and / or regression trees; iterative binary decision trees; C4.5 decision trees; chi-square automatic interactive detection decision trees; decision stumps; conditional decision trees, etc.

[0086] Machine learning module 310 may be or include one or more kernel machines. In some examples, machine learning module 310 may be or include one or more support vector machines. Machine learning module 310 may be or include one or more instance-based learning models, such as, for example, learning vector quantization models; self-organizing map models; locally weighted learning models, etc. In some examples, machine learning module 310 may be or include one or more nearest neighbor models, such as, for example, k-nearest neighbor classification models; k-nearest neighbor regression models, etc. Machine learning module 310 may be or include one or more Bayesian models, such as, for example, Naive Bayes models; Gaussian Naive Bayes models; multinomial Naive Bayes models; averaged one-dependence estimator; Bayesian networks; Bayesian belief networks; hidden Markov models, etc.

[0087] In some examples, the machine learning module 310 may be or include one or more artificial neural networks (also simply referred to as neural networks). A neural network may include a set of connected nodes, which may also be referred to as neurons or perceptrons. A neural network may be organized into one or more layers. A neural network comprising multiple layers may be referred to as a "deep" network. A deep network may include an input layer, an output layer, and one or more hidden layers located between the input and output layers. The nodes of a neural network may be connected or not fully connected.

[0088] The machine learning module 310 may be or include one or more feedforward neural networks. In a feedforward network, connections between nodes do not form loops. For example, each connection may connect a node from the previous layer to a node from the next layer.

[0089] In some instances, the machine learning module 310 may be or include one or more recurrent neural networks. In some instances, at least some nodes of the recurrent neural network may form a loop. Recurrent neural networks can be particularly useful for processing inherently ordered input data. Specifically, in some instances, the recurrent neural network can pass information from the first part of the input data sequence 333 to the subsequent part of the input data sequence 333 by using loops or directed recurrent node connections.

[0090] In some examples, ordered input data can include time-series data (e.g., sensor data relative to time or images captured at different times). For example, recurrent neural networks can analyze sensor data with respect to time to detect or predict swipe direction, perform handwriting recognition, etc. Ordered input data can include words in a sentence (e.g., for natural language processing, speech detection or processing, etc.); notes in a musical piece; ordered actions taken by a user (e.g., to detect or predict ordered application usage); ordered object states, etc.

[0091] Example recurrent neural networks include Long Short-Term (LSTM) recurrent neural networks; gated recurrent units; bidirectional recurrent neural networks; continuous-time recurrent neural networks; neural history compressors; echo-state networks; Elman networks; Jordan networks; recurrent neural networks; Hopfield networks; fully recurrent networks; sequence-to-sequence configurations, etc.

[0092] In some examples, the machine learning module 310 may be or include one or more convolutional neural networks. In some instances, the convolutional neural network may include one or more convolutional layers that perform convolutions on the input data using learned filters.

[0093] Filters can also be referred to as kernels. Convolutional neural networks (CNNs) can be particularly useful for vision problems, such as when the input data includes images like still images or videos. However, CNNs can also be applied to natural language processing.

[0094] In some examples, the machine learning module 310 may be or include one or more generative networks, such as, for example, generative adversarial networks. Generative networks can be used to generate new data, such as new images or other content.

[0095] Machine learning module 310 may be or includes an autoencoder. In some cases, the purpose of an autoencoder is to learn a representation of a set of data (e.g., a low-dimensional encoding), often for the purpose of dimensionality reduction. For example, in some instances, an autoencoder may seek to encode input data 333 and then provide output data reconstructing the input data 333 from the encoding. Recently, the concept of autoencoders has been increasingly used in generative models for learning data. In some instances, an autoencoder may include an additional loss in addition to reconstructing the input data 333.

[0096] Machine learning module 310 may be or include one or more other forms of artificial neural networks, such as, for example, deep Boltzmann machines; deep belief networks; stacked autoencoders, etc. Any of the neural networks described herein may be combined (e.g., stacked) to form more complex networks.

[0097] One or more neural networks can be used to provide embeddings based on input data 333. For example, an embedding can be a knowledge representation abstracted from input data 333 to one or more learned dimensions. In some instances, an embedding can be a useful source for identifying relevant entities. In some instances, an embedding can be extracted from the network's output, while in others, it can be extracted from any hidden node or layer of the network (e.g., near but not the final layer). Embeddings can be used to perform automatic suggestions such as next video recommendations, product suggestions, entity or object recognition, etc. In some instances, an embedding can be a useful input for downstream models. For example, an embedding can be used to generalize input data (e.g., search queries) to downstream models or processing systems.

[0098] The machine learning module 310 may include one or more clustering models, such as, for example, k-means clustering model; k-median clustering model; expectation-maximization model; hierarchical clustering model, etc.

[0099] In some examples, the machine learning module 310 may perform one or more dimensionality reduction techniques, such as, for example, principal component analysis; kernel principal component analysis; graph-based kernel principal component analysis; principal component regression; partial least squares regression; Sammon mapping; multidimensional scaling; projection pursuit; linear discriminant analysis; mixture discriminant analysis; quadratic discriminant analysis; generalized discriminant analysis; flexible discriminant analysis; autoencoding, etc.

[0100] In some examples, the machine learning module 310 may perform or be subjected to one or more reinforcement learning techniques, such as Markov decision processes; dynamic programming; Q-functions or Q-learning; value function methods; deep Q-networks; differentiable neural computers; asynchronous advantage actor-critics; deterministic policy gradients, etc.

[0101] In some examples, the machine learning module 310 can be an autoregressive model. In some instances, the autoregressive model can specify that the output data 335 depends linearly on its own previous values ​​and a random term. In some instances, the autoregressive model can take the form of a stochastic difference equation. An example autoregressive model is WaveNet, which is a generative model used for raw audio.

[0102] In some examples, the machine learning module 310 may include or form part of a multi-model ensemble. As an example, bootstrap aggregating, also known as "bagging," can be performed. In bootstrap aggregating, the training dataset is divided into multiple subsets (e.g., through random sampling with replacement), and multiple models are trained on each subset separately. During inference, the corresponding outputs of the multiple models can be combined (e.g., through averaging, voting, or other techniques) and used as the output of the ensemble.

[0103] One example ensemble is the random forest, also known as a random decision forest. Random forests are ensemble learning methods used for classification, regression, and other tasks. Random forests are generated by producing multiple decision trees during training. In some instances, at inference time, the class predicted as the mode of multiple classes (classification) or the mean of individual trees (regression) can be used as the output of the forest. Random decision forests can correct the tendency for decision trees to overfit their training set.

[0104] Another example of an ensemble technique is stacking, which in some instances may be referred to as stacked generalization. Stacking involves training a combiner model to mix or otherwise combine predictions from several other machine learning models. Thus, multiple machine learning models (e.g., of the same or different types) can be trained based on training data. Additionally, the combiner model can be trained to take predictions from other machine learning models as input and, in response, produce a final inference or prediction. In some instances, a single-layer logistic regression model can be used as the combiner model.

[0105] Another example of an ensemble technique is boosting. Boosting can involve progressively building an ensemble by iteratively training weak models and then adding them to a final strong model. For example, in some instances, each new model can be trained to emphasize training examples that were misclassified (e.g., mis-classified) by the previous model. For example, the weights associated with each of these misclassified examples can be increased. A common implementation of boosting is AdaBoost, which is also known as adaptive boosting. Other example boosting techniques include LPBoost; TotalBoost; BrownBoost; xgboost; MadaBoost; LogitBoost; gradient boosting, etc. Furthermore, any of the models mentioned above (e.g., regression models and artificial neural networks) can be combined to form an ensemble. As an example, an ensemble can include a top-level machine learning model or heuristic function to combine and / or weight the outputs of the models forming the ensemble.

[0106] In some examples, multiple machine learning models (e.g., forming an ensemble model) may be linked and jointly trained (e.g., through model ensemble by backpropagating errors sequentially). However, in other examples, only a subset of the jointly trained models (e.g., one of the models) is used for inference.

[0107] In some examples, the machine learning module 310 can be used to preprocess the input data 333 for subsequent input into another model. For example, the machine learning module 310 can perform dimensionality reduction techniques and embeddings (e.g., matrix factorization, principal component analysis, singular value decomposition, word2vec / GLOVE and / or related methods); clustering; and even classification and regression for downstream consumption.

[0108] As discussed above, the machine learning module 310 can be trained or otherwise configured to receive input data 333 and, in response, provide output data 335. Input data 333 can include different types, forms, or variations of input data. As an example, in various implementations, input data 333 can include features describing the content (or a portion of the content) initially selected by the user, such as the content of the document or image selected by the user, links pointing to the user's selection, links within the user's selection relating to other files available on the device or in the cloud, metadata selected by the user, etc. Additionally, with user permission, input data 333 includes the user's usage context obtained from the app itself or from other sources. Examples of usage contexts include breadth of sharing (public sharing, sharing with a large group, sharing privately, or sharing with specific individuals), shared contexts, etc. When the user grants permission, additional input data can include the state of the device, such as the device's location, the app running on the device, etc.

[0109] In some examples, the machine learning module 310 may receive and use the input data 333 in its raw form. In some examples, the raw input data may be preprocessed. Therefore, as a supplement to or alternative to the raw input data, the machine learning module 310 may receive and use preprocessed input data.

[0110] In some examples, preprocessing the input data 333 may include extracting one or more additional features from the original input data. For example, feature extraction techniques may be applied to the input data 333 to generate one or more new additional features. Example feature extraction techniques include edge detection; corner detection; blob detection; ridge detection; scale-invariant feature transform; motion detection; optical flow; Hough transform, etc.

[0111] In some examples, the extracted features may include or be derived from transformations of the input data 333 to other domains and / or dimensions. As an example, the extracted features may include or be derived from transformations of the input data 333 to the frequency domain. For instance, wavelet transforms and / or fast Fourier transforms may be performed on the input data 333 to generate additional features.

[0112] In some examples, the extracted features may include statistics calculated from input data 333 or certain portions or dimensions of input data 333. Example statistics include the mode, mean, maximum, minimum, or other measures of input data 333 or a portion thereof.

[0113] In some examples, as mentioned above, the input data 333 can be inherently ordered. In some instances, ordered input data can be generated by sampling or otherwise segmenting the input data stream. As an example, frames can be extracted from video. In some examples, ordered data can be transformed into non-ordered data through summarization.

[0114] As another example of preprocessing techniques, portions of the input data 333 can be filled in. For example, additional synthetic input data can be generated through interpolation and / or extrapolation.

[0115] As another example of preprocessing techniques, some or all of the input data 333 can be scaled, standardized, normalized, generalized, and / or regularized. Example regularization techniques include ridge regression; Least Absolute Shrinkage and Selection Operator (LASSO); Elastic Networks; Minimum Angular Regression; Cross-validation; L1 regularization; L2 regularization, etc. As an example, some or all of the input data 333 can be normalized by subtracting the mean of the feature values ​​across a given dimension from each individual feature value and then dividing by the standard deviation or other metric.

[0116] As another example of preprocessing techniques, some or all of the input data 333 can be quantized or discretized. In some cases, qualitative features or variables included in the input data 333 can be converted into quantitative features or variables. For example, one-hot encoding can be performed.

[0117] In some examples, dimensionality reduction techniques can be applied to the input data 333 before it is fed into the machine learning module 310. Several examples of dimensionality reduction techniques are provided above, including, for example, principal component analysis; kernel principal component analysis; graph-based kernel principal component analysis; principal component regression; partial least squares regression; Sammon mapping; multidimensional scaling; projection pursuit; linear discriminant analysis; mixture discriminant analysis; quadratic discriminant analysis; generalized discriminant analysis; flexible discriminant analysis; autoencoding, etc.

[0118] In some examples, during training, the input data can be intentionally deformed in any number of ways to increase model robustness, generalization, or other qualities. Example techniques for deforming the input data include adding noise; changing color, shading, or hue; scaling; segmentation; augmentation, etc.

[0119] In response to receiving input data 333, the machine learning module 310 can provide output data 335. Output data 335 may include different types, forms, or variations of output data. As an example, in various implementations, output data 335 may include content stored locally on the user device or in the cloud, which may be shared in relation to the initial content selection.

[0120] As discussed above, in some examples, output data 335 may include various types of categorical data (e.g., binary classification, multi-class classification, single-label, multi-label, discrete classification, regression classification, probabilistic classification, etc.) or various types of regression data (e.g., linear regression, multinomial regression, nonlinear regression, simple regression, multiple regression, etc.). In other instances, output data 335 may include clustered data, anomaly detection data, recommendation data, or any of the other forms of output data discussed above.

[0121] In some examples, output data 335 can influence downstream processes or decision-making. As an example, in some cases, output data 335 can be interpreted and / or acted upon by rule-based regulators.

[0122] Any of the different types or forms of input data described herein can be combined with any of the different types or forms of machine learning models described herein to provide any of the different types or forms of output data described herein.

[0123] The systems and methods of this disclosure may be implemented by one or more computing devices or otherwise performed on one or more computing devices. Example computing devices include user computing devices (e.g., laptops, desktop computers, and mobile computing devices such as tablets, smartphones, wearable computing devices, etc.); embedded computing devices (e.g., devices embedded in vehicles, cameras, image sensors, industrial machines, satellites, game consoles or controllers, or home appliances such as refrigerators, thermostats, energy meters, home energy managers, smart home assistants, etc.); server computing devices (e.g., database servers, parameter servers, file servers, mail servers, print servers, web servers, game servers, application servers, etc.); dedicated, specialized model processing or training devices; virtual computing devices; other computing devices or computing infrastructure; or combinations thereof. The computing device or system implementing machine learning module 310 or other aspects of this disclosure may include multiple hardware components that implement the techniques described herein.

[0124] In some instances, the output data 335 obtained by the machine learning module 310 at a computing system or device can be used to improve tasks on other devices, or can be used by other non-user devices to improve services performed by or for such other non-user devices. For example, the output data 335 can improve other downstream processes performed by a server device on a user's computing device or embedded computing device. In other instances, the output data 335 obtained by implementing the machine learning module 310 at a computing system or device can be sent to and used by a user's computing device, embedded computing device, or some other client device. In some examples, Figure 2 The computing device 202 can be used as a service to perform machine learning.

[0125] In other implementations, different corresponding parts of the machine learning module 310 may be stored in and / or implemented in a combination of user computing devices, embedded computing devices, server computing devices, etc. In other words, the parts of the machine learning module 310 may be wholly or partially distributed on the client device (e.g., Figure 1 Wearable computing device 102) and computing system (e.g., with Figure 1 In the wearable computing device 102, a remote computing system for communication.

[0126] Such as Figure 1 The wearable computing device 102 can use one or more machine learning platforms, frameworks and / or libraries (such as TensorFlow, Caffe / Caffe2, Theano, Torch / PyTorch, MXnet, CNTK, etc.) to perform graphics processing techniques or other machine learning techniques.

[0127] In some examples, multiple instances of the machine learning module 310 can be parallelized to provide increased processing throughput. For example, multiple instances of the machine learning module 310 can be parallelized on a single processing or computing device, or parallelized across multiple processing or computing devices.

[0128] A computing device implementing machine learning module 310 or other aspects of this disclosure may include multiple hardware components for implementing the techniques described herein. For example, the computing device may include one or more memory devices storing some or all of the machine learning module 310. For example, machine learning module 310 may be a structured digital representation stored in memory. The one or more memory devices may also include instructions for implementing machine learning module 310 or performing other operations. Example memory devices include RAM, ROM, EEPROM, EPROM, flash memory devices, disks, and combinations thereof.

[0129] The computing device may also include one or more processing devices that implement some or all of the machine learning module 310 and / or perform other related operations. Example processing devices include one or more of the following: a central processing unit (CPU); a vision processing unit (VPU); a graphics processing unit (GPU); a tensor processing unit (TPU); a neural processing unit (NPU); a neural processing engine; the core of a CPU, VPU, GPU, TPU, NPU, or other processing device; an application-specific integrated circuit (ASIC); a field-programmable gate array (FPGA); a coprocessor; a controller; or a combination of the above processing devices. The processing device may be embedded within other hardware components, such as, for example, an image sensor, an accelerometer, etc.

[0130] Hardware components (e.g., memory devices and / or processing devices) may be distributed across multiple physically distributed computing devices and / or virtually distributed computing systems.

[0131] In some examples, the machine learning module 310 described herein may be included in different parts of computer-readable code on a computing device. In one example, the machine learning module 310 may be included in and used by a particular application or program (e.g., specifically for use). Thus, in one example, the computing device may include multiple applications, and one or more of these applications may contain their own corresponding machine learning libraries and machine learning models.

[0132] In another example, the machine learning module 310 described herein may be included in the operating system of a computing device (e.g., in the central intelligence layer of the operating system) and may be invoked or otherwise used by one or more applications that interact with the operating system. In some examples, each application may communicate with the central intelligence layer (and the models stored therein) using an application programming interface (API) (e.g., a common public API across all applications).

[0133] In some examples, the central intelligence layer can communicate with a central device data layer. The central device data layer can be a centralized data repository for the computing device. The central device data layer can communicate with many other components of the computing device, such as one or more sensors, a field manager, a device state component, and / or additional components. In some examples, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0134] The technologies discussed in this paper relate to servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information sent to and from such systems. The inherent flexibility of computer-based systems allows for a wide variety of possible configurations, combinations, and divisions of tasks and functionalities among and within components. For example, the processes discussed in this paper can be implemented using a single device or component, or multiple devices or components working in combination.

[0135] Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0136] Furthermore, the machine learning techniques described herein are readily interchangeable and combinable. While some example techniques have been described, many other techniques exist and can be used in conjunction with various aspects of this disclosure.

[0137] In addition to the description above, users are provided with controls that allow them to choose whether and when the system, program, or feature described herein can collect user information (e.g., information about the user's social networks, social actions or activities, occupation, user preferences, or the user's current location) and whether to send content or communications to the user from the server. Furthermore, some data may be processed in one or more ways before being stored or used, resulting in the removal of personally identifiable information. For example, a user's identity may be processed to the point that their personally identifiable information cannot be determined, or the user's geographic location may be generalized (e.g., to a city, zip code, or state level) if location information is available, making it impossible to determine the user's specific location. Therefore, users have control over what information is collected, how that information is used, and what information is provided to them.

[0138] Figure 4 This is a block diagram of an example wearable computing device that communicates with an accessory device configured to handle tasks based on predicted user actions, according to one or more technologies of this disclosure.

[0139] The computing device 402 can be respectively connected with Figure 1 and Figure 2 Wearable computing devices 102 and 202 are, if not similar, substantially similar. Computing device 402 can determine that a companion device 462 from a plurality of companion devices is close to or adjacent to computing device 402. In some examples, “close” can be defined as the range (e.g., 9 kHz to 300 GHz) spanned by wireless communication (e.g., wireless networks including Bluetooth, 3G, LTE, and Wi-Fi). For example, if computing device 402 and companion device 462 communicate via Bluetooth, computing device 402 can determine that companion device 462 is close to computing device 402 when companion device 462 is within approximately 10 meters of computing device 402. In another example, if computing device 402 and companion device 462 communicate via Wi-Fi, computing device 402 can determine that companion device 462 is close to computing device 402 when companion device 462 is within approximately 100 meters of computing device 402. In other words, when the distance between the accessory device 462 and the computing device 402 supports communication between the accessory device 462 and the computing device 402, the computing device 402 can determine that the accessory device 462 is close to the computing device 402.

[0140] The telemetry module 460 of computing device 402 and the telemetry module 464 of auxiliary device 462 can communicate with each other or other external devices via one or more networks such as one or more wireless networks described above. In some examples, such as in Figure 4 In the example, the accessory device 462 uses the telemetry module 464 to communicate wirelessly with the telemetry module 460 of the computing device 402.

[0141] In some examples, in order to preemptively reallocate computing device resources, resource manager module 418 can offload at least a first task to companion device 462, so that wearable computing device 402 can utilize the resources of companion device 462 to complete tasks associated with a predicted state.

[0142] For example, wearable computing device 402 can send instructions to companion device 462 to perform a first task—such as prefetching data for an application associated with a predicted state. That is, generally, resource manager module 470 can receive instructions and reallocate one or more computing resources of companion device 462 to perform the first task. Continuing this example, wearable computing device 402 can receive the prefetched data from companion device 462, and application lifecycle manager module 413 (with...) Figure 2 The application lifecycle manager module 213 (if not substantially similar, then similar) can use the prefetched data to perform a second task. In this way, the wearable computing device 402 can reallocate fewer computing resources to complete the task associated with the predicted state. In some examples, the resource manager module 418 can update the computing resource allocation profile to reflect the reallocation of computing resources to tasks associated with the predicted state.

[0143] The techniques described herein offer various advantages for wearable computing devices. For example, by offloading tasks (e.g., data prefetching, complex computation, predictive generation, etc.) to a companion device 462 with more robust processing power, memory, and battery life, resource-constrained wearable computing devices 402 can conserve their own resources for the final user-facing tasks. This distributed architecture not only improves the performance of wearable devices (e.g., the techniques described herein can help reduce performance issues such as application lag and general unresponsiveness on wearable devices), but also provides more complex predictive capabilities than might be possible on a wearable device alone. In other words, the techniques described herein can provide a sophisticated distributed system for device resource management, ultimately delivering a smoother and more responsive user experience on hardware-constrained devices.

[0144] Figure 5 This is a flowchart illustrating an example operation of a computing device that predicts future user actions based on contextual information according to one or more techniques of this disclosure. (See also: [link to related information]) Figures 1 to 4 To describe Figure 5 .

[0145] Wearable computing device 402 can retrieve context information indicating the current state of wearable computing device 402 and information indicating a computing device resource allocation profile of wearable computing device 402 (590). In some examples, the computing device resource allocation profile indicates the current allocation of multiple computing device resources to multiple tasks based on the corresponding priority level assigned to each task. Wearable computing device 402 determines at least one predicted state of wearable computing device 402 by applying at least the machine learning module 310 to the context information (592). Based on the information indicating the computing device resource allocation profile and at least one predicted state, wearable computing device 402 preemptively reallocates one or more computing device resources from multiple computing device resources to the at least one task by sending an instruction to at least the companion device 462 for executing a first task included in at least one task associated with the at least one predicted state (594). Wearable computing device 402 executes the second task by using one or more computing device resources that have been preemptively reallocated to the second task included in the at least one task (596).

[0146] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on or transmitted over a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to: a tangible medium, such as a data storage medium; or a communication medium, including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0147] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other storage medium that can be used to store desired program code in the form of instructions or data structures and that is accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transient tangible storage media. As used herein, disks and optical discs include compact discs (CDs), laser discs, optical discs, digital universal discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0148] Instructions can be executed by one or more processors such as digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuit systems. Therefore, as used herein, the term "processor" can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules. Moreover, the techniques may be fully implemented in one or more circuit or logic elements.

[0149] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless handheld devices, integrated circuits (ICs), or IC sets (e.g., chip sets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but these various components, modules, or units are not necessarily required to be implemented by different hardware units. Rather, as described above, various units may be combined within hardware units or provided by a collection of interoperable hardware units including one or more processors as described above, combined with suitable software and / or firmware.

[0150] It should be recognized that, depending on the example, certain actions or events of any of the techniques described herein may be performed in a different order, and may be added, combined, or omitted entirely (e.g., not all described actions or events are necessary for the practice of the technique). Furthermore, in some examples, actions or events may be performed concurrently, for example, through multithreading, interrupt handling, or multiple processors, rather than sequentially.

[0151] In some examples, computer-readable storage media include non-transitory media. The term "non-transitory" indicates that the storage medium is not embodied in a carrier wave or propagating signal. In some examples, non-transitory storage media may store data that can change over time (e.g., in RAM or cache).

[0152] Example: Example 1: A method includes: retrieving by a wearable computing device context information indicating the current state of the wearable computing device and information indicating a computing device resource allocation profile of the wearable computing device, wherein the computing device resource allocation profile indicates a current allocation of multiple computing device resources to multiple tasks based on a corresponding priority level assigned to each task; determining by the wearable computing device at least one predicted state of the wearable computing device by applying at least a machine learning model to the context information; preemptively reallocating one or more computing device resources from the multiple computing device resources to the at least one task by the wearable computing device, based on the information indicating the computing device resource allocation profile and the at least one predicted state, by sending an instruction from at least the wearable computing device to an accessory device for executing a first task included in at least one task associated with the at least one predicted state; and executing the second task by the wearable computing device using one or more computing device resources preemptively reallocated to the second task included in the at least one task.

[0153] Example 2: The method as described in Example 1, wherein determining at least one predicted state of the wearable computing device by applying at least a machine learning model to context information further comprises: determining at least one potential state of the wearable computing device based on the context information by applying at least the machine learning model to the context information; determining by the wearable computing device whether a confidence score associated with the at least one potential state of the wearable computing device satisfies a confidence score threshold; and in response to determining that the confidence score satisfies the confidence score threshold, determining at least one predicted state of the wearable computing device, wherein the at least one predicted state of the wearable computing device is the at least one potential state of the wearable computing device.

[0154] Example 3: The method as described in any one of Examples 1 and 2, wherein the context information of the wearable computing device includes one or more of the following: location data, historical application usage data, battery level, calendar data, time of day, network signal strength, and biometric sensor data.

[0155] Example 4: The method as described in any one of Examples 1 to 3, wherein preemptively reallocating one or more computing device resources further includes executing one or more processes.

[0156] Example 5: The method as described in Example 4, wherein one or more processes include one or more of the following: reserving memory for at least one task associated with at least one predicted state; reserving memory for an application associated with at least one predicted state; prefetching data for an application associated with at least one predicted state; adjusting the appropriate priority level for each of the multiple tasks assigned; adjusting the polling frequency of the sensors; and suspending one or more tasks from the multiple tasks.

[0157] Example 6: The method of any one of Examples 1 to 5, wherein, before sending the instruction for performing the first task, the method further includes: determining, by the wearable computing device, that an accessory device is near the wearable computing device; receiving, by the wearable computing device, information indicating the state of the accessory device; determining, by the wearable computing device, based on the information indicating the state of the accessory device, whether the accessory device has a sufficient amount of available computing device resources; and, in response to determining that the accessory device has the sufficient amount of available computing device resources, sending, by the wearable computing device, the instruction for performing the first task to the accessory device.

[0158] Example 7: The method of any one of Examples 1 to 6, wherein the first task is to prefetch data for an application associated with at least one predicted state, and wherein the method further comprises: receiving the prefetched data of the application associated with at least one predicted state from an accessory device by a wearable computing device; and performing a second task by the wearable computing device using the prefetched data.

[0159] Example 8: The method of any one of Examples 1 to 7, wherein the method further comprises: updating the computing device resource allocation profile by the wearable computing device based on one or more computing device resources that have been preemptively reallocated to the second task.

[0160] Example 9: A wearable computing device includes: one or more processors; and one or more storage devices storing instructions that, when executed by the one or more processors, cause the one or more processors to: retrieve context information indicating the current state of the wearable computing device and information indicating a computing device resource allocation profile of the wearable computing device, wherein the computing device resource allocation profile indicates a current allocation of multiple computing device resources to multiple tasks based on a corresponding priority level assigned to each task; determine at least one predicted state of the wearable computing device by applying at least a machine learning model to the context information; preemptively reallocate one or more computing device resources from the multiple computing device resources to the at least one task by sending instructions from at least the wearable computing device to an accessory device for executing a first task included in at least one task associated with the at least one predicted state, based on the information indicating the computing device resource allocation profile and the at least one predicted state; and execute the second task by using one or more computing device resources that were preemptively reallocated to the second task included in the at least one task.

[0161] Example 10: A wearable computing device as described in Example 9, wherein, in order to determine at least one predicted state of the wearable computing device by applying at least a machine learning model to context information, the instruction further causes one or more processors to: determine at least one potential state of the wearable computing device based on the context information by applying at least a machine learning model to the context information; determine whether a confidence score associated with at least one potential state of the wearable computing device satisfies a confidence score threshold; and, in response to determining that the confidence score satisfies the confidence score threshold, determine at least one predicted state of the wearable computing device, wherein the at least one predicted state of the wearable computing device is the at least one potential state of the wearable computing device.

[0162] Example 11: A wearable computing device as described in any one of Examples 9 and 10, wherein the context information of the wearable computing device includes one or more of the following: location data, historical application usage data, battery level, calendar data, time of day, network signal strength, and biometric sensor data.

[0163] Example 12: A wearable computing device as described in any one of Examples 9 to 11, wherein, in order to preemptively reallocate one or more computing device resources, the instruction further causes one or more processors to execute one or more processes.

[0164] Example 13: A wearable computing device as described in Example 12, wherein one or more processes include one or more of the following: reserving memory for at least one task associated with at least one predicted state; reserving memory for an application associated with at least one predicted state; prefetching data for an application associated with at least one predicted state; adjusting the appropriate priority level of each of the plurality of tasks assigned; adjusting the polling frequency of the sensors; and suspending one or more tasks from the plurality of tasks.

[0165] Example 14: A wearable computing device as described in any one of Examples 9 to 13, wherein, before sending instructions for performing a first task, the instructions further cause one or more processors to: determine that an accessory device is near the wearable computing device; receive information indicating the state of the accessory device; determine, based on the information indicating the state of the accessory device, whether the accessory device has a sufficient amount of available computing device resources; and, in response to determining that the accessory device has the sufficient amount of available computing device resources, send instructions to the accessory device for performing the first task.

[0166] Example 15: A wearable computing device as described in any one of Examples 9 to 14, wherein a first task is to prefetch data for an application associated with at least one predicted state, and wherein the instructions further cause one or more processors to: receive the prefetched data from an accessory device for an application associated with at least one predicted state; and perform a second task using the prefetched data.

[0167] Example 16: A wearable computing device as described in any one of Examples 9 to 15, wherein the instructions further cause one or more processors to update a computing device resource allocation profile based on one or more computing device resources that have been preemptively reallocated to a second task.

[0168] Example 17: A non-transitory computer-readable storage medium encoded with instructions that, when executed by one or more processors, cause the one or more processors to: retrieve context information indicating the current state of a wearable computing device and information indicating a computing device resource allocation profile of the wearable computing device, wherein the computing device resource allocation profile indicates a current allocation of multiple computing device resources to multiple tasks based on a corresponding priority level assigned to each task; determine at least one predicted state of the wearable computing device by applying at least a machine learning model to the context information; preemptively reallocate one or more computing device resources from the multiple computing device resources to the at least one task by sending instructions from at least the wearable computing device to an accessory device for executing a first task included in at least one task associated with the at least one predicted state, based on the information indicating the computing device resource allocation profile and the at least one predicted state; and execute the second task by using one or more computing device resources preemptively reallocated to the second task included in the at least one task.

[0169] Example 18: A non-transitory computer-readable storage medium as described in Example 17, wherein, in order to determine at least one predicted state of the wearable computing device by applying at least a machine learning model to context information, the instructions further cause one or more processors to: determine at least one potential state of the wearable computing device based on the context information by applying at least a machine learning model to the context information; determine whether a confidence score associated with at least one potential state of the wearable computing device satisfies a confidence score threshold; and, in response to determining that the confidence score satisfies the confidence score threshold, determine at least one predicted state of the wearable computing device, wherein the at least one predicted state of the wearable computing device is the at least one potential state of the wearable computing device.

[0170] Example 19: A non-transitory computer-readable storage medium as described in any one of Examples 17 and 18, wherein the contextual information of the wearable computing device includes one or more of the following: location data, historical application usage data, battery level, calendar data, time of day, network signal strength, and biometric sensor data.

[0171] Example 20: A non-transitory computer-readable storage medium as described in any one of Examples 17 to 19, wherein, in order to preemptively reallocate one or more computing device resources, the instruction further causes one or more processors to execute one or more processes.

[0172] Example 21: A non-transitory computer-readable storage medium as described in Example 20, wherein one or more processes include one or more of the following: reserving memory for at least one task associated with at least one predicted state; reserving memory for an application associated with at least one predicted state; prefetching data for an application associated with at least one predicted state; adjusting the appropriate priority level of each of a plurality of tasks; adjusting the polling frequency of sensors; and suspending one or more tasks from a plurality of tasks.

[0173] Example 22: A non-transitory computer-readable storage medium as described in any one of Examples 17 to 21, wherein, prior to sending instructions for performing a first task, the instructions further cause one or more processors to: determine that an accessory device is near a wearable computing device; receive information indicating the state of the accessory device; determine, based on the information indicating the state of the accessory device, whether the accessory device has a sufficient amount of available computing device resources; and, in response to determining that the accessory device has the sufficient amount of available computing device resources, send instructions to the accessory device for performing the first task.

[0174] Example 23: A non-transitory computer-readable storage medium as described in any one of Examples 17 to 22, wherein a first task is to prefetch data for an application associated with at least one predicted state, and wherein the instructions further cause one or more processors to: receive the prefetched data for an application associated with at least one predicted state from an accessory device; and perform a second task using the prefetched data.

[0175] Example 24: A non-transitory computer-readable storage medium as described in any one of Examples 17 to 23, wherein the instructions further cause one or more processors to update a computing device resource allocation profile based on one or more computing device resources that have been preemptively reallocated to a second task.

[0176] Example 25: A computer program product for allocating device resources, the computer program product comprising instructions that, when executed by one or more processors, cause the one or more processors to: retrieve context information indicating the current state of a wearable computing device and information indicating a computing device resource allocation profile of the wearable computing device, wherein the computing device resource allocation profile indicates a current allocation of multiple computing device resources to multiple tasks based on a corresponding priority level assigned to each task; determine at least one predicted state of the wearable computing device by applying at least a machine learning model to the context information; preemptively reallocate one or more computing device resources from the multiple computing device resources to the at least one task by sending instructions from at least the wearable computing device to an accessory device for executing a first task included in at least one task associated with the at least one predicted state, based on the information indicating the computing device resource allocation profile and the at least one predicted state; and execute the second task by using one or more computing device resources preemptively reallocated to the second task included in the at least one task.

[0177] Example 26: A computer program product as described in Example 25, wherein, in order to determine at least one predicted state of a wearable computing device by applying at least a machine learning model to context information, the instructions further cause one or more processors to: determine at least one potential state of the wearable computing device based on the context information by applying at least a machine learning model to the context information; determine whether a confidence score associated with at least one potential state of the wearable computing device satisfies a confidence score threshold; and, in response to determining that the confidence score satisfies the confidence score threshold, determine at least one predicted state of the wearable computing device, wherein the at least one predicted state of the wearable computing device is the at least one potential state of the wearable computing device.

[0178] Example 27: A computer program product as described in any one of Examples 25 and 26, wherein the context information of the wearable computing device includes one or more of the following: location data, historical application usage data, battery level, calendar data, time of day, network signal strength, and biometric sensor data.

[0179] Example 28: A computer program product as described in any one of Examples 25 to 27, wherein, in order to preemptively reallocate one or more computing device resources, the instruction further causes one or more processors to execute one or more processes.

[0180] Example 29: A computer program product as described in Example 28, wherein one or more processes include one or more of the following: reserving memory for at least one task associated with at least one predicted state; reserving memory for an application associated with at least one predicted state; prefetching data for an application associated with at least one predicted state; adjusting the appropriate priority level of each of the plurality of tasks; adjusting the polling frequency of sensors; and suspending one or more tasks from the plurality of tasks.

[0181] Example 30: A computer program product as described in any one of Examples 25 to 29, wherein, before sending instructions for performing a first task, the instructions further cause one or more processors to: determine that an accessory device is near a wearable computing device; receive information indicating the state of the accessory device; determine, based on the information indicating the state of the accessory device, whether the accessory device has a sufficient amount of available computing device resources; and, in response to determining that the accessory device has the sufficient amount of available computing device resources, send instructions to the accessory device for performing the first task.

[0182] Example 31: A computer program product as described in any one of Examples 25 to 30, wherein a first task is to prefetch data for an application associated with at least one predicted state, and wherein the instructions further cause one or more processors to: receive the prefetched data of the application associated with at least one predicted state from an accessory device; and perform a second task using the prefetched data.

[0183] Example 32: A computer program product as described in any one of Examples 25 to 31, wherein the instructions further cause one or more processors to update a computing device resource allocation profile based on one or more computing device resources that have been preemptively reallocated to a second task.

[0184] Example 33: A computing system comprising components for performing any combination of the methods described in Examples 1 to 8.

[0185] Example 34: A computing device including components for performing any combination of the methods described in Examples 1 to 8.

[0186] Various examples have been described. These and other examples are within the scope of the appended claims.

Claims

1. A method comprising: The wearable computing device retrieves context information indicating the current state of the wearable computing device and information indicating the computing device resource allocation profile of the wearable computing device, wherein the computing device resource allocation profile indicates the current allocation of multiple computing resources to multiple tasks based on the corresponding priority level assigned to each task. The wearable computing device determines at least one predicted state of the wearable computing device by applying a machine learning model to the context information; Based on the information indicating the computing device resource allocation profile and the at least one predicted state, the wearable computing device preemptively reallocates one or more computing device resources from the plurality of computing device resources to the at least two tasks by sending an instruction from at least the wearable computing device to the accompanying device for executing a first task included in at least two tasks associated with the at least one predicted state; and The second task is performed by the wearable computing device using one or more computing device resources that have been preemptively reallocated to the second task included in the at least two tasks.

2. The method as described in claim 1, wherein, Determining the at least one predicted state of the wearable computing device by applying at least the machine learning model to the context information further includes: Based on the context information, at least one potential state of the wearable computing device is determined by applying the machine learning model to the context information at least once. The wearable computing device determines whether the confidence score associated with the at least one potential state of the wearable computing device meets a confidence score threshold; and In response to determining that the confidence score meets the confidence score threshold, the wearable computing device determines the at least one predicted state of the wearable computing device, wherein the at least one predicted state of the wearable computing device is the at least one potential state of the wearable computing device.

3. The method as described in claim 1, wherein, The contextual information of the wearable computing device includes one or more of the following: location data, historical application usage data, battery level, calendar data, time of day, network signal strength, and biometric sensor data.

4. The method of claim 1, wherein, Preemptive reallocation of the one or more computing device resources further includes executing one or more processes.

5. The method of claim 4, wherein, The one or more processes include one or more of the following: Memory is reserved for the at least two tasks associated with the at least one predicted state. Reserve memory for applications associated with the at least one predicted state. Prefetch data for the application associated with the at least one predicted state. Adjust the corresponding priority level assigned to each of the multiple tasks. Adjust the sensor polling frequency, and Suspend one or more tasks from the plurality of tasks.

6. The method of claim 1, wherein, Before sending the instructions for performing the first task, the method further includes: The wearable computing device determines that the accessory device is close to the wearable computing device; The wearable computing device receives information indicating the status of the accessory device; The wearable computing device determines whether the accessory device has sufficient available computing resources based on the information indicating the state of the accessory device; and In response to determining that the accessory device has the sufficient amount of available computing device resources, the wearable computing device sends the instruction to the accessory device for performing the first task.

7. The method of claim 1, wherein, The first task includes prefetching data for an application associated with the at least one predicted state, and wherein the method further includes: The wearable computing device receives prefetched data from the companion device associated with the at least one predicted state of the application; and The wearable computing device performs the second task using the prefetched data.

8. The method of claim 1, further comprising: The wearable computing device updates the computing device resource allocation profile based on the one or more computing device resources that have been preemptively reallocated to the second task.

9. A wearable computing device, comprising: One or more processors; as well as One or more storage devices storing program instructions that, when executed by the one or more processors, cause the one or more processors to: Retrieve context information indicating the current state of the wearable computing device and information indicating a computing device resource allocation profile of the wearable computing device, wherein the computing device resource allocation profile indicates the current allocation of multiple computing resources to multiple tasks based on the corresponding priority level assigned to each task. At least one predicted state of the wearable computing device is determined by applying a machine learning model to the context information. Based on the information indicating the computing device resource allocation profile and the at least one predicted state, one or more computing device resources from the plurality of computing device resources are preemptively reallocated to the at least two tasks by sending an instruction from at least the wearable computing device to the companion device for performing a first task included in at least two tasks associated with the at least one predicted state; and The second task is executed by using one or more computing device resources that have been preemptively reallocated to the second task included in the at least two tasks.

10. The wearable computing device of claim 9, wherein, In order to determine the at least one predicted state of the wearable computing device by applying at least the machine learning model to the context information, the program instructions further cause the one or more processors to: Based on the context information, at least one potential state of the wearable computing device is determined by applying the machine learning model to the context information at least once. Determine whether the confidence score associated with the at least one potential state of the wearable computing device meets a confidence score threshold; as well as In response to determining that the confidence score satisfies the confidence score threshold, the at least one predicted state of the wearable computing device is determined, wherein the at least one predicted state of the wearable computing device is the at least one potential state of the wearable computing device.

11. The wearable computing device of claim 9, wherein, The contextual information of the wearable computing device includes one or more of the following: location data, historical application usage data, battery level, calendar data, time of day, network signal strength, and biometric sensor data.

12. The wearable computing device of claim 9, wherein, In order to preemptively reallocate the one or more computing device resources, the program instructions further cause the one or more processors to execute one or more processes.

13. The wearable computing device of claim 12, wherein, The one or more processes include one or more of the following: Memory is reserved for the at least two tasks associated with the at least one predicted state. Reserve memory for applications associated with the at least one predicted state. Prefetch data for the application associated with the at least one predicted state. Adjust the corresponding priority level assigned to each of the multiple tasks. Adjust the sensor polling frequency, and Suspend one or more tasks from the plurality of tasks.

14. The wearable computing device of claim 9, wherein, Before sending the instructions for performing the first task, the program instructions further cause the one or more processors to: It is determined that the accessory device is close to the wearable computing device; Receive information indicating the status of the supporting device; Based on the information indicating the state of the supporting device, it is determined whether the supporting device has a sufficient amount of available computing device resources; as well as In response to determining that the accessory device has the sufficient amount of available computing device resources, the instruction for performing the first task is sent to the accessory device.

15. The wearable computing device of claim 9, wherein, The first task includes prefetching data for an application associated with the at least one predicted state, and wherein the program instructions further cause the one or more processors to: Receive prefetched data from the accompanying device associated with the application in relation to the at least one predicted state; and The second task is performed using the prefetched data.

16. The wearable computing device of claim 9, wherein, The program instructions further cause the one or more processors to: The computing device resource allocation profile is updated based on the one or more computing device resources that have been preemptively reallocated to the second task.

17. A non-transitory computer-readable storage medium encoded with program instructions, said program instructions, when executed by one or more processors of a wearable computing device, causing said one or more processors to: Retrieve context information indicating the current state of the wearable computing device and information indicating the computing device resource allocation profile of the wearable computing device, wherein... The computing device resource allocation profile indicates the current allocation of multiple computing device resources to multiple tasks based on the corresponding priority level assigned to each task. At least one predicted state of the wearable computing device is determined by applying a machine learning model to the context information. Based on the information indicating the computing device resource allocation profile and the at least one predicted state, one or more computing device resources from the plurality of computing device resources are preemptively reallocated to the at least two tasks by sending an instruction from at least the wearable computing device to the companion device for performing a first task included in at least two tasks associated with the at least one predicted state. as well as The second task is executed by using one or more computing device resources that have been preemptively reallocated to the second task included in the at least two tasks.

18. The non-transitory computer-readable storage medium of claim 17, wherein, In order to determine the at least one predicted state of the wearable computing device by applying at least the machine learning model to the context information, the program instructions further cause the one or more processors to: Based on the context information, at least one potential state of the wearable computing device is determined by applying the machine learning model to the context information at least once. Determine whether the confidence score associated with the at least one potential state of the wearable computing device meets a confidence score threshold; as well as In response to determining that the confidence score satisfies the confidence score threshold, the at least one predicted state of the wearable computing device is determined, wherein the at least one predicted state of the wearable computing device is the at least one potential state of the wearable computing device.

19. The non-transitory computer-readable storage medium of claim 17, wherein, The contextual information of the wearable computing device includes one or more of the following: location data, historical application usage data, battery level, calendar data, time of day, network signal strength, and biometric sensor data.

20. The non-transitory computer-readable storage medium of claim 17, wherein, Before sending the instructions for performing the first task, the program instructions further cause the one or more processors to: It is determined that the accessory device is close to the wearable computing device; Receive information indicating the status of the supporting device; Based on the information indicating the state of the supporting device, it is determined whether the supporting device has a sufficient amount of available computing device resources; as well as In response to determining that the accessory device has the sufficient amount of available computing device resources, the instruction for performing the first task is sent to the accessory device.

Images