Device usage processing for generating inferred user state

Abstract

Techniques for controlling operation of a device based on inferred state are disclosed. More specifically, at each point in a set of time points, an application execution at an electronic device is detected. For each detected execution, an application usage variable is determined. One or more aggregate metrics are generated based on an aggregation of at least some of the application usage variables. Based on the one or more aggregate metrics, a state identifier corresponding to an inferred state of a user of the electronic device is identified. A device operation identifier associated with the state identifier is retrieved. A device operation associated with the device operation identifier is performed.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of and priority to U.S. Provisional Application No. 62 / 899,598, filed September 12, 2019; and U.S. Patent Application No. 16 / 848,421, filed April 14, 2020; each of which is incorporated herein by reference in its entirety and for all purposes. Technical Field

[0003] The present disclosure generally relates to using artificial intelligence to convert application usage and session data into inferred user state. Background Art

[0004] Device users can transition through various states over a period of years or even shorter periods. For example, a user's intelligence may change over time, while a user's distraction or stress level may fluctuate within a single hour. In some cases, it's impossible to definitively identify a user's state. State detection can be particularly difficult when the state includes mental state, as the user may be unwilling or unable to accurately identify their state. However, such information can be used to alter device functionality and / or add value to the user experience. Therefore, controlling device functionality based on automatically generated inferences about the user's state would be advantageous. Summary of the Invention

[0005] In some embodiments, a computer-implemented method is provided. At each of a set of time points, execution of an application at an electronic device may be detected. For each detected execution, application usage variables may be determined. One or more aggregate metrics may be generated based on an aggregation of at least some of the application usage variables. Based on the one or more aggregate metrics, a state identifier corresponding to an inferred state of a user of the electronic device may be determined. A device operation identifier associated with the state identifier may be retrieved. A device operation associated with the device operation identifier may be executed.

[0006] Some embodiments of the present disclosure include a system comprising one or more data processors. The system may include a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform part or all of one or more methods and / or part or all of one or more processes disclosed herein.

[0007] Some embodiments of the disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium including instructions configured to cause one or more data processors to perform part or all of one or more methods disclosed herein and / or part or all of one or more processes disclosed herein.

[0008] The following DETAILED DESCRIPTION provides a better understanding of the nature and advantages of the application. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 An illustration of techniques for processing application usage data to trigger specific device operations is shown in accordance with some embodiments of the application.

[0010] Figure 2 An illustration of techniques for processing application usage data to infer target metrics is shown in accordance with some embodiments of the application.

[0011] Figure 3 is a simplified block diagram of an electronic device in accordance with some embodiments of the application.

[0012] Figure 4 is a flowchart of a process for controlling device operations based on detected application execution in accordance with some embodiments of the application.

[0013] Figure 5 is a flowchart of a process for processing session data to infer user state in accordance with some embodiments of the application.

[0014] In the drawings, like reference numerals can be used to denote like components and / or features throughout the several views. Further, like components can be distinguished by following the reference numeral with a dashed line and a second label for the component. If merely the first reference numeral is used in the specification, the description is applicable to any one of the like components having the same first reference numeral irrespective of the second reference label identifying a type of the component. DETAILED DESCRIPTION

[0015] Techniques are disclosed for controlling operation of a device based on inferred states. More specifically, event streams can be collected at a user device to track usage of various applications. For example, each of a plurality of session event streams can characterize each session during device usage (e.g., by identifying times at which the device is unlocked and locked). Each of a plurality of application event streams can identify each application that is in focus (i.e., executing, in view, receiving user interaction input, and / or object of user attention) and can further characterize a time period in which the application is in view (e.g., by identifying a start and end time and / or a duration). An embedded representation can be generated for each application represented in the application event streams, and a session-level embedded representation can be generated for each session based on the embedded representations of the applications used in the session. Each session can be assigned to a cluster based on the session-level embedded representations. A state (e.g., a cognitive state of the user) can be inferred based on the aggregated cluster assignments. A device operation can be identified based on the state, and the device operation can be performed. For example, a notification with a recommendation to perform a particular activity (e.g., communicate a state to a medical professional or perform a prescribed exercise?) can be presented and / or a communication can be transmitted. By processing event streams in this way, a state can be inferred using passive monitoring.

[0016] Figure 1 A representation 100 of techniques for processing application usage data to trigger particular device operations is shown in accordance with some embodiments of the application. Application event streams are collected and processed at an electronic device. An application event stream can correspond to a single usage session. A single usage session can correspond to a period of time in which a user is inferred to be (e.g., actively) engaged with the device, a period of time between when the device is unlocked and when the device is subsequently locked, or a period of time of continuous user input with an inter-input interval that remains below a predefined threshold. An application event stream can identify applications that are used during a session. In some embodiments, an application event stream can further identify a time period in which each application is in view (e.g., by identifying a start and end time and / or a duration). Figure 1 In the depiction, each box in the event stream represents usage of a single application. Each pattern represents a particular application. For example, box 105a represents a first use of a particular application, and box 105b represents a second use of the particular application. In this case, three applications are used, but one of the applications is used twice. An application can be identified as being in use when it is in view (e.g., such that a window or interface of the application is in front of a window or interface of each other open application and / or such that at least a portion of the window or interface of the application is being presented on the screen of the device), for example. For each application use, the application event stream can further include a duration of time in which the application is in use.

[0017] An application-specific embedded representation can be generated for each of the applications in the event stream. The embedded representation can include a vector in a high-dimensional vector space (e.g., having at least 20 dimensions, at least 50 dimensions, or at least 100 dimensions). Figure 1 A set of application-level embedded representations 110 is depicted that includes 4 total (and 3 unique) application-specific embedded representations. Each application-specific embedded representation includes values in 5 dimensions (corresponding to various patterns). Each dimension / pattern can correspond to a particular feature. Thus, for example, box 110a can represent a value in the 5th dimension that represents a first use of the first application. The features can be identified using, e.g., principal component analysis or a neural network. The vector space and embedding can be defined to facilitate supporting (1) predicting a given application based on other applications included in the same sequence and / or (2) predicting one or more other applications in the sequence based on a given application in the sequence. The predictions can but need not be based on and / or take into account the order in which the applications were used.

[0018] One or more aggregate metrics that can include a session-level vector representation (e.g., corresponding to representation 115) can be generated based on the application-specific embedded representations associated with a session. In some cases, the session-level vector representation is defined to be equal to an average or weighted average (based on usage duration, based on detected interactions with the applications, based on cross-user popularity of the applications, etc. set weights) across the embedded representations of the applications used in the session. In some cases, the session-level vector representation is defined to be equal to one of the application-specific embedded representations associated with a cluster. For example, the session-level vector representation can be defined to be set to the application-specific embedded representation associated with the particular application that was opened the highest number of times and / or is associated with the highest cumulative view-in-time within the session.

[0019] A set of clusters can be identified using a clustering technique and the session-level vector representations. In the example depiction, the sessions are assigned to a third cluster, c3. The clustering technique can include, e.g., a latent Dirichlet allocation, a K-means clustering, a mean shift clustering, a density-based spatial clustering of applications with noise (DBSCAN), an expectation maximization (EM) clustering, and / or a hierarchical clustering. The clustering technique can receive as input, e.g., a plurality of session-level vector representations corresponding to a single user and a plurality of sessions. The output of the clustering technique can include a cluster assignment for each session-level vector representation (and thus for each session). Figure 1

[0020] ​The session-specific cluster assignments can, but need not, be further based on one or more other variables. In the depicted case, the cluster assignments are further based on a number of application usage (4 in the depicted case) and total time of the session in the application event stream, The cluster assignments can (in addition or alternatively) depend on data that is not related to application usage but is collected or identified during the session (e.g., heart rate, time of day, day of week, acceleration data, and / or location). The cluster assignments can (in addition or alternatively) depend on user information, such as age of the user and / or gender of the user.

[0021] In addition to the cluster assignments generated for the particular session, cluster assignments can similarly be generated for each of a plurality of other sessions to produce cluster assignments for the other sessions (c5, c2, c3, c3, cl, c5, c2). The particular session and the other sessions can include every session that occurs within a continuous time window, such as a portion of a day, a day, a week, a month, a year, or a number of years. In some cases, the particular session and the other sessions can include every session that occurs within a non-continuous time window (e.g., every morning within a month; every weekday afternoon within a week; every time interval within a year during which acceleration and / or gyroscope data indicates that the user is walking, standing, or sitting).

[0022] The cluster assignments can be aggregated to generate, for example, histogram data (e.g., indicating the amount or fraction of sessions assigned to each of a set of clusters); a string or list of cluster assignments; and / or an array identifying the cluster assignment and one or more other session variables (e.g., session duration) for each session. In some cases, a cross-session metric can be determined based on the cluster assignments. In the depicted case, the cross-session metric is the most common cluster assignment Mo(c) (e.g., corresponding to a mode statistic). Other potential cross-session metrics that can be generated alternatively or in addition can include, for example, a fraction of the cluster assignments that are equal to the most common cluster assignment, a percentage of the cluster assignments that identify a particular predefined cluster, and / or a weighted average of the cluster assignments. The cross-session metric can additionally depend on other session data beyond the cluster assignments. For example, each session can be associated with a cluster assignment and a session duration statistic. In the illustrated case, the cross-session metric Mo(c) depends on the cluster assignments for the other sessions (c5, c2, c3, c3, cl, c5, c2). The cross-session metric can then represent a particular cluster that is associated with the highest cumulative duration.

[0023] A state can be inferred based on the cross-session metric and / or the cluster assignments (e.g., including a distribution and / or histogram of the cluster assignments across clusters). For example, in the depicted case, the inferred state is s jThe state can include a state of the user, such as a cognitive state, a cognitive health state, an emotion, an energy level, or a recent sleep quality. Inferring the state can include using a trained model to transform the cross-conversation metrics, the cluster assignments, and / or one or more other variables (start and / or time of each session, duration of each session, frequency of sessions; age of the user; average steps or movement in a recent time period; etc.). For example, the trained model can include a logistic regression model (e.g., an LI -regularized logistic-regression model), a linear regression model, an explicit generative model (e.g., linear discriminant analysis), a neural network, or a gradient boosting tree.

[0024] In some cases, inferring the state can include retrieving a stored state identifier associated with the cross-conversation metrics. For example, when the cross-conversation metrics are defined as the most common cluster, a data store can include a representation of each of a set of clusters and an associated state identifier for each cluster representation. In some cases, multiple clusters can be associated with a single identifier. In some cases, each state identifier is associated with only a single cluster. In various cases, a single cluster can be associated with only a single state identifier or with multiple state identifiers (e.g., corresponding to different types of states).

[0025] One or more device operation identifiers (e.g., o 1,j and o 2,j ) can be determined based on the inferred state. The determining can include querying a data store with an identifier of the inferred state and receiving a query result that includes a device operation identifier. The device operation identifier can correspond to a device operation that includes, for example, presenting a notification that indicates or identifies the inferred state; presenting a notification that identifies a suggested action for the user to perform (e.g., see a doctor; perform physical activity; get more sleep; use a particular type of device application; etc.); generating a communication that indicates the inferred state and identifies a user of the device, and sending the communication to another device (e.g., of a medical care provider); launching a particular application on the device; etc.

[0026] Figure 2An illustration 200 of techniques for processing application usage data to infer target metrics according to some embodiments of the application is shown. At block 205, session-specific data is accessed. The session-specific data can identify, for each of a set of sessions, a duration of the session, a number of applications used (e.g., selected, opened, and / or viewed in) during the session, an embedded representation of the applications used during the session, a category of each application used during the session, a distribution across application categories, and the like. Additional contextual features associated with each session can further be accessed. For example, the contextual features can include a start time of the session, an end time of the session, a day of the week, a motion state of the device (e.g., detected using a GPS receiver, a gyroscope, and / or an accelerometer), a heart rate of the user, acceleration data, an age of the user (e.g., determined based on previously received user input), and / or an activity engaged in by the user during the session.

[0027] At block 210, each of the sessions is assigned to a cluster. The cluster assignment can be made using clustering and / or classification techniques. For example, the cluster assignment can be made using k-means analysis, neural networks, or distance-based methods.

[0028] At block 215, a time series data structure is generated. The time series data structure can include a set of data points, each data point corresponding to a session. Each data point can include an identifier of the cluster to which the session is assigned and a time variable indicating when the session occurred. In some cases, the time variable includes multiple sub-variables, such as a time of day, a day of the week, and / or a date. The time series data structure can then group or order the session data based on one or more of the sub-variables. For example, the cluster assignments of sessions that occurred in the morning can be analyzed separately from (e.g., can but need not be analyzed separately from) the cluster assignments of sessions that occurred in the afternoon, evening, or night. As another example, the cluster assignments of sessions that occurred on weekdays can be analyzed separately from the cluster assignments of sessions that occurred on weekends.

[0029] At block 220, features are identified based on the cluster assignments. In some cases, for each cluster represented in the cluster assignments, a number of sessions assigned to the cluster and / or a duration variable (e.g., total duration or average duration) is determined. In some cases, the features include the time series data generated at block 215. The features can include identifying the cluster to which the highest number of sessions were assigned. In some cases, contextual information associated with the sessions can be used during feature extraction. For example, a distribution can be generated using data (e.g., identifying a number of sessions assigned to different clusters or indicating a duration variable for each cluster) when the user’s heart rate is below a threshold. As another example, a multi-dimensional feature can be generated that identifies a session metric for each combination of a particular cluster and a particular contextual information bin (e.g., a time period of the day).

[0030] At block 225, the features are processed by a trained model. The trained model can include a machine learning model, a classifier, a regression model, a neural network, etc. The output of the model can include numerical and / or categorical values.

[0031] At block 230, a state identifier is determined. The state identifier can correspond to an inferred state of the user, such as the user’s cognitive health, mood, energy level, or recent sleep quality. The state identifier can be determined by, for example, comparing the output of the model to one or more thresholds, applying an activation function to the output of the model, and / or identifying a state identifier associated with the output. In some cases, the state identifier variable includes the output of the model. In some cases, the state identifier is rendered at the user device and / or transmitted to another device. In some cases, the determination of the state identifier triggers another type of device operation (e.g., adjusting a morning alarm time or a bedtime; changing a station on a music application operating at the device; etc.). In some cases, the state identifier can be logged locally and / or remotely (e.g., associated with a timestamp and / or identifier of the user device).

[0032] Figure 3 is a simplified block diagram of a device 300 according to an embodiment of the application. The device 300 can include, for example, a smartphone, a wearable device, a laptop computer, a tablet computer, or a desktop computer. The device 300 can include a processing subsystem 302, a storage subsystem 304, a user interface 306, one or more connectivity components (e.g., an RF interface 308 and / or a connector interface 310), a power subsystem 312, and an environmental sensor 314. The device 300 can also include other components (not explicitly shown).

[0033] Storage subsystem 304 can be implemented, for example, using magnetic storage media, flash memory, other semiconductor memory (e.g., DRAM, SRAM), or any other non-transitory storage medium or combination of media, and can include volatile and / or non-volatile media. In some embodiments, storage subsystem 304 can store media items, such as audio files, video files, images or graphics files; information about user contacts (names, addresses, phone numbers, etc.); appointments and events; memos; and / or other types of information, examples of which are described below. In some embodiments, storage subsystem 304 can also store one or more application programs (or apps) 333 to be executed by processing subsystem 302 (e.g., a video game program, a personal information management program, a media playback program, an interface program associated with a particular host device and / or host device function, etc.).

[0034] Storage subsystem 304 can include a device lock control 334 that controls whether the device is in a locked or unlocked configuration. The default configuration can be the locked configuration, and device lock control 334 can transition device 300 to the unlocked configuration upon detecting that a valid numeric, text, or biometric password has been entered. Device lock control 334 can transition device 300 back to the locked configuration upon detecting, for example, inactivity for a particular period of time (e.g., a predefined period of time and / or a period of time as indicated in settings). Device 300 can use a reduced number of application programs 333 while in the locked configuration. For example, device 300 can generally prevent access to some or all of application programs 333. Device lock control 334 can generate a session event stream that identifies the time of unlocking or locking each time device 300 is unlocked or locked. Time can be determined using clock application 335.

[0035] Storage subsystem 304 can include an application usage monitor 336. Application usage monitor 336 can detect which application program is in use at various points in time. An application program can be characterized as being in use if, for example, a window or interface of the application program is being presented and / or is in front of any other window or interface. Application usage monitor 336 can selectively monitor application usage during a session during application unlocking. Application usage monitor 336 can generate an application event stream that identifies each application program that was used during a session and the period of time (e.g., via a start and end time) that each application program was used. Time can be determined using clock application 335.

[0036] The storage subsystem 304 can include a state detector 337 that infers a state based on the application event stream and the potential session event stream. The state can include a state of a user of the device 300. Inferring the state can include identifying a cluster assignment for each session based on which application(s) are used during the session and identifying the state based on the cluster assignment. The state detector 337 can trigger and / or perform actions based on the inferred state. For example, a particular notification (e.g., identifying the state) can be presented, a communication can be generated and transmitted to another device, and / or a particular application can be opened.

[0037] The user interface 306 can include any combination of input devices and output devices. A user can operate input devices of the user interface 306 to invoke functions of the device 300, and can view, listen to, and / or otherwise experience output from the device 300 via output devices of the user interface 306. The device lock control 334 can use data from the user interface 306 to determine whether to change a locked / unlocked configuration of the device 300. The application usage monitor 336 can use data from the user interface 306 to determine which application is used during different points of a session. The state detector 337 can use output components of the user interface 306 to perform actions in response to state inferences.

[0038] Examples of output devices include a display 320, a tactile output generator 321, and a speaker 322. The display 320 can be implemented using compact display technologies such as LCD (liquid crystal display), LED (light-emitting diode), OLED (organic light-emitting diode), etc. In some embodiments, the display 320 can incorporate a flexible display element or a curved glass display element, allowing the device 300 to conform to a desired shape. One or more speakers 321 can be provided using small speaker technologies, including any technology capable of converting an electronic signal into an audible sound wave. In some embodiments, the speaker 321 can be used to produce tones (e.g., a bleep or ring) and can or can not be capable of reproducing sound such as speech or music with any particular fidelity. The tactile output generator 322 can be, for example, a device that converts electrical signals into vibrations; in some embodiments, the vibrations can be strong enough to be felt by a user wearing the device 300, but not so strong as to produce a unique sound.

[0039] Examples of input devices include a touch sensor 325, a camera 327, and a microphone 328. The touch sensor 325 can include, for example, a capacitive sensor array having the ability to locate a particular point or area on the surface of the sensor where contact is made, and in some cases, the ability to distinguish multiple concurrent contacts. In some embodiments, the touch sensor 325 can be overlaid on the display 320 to provide a touch screen interface, and the processing subsystem 304 can translate the touch events (including taps and / or other gestures made with one or more contacts) into particular user input depending on what is currently being displayed on the display 320.

[0040] The camera 327 can include, for example, a compact digital camera including an image sensor such as a CMOS sensor and optical components (e.g., a lens) arranged to focus an image onto the image sensor, along with control logic operable to capture and store still and / or video images using the imaging components. The images can be stored, for example, in the storage subsystem 304 and / or transmitted by the device 300 to other devices for storage. Depending on the implementation, the optical components can provide a fixed focal length or a variable focal length; in the latter case, autofocus can be provided. In some embodiments, the camera 327 can be disposed along an edge (e.g., a top edge) of a surface member of the device, and oriented to allow a user to capture images of nearby objects in the environment, such as barcodes or QR codes. In other embodiments, the camera 327 can be disposed on a front surface of a surface member of the device, for example, to capture images of a user. Zero, one, or more cameras can be provided depending on the implementation.

[0041] The microphone 328 can include any device that converts sound waves into electrical signals. In some embodiments, the microphone 328 can be sensitive enough to provide a representation of particular words spoken by a user; in other embodiments, the microphone 328 can be used to provide an indication of the overall ambient sound level without necessarily providing a high-quality electronic representation of particular sounds.

[0042] In some embodiments, the user interface 306 can provide output to and / or receive input from an auxiliary device such as a headset. For example, an audio jack 323 can connect to an auxiliary device via an audio cable (e.g., a standard 2.5 mm or 3.5 mm audio cable). The audio jack 323 can include an input path and / or an output path. Thus, the audio jack 323 can provide audio to and / or receive audio from an auxiliary device. In some embodiments, a wireless connection interface can be used to communicate with an auxiliary device.

[0043] The processing subsystem 302 can be implemented as one or more integrated circuits, such as one or more single-core or multi-core microprocessors or microcontrollers, examples of which are known in the art. In operation, the processing system 302 can control the operation of the device 300. In various embodiments, the processing subsystem 304 can execute a variety of programs in response to program code and can maintain multiple simultaneously executing programs or processes. At any given time, some or all of the program code to be executed can be resident in the processing subsystem 304 and / or in storage such as the storage subsystem 304.

[0044] With appropriate programming, the processing subsystem 302 can provide various functionality for the device 300. For example, in some embodiments, the processing subsystem 302 can execute an operating system (OS) 332 and various applications 333, such as a phone interface application, a text messaging interface application, a media interface application, a fitness application, and / or other applications. In some embodiments, some or all of these applications can interact with a host device, e.g., by generating messages to be sent to the host device and / or by receiving and interpreting messages from the host device. In some embodiments, some or all of the applications can operate locally on the device 300. For example, if the device 300 has a local library of media stored in the storage subsystem 304, the media interface application can provide a user interface to select and play locally stored media items.

[0045] The processing subsystem 302 can also execute code for one or more components of the device 300. For example, the processing subsystem 302 can execute code for a device lock control 334, a clock 335, an application usage monitor 336, and / or a status detector 337.

[0046] The RF (radio frequency) interface 308 can allow the device 300 to communicate wirelessly with various devices. The RF interface 308 can include RF transceiver components such as antennas and supporting circuitry to enable data communication over a wireless medium, e.g., using WiFi (IEEE 802.11 family of standards), Bluetooth® (a family of standards promulgated by Bluetooth SIG, Inc.), or other protocols for wireless data communication. In some embodiments, the RF interface 308 can implement a Bluetooth LE (low energy) proximity sensor 309 that supports proximity detection by signal strength assessment and / or other protocols for determining proximity to another electronic device.

[0047] ​In some embodiments, RF interface 308 can provide near-field communication ("NFC") capabilities, e.g., implementing the ISO / IEC 18092 standard or the like; NFC can support wireless exchange of data between devices at very short range (e.g., 20 centimeters or less). RF interface 308 can be implemented using a combination of hardware components (e.g., driver circuitry, antennas, modulators / demodulators, encoders / decoders, and other analog signal processing circuitry and / or digital signal processing circuitry) and software components. Multiple different wireless communication protocols and associated hardware can be incorporated into RF interface 308.

[0048] Connector interface 310 can allow device 300 to communicate with various devices (e.g., servers or collaborative devices) via wired communication paths, e.g., using Universal Serial Bus (USB), Universal Asynchronous Receiver / Transmitter (UART), or other protocols for wired data communication. In some embodiments, connector interface 310 can provide a power port, allowing device 300 to receive power, e.g., to charge an internal battery. For example, connector interface 310 can include a connector such as a micro-USB connector or a custom connector and supporting circuitry. In some embodiments, a device can be powered or charged via inductive charging. In some embodiments, the connector can be a custom connector that provides dedicated power and ground contacts, as well as digital data contacts that can be used to implement different communication technologies in parallel; for example, two pins can be allocated as USB data pins (D+ and D-) and two other pins can be allocated as serial transmit / receive pins (e.g., implementing a UART interface). The pin configuration for a particular communication technology can be hardwired or negotiated at the time a connection is being established. In some embodiments, the connector can also provide for connection of audio and / or video signals, which can be transmitted to or from another device in analog and / or digital format.

[0049] In some embodiments, the connector interface 310 and / or the RF interface 308 can be used to support synchronization operations in which data is transferred from another device to the device 300 (or vice versa). For example, a user can be able to customize settings and other information for the device 300. Although the user interface 306 can support data entry operations, the user can find it more convenient to define the custom information on a separate device (e.g., a tablet or smart phone) that has a larger interface (e.g., including a real or virtual alphanumeric keyboard) and then transfer the custom information to the device 300 via a synchronization operation. The synchronization operation can also be used to load and / or update other types of data in the storage subsystem 304, such as media items, applications, personal data, and / or operating system programs. The synchronization operation can be performed in response to an explicit user request and / or automatically, for example when the device 300 resumes communication with a particular host device or in response to either device receiving an update to its copy of the synchronization information.

[0050] The environmental sensors 314 can include various electronic, mechanical, electro-mechanical, optical, or other devices that provide information about the external conditions surrounding the device 300. In some embodiments, the sensors 314 can provide digital signals to the processing subsystem 302, for example on a streaming basis or in response to polling by the processing subsystem 302, as desired. Any type and combination of environmental sensors can be used; shown by way of example are an accelerometer 342, a magnetometer 344, a gyroscope 346, and a GPS receiver 348.

[0051] Some environmental sensors can provide information about the location and / or motion of the device 300. For example, the accelerometer 342 can sense acceleration (relative to free fall) along one or more axes, for example using piezoelectric or other components in combination with associated electronics to generate a signal. The magnetometer 344 can sense the ambient magnetic field (e.g., the Earth's magnetic field) and generate a corresponding electrical signal that can be interpreted as a compass direction. The gyroscope sensor 346 can sense rotational motion in one or more directions, for example using one or more MEMS (Micro-Electro-Mechanical System) gyroscopes in combination with associated control and sensing circuitry. The Global Positioning System (GPS) receiver 348 can determine a location based on signals received from GPS satellites. In some cases, data detected from one or more environmental sensors can be used to recognize a gesture that indicates a particular input. For example, rather than using a touch screen to select a notification action and / or dismiss an alarm or notification, such information can be conveyed by a particular arm motion.

[0052] Other sensors can also be included in addition to or instead of these examples. For example, a sound sensor can combine the microphone 326 along with associated circuitry and / or program code to determine, for example, the decibel level of ambient sound. A temperature sensor, a proximity sensor, an ambient light sensor, and so on can also be included.

[0053] Power subsystem 312 can provide power and power management capabilities for device 300. For example, power subsystem 314 can include a battery 340 (e.g., a rechargeable battery) and associated circuitry to distribute power from battery 340 to other components of device 300 that require power. In some embodiments, power subsystem 312 can also include circuitry to charge battery 340, e.g., when connector interface 310 is connected to a power source. In some embodiments, power subsystem 312 can include a“wireless” charger such as an inductive charger for charging battery 340 without relying on connector interface 310. In some embodiments, power subsystem 312 can include other power sources in addition to or instead of battery 340, such as a solar cell.

[0054] In some embodiments, power subsystem 312 can control power distribution to components within device 300 to efficiently manage power consumption. For example, power subsystem 312 can place device 300 into a hibernation state (e.g., a locked configuration) automatically when receiving a signal from device lock control 334 to transition and / or when indicating that no input activity has been detected for at least a predefined period of time. The hibernation state can be designed to reduce power consumption; thus, user interface 306 (or components thereof), RF interface 308, connector interface 310, and / or environmental sensors 314 can be powered down (e.g., enter a low power state or be completely shut off), while other sensors can be powered up (continuously or intermittently) to detect when a user puts on or uses the device.

[0055] Power subsystem 312 can also provide other power management capabilities, such as regulating power consumption of other components of device 300 based on the source and amount of available power, monitoring the power stored in battery 340, generating a user alert if the stored power falls below a minimum level, and the like.

[0056] In some embodiments, the control functions of power subsystem 312 can be implemented using programmable or controllable circuitry operating in response to control signals generated by processing subsystem 302 in response to program code executed thereon (e.g., of device lock control 334) or as a standalone microprocessor or microcontroller. In some embodiments, the control functions of power subsystem 312 can be based on user interaction with the device (e.g., powering down components if the device has not interacted with the device in a particular manner such as a touch, button press, or network activity for a defined period of time).

[0057] In various embodiments, the user interface can include any or all of the above-described components, as well as other components not explicitly described, in any combination. For example, in some embodiments, the user interface can include, e.g., only a touch screen, or a touch screen and a speaker, or a touch screen and a tactile device. Where the device has an RF interface, the connector interface can be omitted, and all communication between the device and other devices can be conducted using wireless communication protocols. Wired power connections, e.g., for charging the device's battery, can be provided independent of any data connections.

[0058] Furthermore, although the device is described with reference to particular blocks, it is to be understood that these blocks are merely illustrative of the components that can be employed in the device and are not intended to suggest any particular physical arrangement of the components. Furthermore, it is not intended for these blocks to correspond directly to physical components. One or more of the blocks can be configured to perform various operations, e.g., by programming a processor or providing an appropriate control circuitry, and each block can be reconfigurable or non-reconfigurable as desired. Embodiments of the present application can be implemented in various apparatuses including electronic devices implemented using any combination of circuitry and software. It also not required that a given embodiment implement each block of Figure 3

[0059] Figure 4 is a flowchart of a process 400 for controlling device operation based on detected application program execution in accordance with some embodiments of the present application. Process 400 can be performed in part or in whole at a user device. Process 400 begins at block 405 where execution of an application program at the device is detected. For example, block 405 can include detecting events including each time an application program is opened, each time an application program is closed, and / or which application program is in use and / or presented in the foreground. In some cases, event monitoring is limited to only events detected when the user device is in an unlocked configuration.

[0060] At block 410, for each of the detected application program executions, one or more application program usage variables are determined. The application program usage variables can identify the particular application program that was opened, used, closed, and / or presented in the foreground. In some cases, different types of application program usage variables are determined. For example, the one or more application program usage variables can include an identifier of the particular application program; a duration of time that the particular application program was in use; and one or more characteristics of the particular application program.

[0061] ​At block 415, one or more aggregated metrics are generated based on the application usage variables or their processed versions. The one or more aggregated metrics can include a single variable or a set of variables representing multiple application executions (e.g., occurring within a single session). The one or more aggregated metrics can be determined by, for example, identifying the application that is used the most times and / or has the longest cumulative time across multiple application executions. The one or more aggregated metrics can include session-level embedded representations and / or vectors.

[0062] In some cases, the application usage variables including the identifiers of the applications are converted into embedded representations of the applications using embeddings. The embeddings can be learned based on a training dataset that includes vectors identifying the applications used in various sessions (e.g., preserving the order of usage or not indicating the order of usage). The embeddings can be generated to facilitate predicting one or more application identifiers in a given vector in the case where the one or more other application identifiers in the vector are identified. The aggregated metrics can then be determined based on the embeddings. For example, the aggregated metrics can be defined as the average of the embedded representations of the applications used in a session or the embedded representation of the application that is used the most times or has the longest cumulative time in the session.

[0063] At block 420, a state identifier corresponding to an inferred state of a user of the device is determined. For example, the inferred state can correspond to a cognitive health level, an emotion, a degree of focus, and / or whether the user is sufficiently rested. The state identifier can be determined based on the one or more aggregated metrics. In some cases, the one or more aggregated metrics include multiple aggregated metrics corresponding to a set of sessions. Thus, an identifier for each session can be associated with at least one aggregated metric that can be used to identify a cluster of sessions. The cluster assignment can be used to determine the inferred state.

[0064] At block 425, a device operation identifier associated with the state identifier is retrieved. The device operation identifier can correspond to a device operation including, for example, presenting a particular type of notification, transmitting a particular type of communication to another device, opening an application, scheduling one or more notifications (e.g., a recommended sleep time), and / or scheduling another state inference process. At block 430, the device operation associated with the device operation identifier is performed. The device operation can include, for example, presenting (or not presenting) a notification. The notification can include an alert, a recommendation, and / or information. For example, the notification can identify the inferred state as being consistent with a suboptimal level of alertness (and / or cognitive health) and can include a recommendation for the user to call a health provider. The device operation can include, for example, additionally or alternatively, changing a device associated scheduled do not disturb period and / or bedtime. For example, when the inferred state corresponds to low alertness, a device associated do not disturb (e.g., during which no alerts and / or notifications are presented) can be automatically modified to begin at an earlier time. As another example, a bedtime can be shifted to an earlier time in response to an inferred state corresponding to low attentiveness (e.g., to prompt the user to go to sleep at an earlier time).

[0065] Figure 5 is a flowchart of a process 500 for processing conversation data to infer a user state according to some embodiments of the present application. The process 500 can be performed partially or entirely at a user device. The process 500 begins at block 505 where conversation data is collected. The conversation data can correspond to a plurality of conversations. The conversation data can include, for example, an identifier of each application used during a conversation; a duration of time each application was used during the conversation; a duration of time of the conversation; and / or contextual information of the conversation (e.g., start and / or end time, day of the week, acceleration data, location data, etc.).

[0066] Blocks 510-520 can be performed for each conversation represented in the conversation data. At block 510, each application used in the conversation can be detected. Additional information about the applications can be further detected, such as an order in which the applications were used, a duration of time each individual application was used, etc. The detection and determination of block 510 can be performed using the conversation data collected at block 505.

[0067] At block 515, one or more conversation variables of the conversation are determined based on the applications used in the conversation. The conversation level variables can represent characteristics of the applications used during the conversation, an order of use of various types of applications in the conversation, duration of use data, etc.

[0068] In some cases, the session-level variable includes an embedded representation. For example, an embedded representation specific to an application can be generated using embeddings for each application detected in block 510. The embeddings can be generated based on the extent to which various applications commonly occur within various sessions and the relative order of use of applications within a session. The session-level embedded representation can then be determined based on the application-level embedded representations of the applications within the session. The session-level embedded representation can be defined as, for example, a mean, median, or mode of the corresponding application-level embedded representations.

[0069] At block 520, a session identifier associated with a session can be assigned to a cluster based on the session variables. For example, the cluster assignment can be performed using k-means, component analysis, distance-based methods, etc.

[0070] At block 525, a user state can be inferred based on the session cluster assignment. The user state can be one of a set of predefined user states. The cluster assignment can be fed (e.g., processed or in raw form) to a trained model, with or without accompanying contextual data. For example, the session date and cluster assignment can be processed by a recurrent neural network model. In some cases, a histogram of the cluster assignments can be generated and processed by the trained model to identify the state.

[0071] At block 530, an indicator of the user state is output. Block 530 can include presentation and / or transmission. The presentation and / or transmission can, but need not, identify the user state. The presentation and / or transmission can include an action recommended for performance. In some cases, a device action is automatically performed instead of or in addition to outputting the indicator.

[0072] It should be appreciated that variations of the disclosed embodiments are contemplated. For example, processing session and application usage data to infer a state can be performed partially or entirely in a remote and / or cloud computing system. As another example, interpretable features of an application or session can be used instead of or in addition to embedded representations of the application or session. The interpretable features can include, for example, a category of the application (e.g., as indicated via metadata of the application or by querying a source); a popularity of the application (e.g., generally or across a given demographic, either of which can be determined by querying a source), a time period since last use of the application associated with the session on the device, etc.

[0073] Embodiments

[0074] Phone usage data was collected for each of 31 people with clinically diagnosed cognitive impairment under normal life conditions and 82 healthy controls over a period of 12 weeks. The subjects were between 60 and 75 years of age, with a median age of 66. The phone usage data included application usage event streams consisting of timestamps of all openings and closings of applications and the identification of the application that was opened or closed (corresponding to when the application was in view). In addition, the phone usage data included lock / unlock event streams consisting of timestamps of all phone unlock and lock events and an indication of whether the event was a lock event or an unlock event. Across subjects, the application usage event streams represented approximately one million application launches, and the lock / unlock event streams represented approximately 500,000 phone unlock events.

[0075] An application-level embedded representation was generated for each application identified in any of the application usage event streams. The embeddings were learned by training a model that helped predict an application in an application usage event stream based on neighboring applications in the event stream.

[0076] The lock / unlock event streams were used to identify individual usage sessions. A session was defined as the time period between an unlock event and a lock event. For each session, a session-level embedded representation was defined as the average of the application-level embedded representations of all applications used during the session. Clusters were assigned to each session using k-means clustering that represented session types. Thus, the phone usage of a given user was then represented by a time series of session types. A histogram of session types was generated, and a cognitive health was classified using L1-regularized logistic regression.

[0077] The model parameters learned by training included the weights of the logistic regression classifier. The model's hyperparameters included the number of session types used to cluster sessions and the strength of regularization used in the logistic regression. Since the number of users in the dataset was relatively small, the analysis was performed using N-fold leave-one-out training / test splits. For each N-fold split, the model hyperparameters were selected via a second leave-one-out cross-validation loop over N-1 training subjects. The final performance was evaluated by computing the area under the receiver operating characteristic curve using the prediction scores from each of the N test predictions.

[0078] The complete model required learning application embeddings, grouping applications into sessions, and clustering sessions into session types. Here, the impact of each of several model configurations was evaluated as the ability to infer cognitive health. For each baseline, the representation of the phone usage was varied, while cognitive health was predicted using logistic regression from histograms of representations as in the complete model.

[0079] The first baseline (B1) aims to isolate the effect of grouping application event streams into interactive sessions. Each application is represented by its app2vec embedding and clustering is performed on the application embeddings to identify different application types. Users are represented by a histogram of application types rather than session types. The next three baselines (B2, B3, B4) aim to isolate the effect of clustering sessions based on learned application embeddings. Each application is represented by one-hot encoding over all applications (B2) or one-hot encoding over all app store categories in the dataset (B3). The learned application embeddings (B4) are randomly permuted to destroy the learned structure in the embedding space but preserve the statistics of the embeddings. In these baselines, applications are still grouped by session. For B2 and B3, session types are represented by the one-hot encoding of the application that was open the longest during the session. For B4, sessions are represented and clustered into session types in the same way as the full model.

[0080] Two baselines with minimal structure (B5, B6) are evaluated. For B2 and B3, we represent applications using one-hot encoding but do not group them into interactive sessions using phone unlock / lock events.

[0081] The results show that the cognitive-health estimates generated using the full model are more accurate than baseline state-of-the-art estimates. Grouping application-level data into sessions improves accuracy when using learned embeddings and when using one-hot encodings. Specialized classification using application identifiers or app store categories is associated with improved accuracy when classifying applications and sessions using learned embeddings.

[0082] Thus, the results show that passive monitoring of phone usage is able to infer information about a user's cognitive health.

[0083] As described above, one aspect of the present technology is the collection and use of data derived from various sources (a user's phone) to infer user states and / or to initiate device operations. The present disclosure contemplates that, in some instances, such collected data can include personal information data that uniquely identifies or can be used to identify or contact a specific person. Such personal information data can include demographic data, location-based data, phone numbers, email addresses, twitter ID's, home addresses, data or records relating to a user's health or level of fitness (e.g., vital signs measurements, medication information, exercise information), date of birth, or any other identifying or personal information.

[0084] The present disclosure recognizes that the use of such personal information data, in the present technology, can be used to the benefit of users. For instance, personal information data can be used to estimate suboptimal, impaired, and / or declining user states, and to recommend that the user seek a follow-up assessment. As another example, personal information data can be used to estimate a user’s mood or energy level, and to adjust application operations and / or recommendations accordingly. Further, other uses of personal information data are also

[0085] The present disclosure contemplates that the entities responsible for the collection, analysis, disclosure, transfer, storage, or other use of such personal information data will comply with well-established privacy policies and / or privacy practices. In particular, such entities should implement and consistently use privacy policies and practices that are generally recognized as industry or government standards. Such policies should be accessible by users and should be updated as the collection and / or use of data changes. Personal information from users should be collected for legitimate and reasonable uses of the entity and not shared or sold outside of those legitimate uses. Further, such collection / sharing should occur after receiving the informed consent of the users. Additionally, such entities should consider taking any needed steps for safeguarding and securing access to such personal information data and ensuring that others with access to the personal information data adhere to their privacy policies and procedures. Further, such entities can subject themselves to evaluation by third parties to certify their adherence to widely accepted privacy policies and practices. In addition, specific privacy

[0086] Regardless of the foregoing, the present disclosure also contemplates embodiments in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates that hardware and / or software elements can be provided to prevent or block access to such personal information data. For example, in the case of monitoring application usage, monitoring lock / unlock events, and / or triggering device operations, the present technology can be configured to allow users to selectively "opt in" or "opt out" of participation in the collection of personal information data during registration for services or anytime thereafter. In another example, users can be provided with the ability to opt-out of allowing the transmission of transmissions to other devices identifying inferred states and / or the inferred states not being stored locally or remotely. In addition to providing the "opt in" and "opt out" options, the present disclosure contemplates providing notifications relating to the access or use of personal information. For instance, a user can be notified upon download of an application that their personal information data will be accessed and then again just prior to the application accessing the personal information data.

[0087] Moreover, it is the intent of the present disclosure that personal information data should be managed and handled in a manner that minimizes risks of unintentional or unauthorized access or use. Risk can be minimized by limiting the collection of data and deleting data once it is no longer needed. In addition, and when applicable, including in certain health related applications, data de-identification can be used to protect a user's privacy. De-identification can be facilitated, when appropriate, by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or

[0088] Thus, while the present disclosure broadly covers technologies using personal information data to implement various disclosed embodiments, the present disclosure also contemplates embodiments that can be implemented without the need for accessing such personal information data. That is, the various embodiments of the present technology are not rendered inoperable due to the lack of or inaccessibility to such personal information data. For example, an inferred state can be based on temporal characteristics of application usage (e.g., how long various applications are used, when they are used, and how many applications are used in a session), on non-personal information data, or a minimal amount of personal information (e.g., without identifying which application is in use).

[0089] This specification can make reference to specific examples of user devices, such as a device. It will be understood that these examples are illustrative and not limiting; other devices can be substituted and can implement similar functional blocks and / or algorithms to perform the operations described herein and / or other operations. In some cases, the user device can be non-wearable and / or non-mobile.

[0090] Embodiments of the present application, for example, methods, apparatus, computer readable medium, etc., can be realized in any combination of various computing devices and / or components, as desired. The various processes outlined herein can be implemented on the same processor or different processors in any combination. Where a component, e.g., a processor, has been described as being configured to perform certain operations, such configuration can be accomplished, e.g., by designing the component, e.g., the processor, to perform the operations, by programming the component, e.g., the processor, to perform the operations, or any combination thereof. Further, while the embodiments described above can make reference to specific hardware components and software components, those skilled in the art will appreciate that different combinations of hardware components and / or software components can also be used, and that particular operations described as being implemented in hardware might also be implemented in software or vice versa.

[0091] Computer programs incorporating various features of the present application can be encoded and stored on various computer readable storage media. Suitable media include magnetic disk or tape, optical storage media such as compact disk (CD) or DVD (digital versatile disk), flash memory, and other non-transitory media. Computer readable media encoded with the program code can be packaged with a compatible electronic device, or the program code can be provided separately from electronic devices (e.g., via Internet download or as a separately packaged computer readable storage medium).

[0092] Therefore, although the present application has been described in relation to particular embodiments, it is to be understood that the application disclosed herein is intended to cover all modifications and equivalents of the above-described embodiments.

Claims

1. A computer-implemented method comprising: detecting, at each of a set of time points, an execution of an application at an electronic device; determining, for each detected execution, an application usage variable; generating, for each session of the electronic device, one or more aggregate metrics for the session of the electronic device based on an aggregation of at least some of the application usage variables generated during the session of the electronic device, thereby generating a plurality of aggregate metrics, wherein a session of the electronic device is a period of time between an unlock event and a subsequent lock event, and wherein each of the plurality of aggregate metrics is associated with a session identifier of a plurality of session identifiers; assigning each of the plurality of aggregate metrics to a cluster of a plurality of clusters; determining, based on a number of sessions assigned to each cluster associated with the plurality of session identifiers, a state identifier corresponding to an inferred cognitive state of a user of the electronic device; retrieving a device operation identifier associated with the state identifier; and performing a device operation associated with the device operation identifier.

2. The method of claim 1, wherein: a first subset of the set of time points is during a first usage session of the electronic device; a second subset of the set of time points is during a second usage session of the electronic device; the electronic device is in a locked state during at least a portion of a period of time between the first usage session and the second usage session; and generating the one or more aggregates comprises: generating a first aggregate metric of the one or more aggregate metrics based on an aggregation of a first subset of the application usage variables corresponding to the first usage session; and generating a second aggregate metric of the one or more aggregate metrics based on an aggregation of a second subset of the application usage variables corresponding to the first usage session.

3. The method of claim 1, wherein: the one or more aggregate metrics comprise a plurality of aggregate metrics, each of the plurality of aggregate metrics being associated with a usage session of a plurality of usage sessions; and determining the state identifier comprises: assigning each of the plurality of aggregate metrics to a cluster of a plurality of clusters; and determining the state identifier based on the cluster assignments.

4. The method of any one of claims 1-3, wherein determining the state identifier comprises using a regression technique, a component analysis technique, a distance-based technique, and / or a neural network.

5. The method of any one of claims 1-3, wherein the application usage variables are temporal characteristics of application usage.

6. The method of any one of claims 1-3, wherein the device operation comprises presenting a notification identifying the inferred cognitive state.

7. The method of any one of claims 1-3, wherein: each determined application usage variable comprises an identifier of a particular application that was executed; and generating the one or more aggregate metrics comprises: ​ ​ ​ ​ ​ generating a data object comprising a set of particular applications and indicating an order of execution of the set of particular applications; embedding the data object; and generating an aggregate metric based on the embedded data object, the aggregate metric having a reduced dimensionality compared to a dimensionality of the embedded data object.

8. An electronic device, comprising: one or more processors coupled to a user interface component and a connectivity component; and a computer-readable storage medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising: detecting, at each of a set of time points, an execution of an application at the electronic device; determining, for each detected execution, an application usage variable; for each session of the electronic device, generating one or more aggregate metrics for the session of the electronic device based on an aggregation of at least some of the application usage variables generated during the session of the electronic device, thereby generating a plurality of aggregate metrics, wherein a session of the electronic device is a period of time between an unlock event and a subsequent lock event, and wherein each aggregate metric of the plurality of aggregate metrics is associated with a session identifier of a plurality of session identifiers; assigning each aggregate metric of the plurality of aggregate metrics to a cluster of a plurality of clusters; determining, based on a number of sessions assigned to each cluster associated with the plurality of session identifiers, a state identifier corresponding to an inferred cognitive state of a user of the electronic device; retrieving a device operation identifier associated with the state identifier; and performing a device operation associated with the device operation identifier.

9. The electronic device of claim 8, wherein: a first subset of the set of time points is during a first usage session of the electronic device; a second subset of the set of time points is during a second usage session of the electronic device; the electronic device is in a locked state during at least a portion of a period of time between the first usage session and the second usage session; and generating the one or more aggregates comprises: generating a first aggregate metric of the one or more aggregate metrics based on an aggregation of a first subset of the application usage variables corresponding to the first usage session; and generating a second aggregate metric of the one or more aggregate metrics based on an aggregation of a second subset of the application usage variables corresponding to the first usage session.

10. The electronic device of claim 8, wherein: the one or more aggregate metrics comprise a plurality of aggregate metrics, each of the plurality of aggregate metrics being associated with a usage session of a plurality of usage sessions; and determining the state identifier comprises: assigning each of the plurality of aggregate metrics to a cluster of a plurality of clusters; and determining the state identifier based on the cluster assignments. ​ ​ ​ 11. The electronic device of any of claims 8-10, wherein determining the state identifier comprises using a regression technique, a component analysis technique, a distance-based technique, and / or a neural network.

12. The electronic device of any of claims 8-10, wherein the application usage variable is a temporal characteristic of application usage.

13. The electronic device of any of claims 8-10, wherein the device operation comprises presenting a notification identifying the inferred cognitive state.

14. The electronic device of any of claims 8-10, wherein: each determined application usage variable comprises an identifier of a particular application that was executed; and generating the one or more aggregate metrics comprises: generating a data object comprising a set of particular applications and indicating an order in which the set of particular applications were executed; embedding the data object; and generating an aggregate metric based on the embedded data object, the aggregate metric having a reduced dimensionality compared to a dimensionality of the embedded data object.

15. A computer program product, the computer program product being tangibly embodied in a non-transitory machine-readable storage medium, the computer program product including instructions configured to cause one or more data processors to perform actions comprising: detecting, at each point in a set of time points, an execution of an application at an electronic device; determining, for each detected execution, an application usage variable; generating, for each session of the electronic device, one or more aggregate metrics for the session of the electronic device based on an aggregation of at least some of the application usage variables generated during the session of the electronic device, thereby generating a plurality of aggregate metrics, wherein a session of the electronic device is a period of time between an unlock event and a subsequent lock event, and wherein each aggregate metric in the plurality of aggregate metrics is associated with a session identifier in a plurality of session identifiers; assigning each aggregate metric in the plurality of aggregate metrics to a cluster in a plurality of clusters; determining, based on a number of sessions assigned to each cluster associated with the plurality of session identifiers, a state identifier corresponding to an inferred cognitive state of a user of the electronic device; retrieving a device operation identifier associated with the state identifier; and performing a device operation associated with the device operation identifier.

16. The computer program product of claim 15, wherein: a first subset of the set of time points is during a first usage session of the electronic device; a second subset of the set of time points is during a second usage session of the electronic device; the electronic device is in a locked state during at least a portion of a period of time between the first usage session and the second usage session; and generating the one or more aggregates comprises: generating a first aggregate metric in the one or more aggregate metrics based on an aggregation of a first subset of the application usage variables corresponding to the first usage session; and generating a second aggregate metric of the one or more aggregate metrics based on an aggregation of a second subset of the application usage variables corresponding to the first usage session.

17. The computer program product of claim 15, wherein: the one or more aggregate metrics comprise a plurality of aggregate metrics, each of the plurality of aggregate metrics being associated with a usage session of a plurality of usage sessions; and determining the state identifier comprises: assigning each of the plurality of aggregate metrics to a cluster of a plurality of clusters; and determining the state identifier based on the cluster assignments.

18. The computer program product of any of claims 15 to 17, wherein determining the state identifier comprises using a regression technique, a component analysis technique, a distance-based technique, and / or a neural network.

19. The computer program product of any of claims 15 to 17, wherein the application usage variables are temporal characteristics of application usage.

20. The computer program product of any of claims 15 to 17, wherein the device operation comprises presenting a notification identifying the inferred cognitive state.

Images