Transitioning a computing device from a low power state based on sensor input from a pen device

The wake-up event message is generated by matching the sensor data of the pen device with the posture pattern, which solves the problem of high interactive power consumption of computing devices in the low power state, and realizes efficient wake-up and operational efficiency of computing devices.

CN112673333BActive Publication Date: 2024-11-05MICROSOFT TECHNOLOGY LICENSING LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201980058922.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-09-09
Filing Date
2019-06-25
Publication Date
2024-11-05
Estimated Expiration
2039-06-25

AI Technical Summary

Technical Problem

The interaction of existing computing devices with the pen device in a low power state requires a large amount of power, resulting in a reduced power efficiency of the computing device in a low power state, and it is difficult for the pen device to wake up effectively in a low power state.

Method used

The pen device obtains posture input data from the sensor, compares it with the stored posture mode, generates a wake-up event message, and uses the communication protocol supported by the computing device in a low power state to transmit a wake-up event message to the computing device, so that the computing device can get from low power The state changes to active state.

Benefits of technology

It realizes efficient and flexible wake-up of computing devices, reduces power consumption of computing devices and pen devices, improves operational efficiency, provides intuitive user interaction methods, and allows users to customize wake-up posture modes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112673333B_ABST
    Figure CN112673333B_ABST
Patent Text Reader

Abstract

The disclosure herein describes using a pen device to transition a computing device from a low power state. The pen device obtains gesture input data from at least one sensor of the pen device, and the obtained gesture input data is compared to at least one gesture pattern. Based on the gesture input data matching the at least one gesture pattern, the pen device generates a wake event message. The pen device transmits the wake event message to the computing device using a communication protocol that the computing device is configured to receive in a low power state. The described power state transition method provides a user with a flexible, customizable way to wake up a computing device using a pen device.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] background

[0002] Many modern computing devices include an associated pen device or stylus that is configured for use with the computing device. For example, a pen device can be used to write and / or draw on a touch screen interface of a computing device, thereby simulating writing or drawing on paper with a pen, pencil, or other writing instrument. The pen device can also be equipped with various sensors (e.g., pressure sensors, inertial measurement units (IMUs), buttons, switches, microphones, etc.) that can be used to collect data to enable interaction with a computing device (e.g., sensors or other components for interacting with a digitizer component of a computing device (such as proximity sensing)).

[0003] However, many interactions with a pen device, such as pen detection by a digitizer of the computing device, require a significant amount of power compared to the power budget of the computing device in a low power state. For example, low latency detection of movement and proximity of a pen device by a digitizer of the computing device consumes a significant amount of the power budget of the computing device in a low power state. Thus, operations enabled by the pen device are not suitable for use with the computing device in a low power state. For example, triggering a transition of the computing device from a low power state to an active state based on low latency detection of pen device activity by the computing device results in consistent, relatively high power consumption by the computing device, thereby reducing the power efficiency of the low power state of the computing device.

[0004] Overview

[0005] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0006] A computerized method and system for transitioning a computing device from a low power state using a pen device is described. The pen device obtains gesture input data from at least one sensor of the pen device, and the obtained gesture input data is compared to at least one gesture pattern. Based on the gesture input data matching the at least one gesture pattern, the pen device generates a wake event message. The pen device then transmits the wake event message to the computing device via a network interface of the pen device, wherein the wake event message is transmitted using a communication protocol that the computing device is configured to receive in the low power state.

[0007] Many of the accompanying features will become more readily appreciated as the same become better understood by reference to the following detailed description considered in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present specification will be better understood from the following detailed description when read in conjunction with the accompanying drawings, in which:

[0010] Figure 1 is an exemplary block diagram illustrating a system configured to transition a computing device out of a low power state based on gesture input from a pen device according to an embodiment;

[0011] Figure 2A and 2B is an exemplary block diagram illustrating a method for a pen device to obtain gesture input data according to a corresponding embodiment;

[0012] Figure 3A and 3B is an exemplary flow chart illustrating a pen device being used to generate a wake-up event message and transmit the wake-up event message to a computing device to cause the computing device to transition out of a low power state and / or to detect an uplink signal from the computing device and send a signal to change an operating mode of the computing device according to an embodiment;

[0013] Figure 4 is an exemplary flow chart illustrating a computing device receiving a wake event message and transitioning out of a low power state when the wake event message is validated according to an embodiment;

[0014] Figure 5 is an exemplary sequence diagram illustrating interaction between a pen device and a computing device including adjusting a wake-up gesture mode based on user feedback according to an embodiment; and

[0015] Figure 6 A computing device according to an embodiment is illustrated as a functional block diagram.

[0016] Corresponding reference numerals indicate corresponding parts throughout the drawings. Figures 1 to 6 In the drawings, the various systems are illustrated as schematic diagrams. The drawings may not be drawn to scale.

[0017] Detailed Description

[0018] Aspects of the present disclosure provide a system and method for transitioning a computing device from a low-power state based on sensor data from a pen device. The pen device obtains gesture input data from an associated sensor. The gesture input data is compared to a gesture pattern stored on the pen device, and when the gesture input data matches the gesture pattern, a wake-up event message is generated. The pen device transmits the wake-up event message to the computing device using a network connection / protocol that the computing device is configured to use when in a low-power state. When the computing device receives the wake-up event message, it can verify that the message is accurate based on a calculated confidence value. When the message is verified, the computing device transitions from a low-power state to an active state or other similar state.

[0019] Additionally or alternatively, the computing device and the pen device may be configured to cause the computing device to change an operating mode of the computing device based on an interaction with the pen device. The pen device may be configured to wake up based on detecting a wake-up gesture pattern from a user, and upon waking up, the pen device may detect an uplink signal from the computing device via an electrostatic communication channel. In response to the uplink signal, the pen device may send a signal to the computing device via a network interface or a communication channel other than the electrostatic communication channel to change the operating mode of the computing device. The computing device may receive the signal for changing the operating mode, and in response, perform an operation based on the received signal to change the operating mode. For example, the computing device and the pen device may begin an initial pairing process.

[0020] The described methods and systems provide a user with an efficient and flexible way to "wake up" a computing device, transition the computing device to an active state, and / or otherwise change the operation of the computing device, including using intuitive, user-friendly movements of a pen device and other interactions with the pen device and / or electrostatic communication between the computing device and the pen device. In addition, users are allowed to create and customize wake-up gesture patterns for use with their computing devices, and such wake-up gesture patterns can also be used to trigger quick access to suitable applications when the computing device transitions from a low-power state (e.g., an ink-based note-taking application opens immediately when the computing device transitions to an active state in response to a gesture from the pen (which includes a gesture simulating writing on a surface), etc.). The described system operates in an unconventional manner to utilize sensors of the pen device and a link between the pen device and the computing device to provide an intuitive method for waking a computing device in a low-power state and / or otherwise changing the operating mode of the computing device. In this way, the present disclosure reduces memory and power consumption of a computing device, thereby improving the operation of the computing device and solving technical problems. Furthermore, the present disclosure at least reduces the power consumption of the pen device because the pen device does not have to transfer and / or exit its low power state until the pen device is confident of the wake-up process, thereby improving the operation of the pen device and solving the technical problem.

[0021] Figure 1is an exemplary block diagram illustrating a system 100 configured to transition a computing device 104 out of a low power state based on a gesture input 108 from a pen device 102, according to an embodiment. The system 100 includes a computing device 104 and a pen device 102 (illustrated as a pen and nested boxes representing components of the pen device 102). In some examples, the pen device 102 is linked to the computing device 104 and is configured to work only or primarily with the computing device 104 (e.g., a pen device or stylus included with a tablet device, a tablet phone device, or a laptop device, etc.). In addition, the computing device 104 and the pen device 102 are configured to perform the operations described herein as well as other conventional computer operations (e.g., receiving user input, displaying output, executing applications, etc.).

[0022] In some examples, the pen device 102 or stylus is a pen-shaped device that includes a network interface 105 that is configured to enable the pen device 102 to communicate with the computing device 104 and / or other computing devices (e.g., WI-FI communication, Bluetooth communication, near field communication (NFC), etc.) and other computer components (such as processors, memory, and input / output (I / O) components). For example, the pen device 102 includes a sensor 106 that is configured to collect sensor data indicating the interaction of the pen device 102 with a user, other devices, or other objects. The sensors 106 may include an inertial measurement unit (IMU) configured to detect movement and / or impact of the pen device 102, one or more pressure sensors configured to detect contact with the pen device 102 (e.g., a pressure sensor in the tip of the pen device 102 for detecting when it is in contact with a surface for collecting writing data, a pressure sensor in the shaft of the pen device 102 for detecting when and / or how the pen device 102 is being held by a user, etc.), a button sensor, a switch sensor, a microphone configured to collect audio data near the pen device 102, and the like.

[0023] In some examples, data collected by the pen device 102 from the sensor 106 is obtained as gesture input data 108, which is input data used to identify when a user makes a wake-up gesture using the pen device 102. The gesture input data 108 may include sensor data from any and / or all sensors 106. The data obtained as the gesture input data 108 may be based on the type of data that constitutes a wake-up gesture pattern 110, which is a stored and / or registered data pattern configured to indicate sensor data obtained when a wake-up gesture is performed with the pen device 102. For example, the wake-up gesture pattern 110 may include two shocks detected from a pressure sensor on the tip of the pen device 102, with a defined time interval between the two shocks. Such a wake-up gesture pattern 110 may include a series of pressure sensor data points over a time period, wherein relatively high data point values ​​at the beginning and end of the series represent the two shocks. Alternatively, the pressure sensor may be configured to provide a binary output such that the sensor is either activated or not activated. In this case, the wake-up gesture pattern 110 may require detection of two pressure sensor activations in succession, and optionally, the wake-up gesture pattern 110 may include a defined time interval between the two activations.

[0024] In some examples, the pen device 102 and / or computing device 104 in combination with the pen device 102 may be configured to detect other gesture types based on sensors of the pen device 102 and / or computing device 104. For example, a gesture may be associated with "writing" with the pen device 102 (e.g., causing the computing device 104 to receive writing or drawing data from the motion and / or contact of the pen device 102, etc.), "erasing" with the pen device 102 (e.g., causing writing or drawing data to be erased in response to an associated "erase" gesture, etc.), activation of an application, navigation in interfaces, etc. Such gestures may also be represented by gesture patterns as described herein and / or these gestures may be detected according to any gesture detection process as would be understood by one of ordinary skill in the art. In addition, some or all other gestures that the described system is configured to detect may be considered wake-up gestures, such that when one or more of these other gestures are detected (whether by matching to a gesture pattern or by another gesture detection method), a wake-up event may be triggered in the same manner as when the gesture input data 108 matches the wake-up gesture pattern 110 as described herein.

[0025] The pen device 102 is configured to compare the obtained gesture input data 108 to the wake-up gesture pattern 110 to determine when the user intends to perform a wake-up event on the computing device 104 (e.g., to cause the computing device 104 to transition out of a low-power state or mode (sleep state, suspend state, standby state, etc.), etc.). If the gesture input data 108 matches the wake-up gesture pattern 110, the pen device 102 is configured to generate a wake-up event message 113 using the wake-up message generator 112. In some examples, the wake-up gesture pattern 110 may include a defined acceptable margin of error for the purpose of matching to the obtained gesture input data 108. For example, if the pattern 110 requires that the received sensor data points have a time period interval of 1 second, the pattern 110 may include a 20% margin of error so that received data points with a time period interval of 0.8 seconds or 1.2 seconds are considered to match the pattern 110. Such an error margin may be defined by default for the mode 110 (e.g., a default error margin of 10% may be defined for each requirement of the mode 110, etc.) and / or may be defined by a user when creating or editing the mode 110. Alternatively or additionally, the error margin of the mode 110 may be adjusted based on received feedback and / or machine learning, as described herein. It should be understood that the error margin may be applied to time-based mode requirements, data value-based mode requirements, or other types of mode requirements without departing from the description herein.

[0026] Figure 2A and 2B is an exemplary block diagram illustrating a method for a pen device to obtain gesture input data according to a corresponding embodiment. Figure 2A In the embodiment of the present invention, the pen device 202 is configured to include a pressure sensor 232 in the tip of the pen device 202. The gesture input data 108 can be collected based on the pressure sensor 232 being tapped, pressed, or otherwise placed in contact with the surface 234 (e.g., the surface of a computing device, a table, etc.). In some examples, the gesture input data 108 includes sequential taps on the pressure sensor 232, as illustrated at 236. The wake-up gesture pattern 110 can include a pattern that includes pressure input from the pressure sensor 232 occurring twice in sequence, with a defined time period between the inputs (e.g., between .5 seconds and 1.5 seconds, etc.). If the two taps of the pen device 202 at 236 match the pattern (e.g., the taps are sufficient to provide pressure input from the sensor 232, and the taps are timed to match the defined time period between the pressure inputs of the pattern, etc.), the pen device 202 will generate a wake-up event message 113, as described below.

[0027] Alternatively or additionally, in Figure 2B, the pen device 202 is configured to include a button 238 and an inertial measurement unit (IMU) 240. The gesture input data 108 may include activation data points from the button 238 and / or motion data from the IMU 240. The wake-up gesture pattern 110 may include a combination of activation data from the button 238 and motion data from the IMU 240. For example, the pattern may require the user to press and / or hold the button 238, and then move the pen device 202 in one or more motions after the press (e.g., within a defined time period after the button 238 is pressed, such as within 1 second, within 2 seconds, etc.) and / or while holding the button 238, as illustrated at 242. As a result, inadvertent movement of the pen device 202 is less likely to accidentally cause a wake-up event message to be generated because the button 238 must also be pressed or held. The motion data from the IMU 240 required for this pattern may include one or more movements in a particular direction, simply moving the pen device 202 back and forth or shaking it, or other motion patterns detectable by the IMU 240. It should be understood that the required sensor data for matching to the wake gesture pattern 110 may include more, fewer, and / or different sensor data points without departing from the description herein.

[0028] return Figure 1 , the wake-up message generator 112 is a software component of the pen device 102 that is configured to take gesture input data 108 as input and generate a wake-up event message 113 as output, as described herein. The generation of the wake-up event message 113 may include transforming some or all of the gesture input data 108 into a format that is compatible with a defined network communication protocol associated with the computing device 104 and the pen device 102. Alternatively or additionally, the wake-up event message 113 may include a wake-up event code that is defined to indicate to the computing device 104 that a wake-up gesture has been performed and / or identify a wake-up gesture pattern 110 associated with the wake-up event message 113.

[0029] The computing device 104 includes a display 114 and a memory 122, the display 114 further including a display panel 116 subcomponent configured to display output of the computing device 104 to a user and a digitizer 118 subcomponent configured to detect contact and / or proximity of a pen device 102 or other object to a surface of the display 114, etc., a network interface 120, a sensor 121. The computing device 104 is further configured to operate in a plurality of power states, the plurality of power states including at least: a low power state (e.g., a sleep state, a suspended state, etc.), during which some subcomponents and / or processes of the computing device 104 are disabled or otherwise configured to consume less power; and an active state (e.g., a wake state, a default state, etc.), during which these subcomponents and / or processes operate according to default behavior and potentially consume higher power levels than when in a low power state. For example, when the computing device is in a low power state, the display 114 can be disabled (including the digitizer 118), and the network interface 120 can be configured to operate using only a subset of potential communication protocols that require less power (e.g., WI-FI can be disabled while Bluetooth remains enabled, etc.). Other components (such as other user interfaces) may also be disabled. In addition, subcomponents and / or processes that remain active in the low power state may be configured to operate with higher latency (e.g., operate less frequently, such as once per second, rather than once every 50 milliseconds, etc.) to reduce the power consumed. The method of transitioning the computing device 104 out of the low power state or waking up the computing device 104 utilizes the processes of the computing device 104 that are still active in the low power state (e.g., if the digitizer 118 is completely disabled in the low power state, a tap on the surface of the display 114 cannot be detected, and therefore the computing device 104 cannot be woken up from the low power state, etc.).

[0030] The network interface 120 of the computing device 104 may include hardware, firmware, and / or software configured to enable the computing device 104 to communicate with other devices, including the pen device 102. The network interface 120 may be configured to communicate via one or more wireless communication protocols and / or radio communication protocols (e.g., WI-FI, Bluetooth, near field communication (NFC), cellular communication protocols, etc.), one or more wired communication protocols (e.g., Ethernet, etc.), etc. As mentioned above, some or all capabilities of the network interface 120 may be disabled or reduced when the computing device is in a low power state.

[0031] Memory 122 is configured to include instructions and / or data associated with software applications executed on the computing device, as will be understood by one of ordinary skill in the art. In some examples, the applications and data in memory 122 include a wake-up message validator 124, a power state manager 128, and a wake-up event manager 130. When computing device 104 is in a low-power state and it receives a wake-up event message 113, computing device 104 may use components 124-130 to transition from the low-power state to an active power state or other power state.

[0032] The wake-up message validator 124 is a software application configured to receive the wake-up event message 113 as input and verify that it is the result of a user attempting to wake up the computing device 104. The wake-up message validator 124 may be further configured to use sensor data from sensors 121 (e.g., internal IMU, touch sensor, microphone or other audio sensor, etc.) of the computing device 104 as input in addition to the wake-up event message 113. In some examples, the wake-up gesture pattern 110 may be relatively simple so that the user may inadvertently trigger the wake-up event message 113 while carrying the pen device 102, playing with the pen device 102, bumping into the pen device 102, etc. The wake-up message validator 124 is configured to calculate a confidence value 125 and compare the confidence value 125 to a confidence threshold 126. If confidence value 125 exceeds or otherwise transcends confidence threshold 126, received wake event message 113 is deemed to indicate a legitimate wake event, and power state manager 128 is used to transition computing device 104 from a low power state to an active power state or other power state. The calculation of confidence value 125 and the comparison with confidence threshold 126 are described in more detail below.

[0033] In some examples, the computing device 104 includes a wake-up event manager 130 that is configured to enable creation of a wake-up gesture pattern 110 for use with the pen device 102, and to enable adjustment of the confidence threshold 126 and / or the wake-up gesture pattern 110 to improve the accuracy and efficiency of the wake-up operation described herein based on user feedback obtained after the wake-up event is processed. For example, the wake-up event manager 130 can be configured to provide a user with an interface through which the user can request to create a new wake-up gesture pattern 110. The computing device 104 and the pen device 102 can then be transformed into a wake-up gesture pattern learning state so that the user can input the desired wake-up gesture pattern into the pen device 102 (e.g., pressing a button, activating a pressure sensor, moving and / or tapping the pen device 102, etc.). The wake-up event manager 130 obtains the new gesture input data and defines a new wake-up gesture pattern based thereon. In addition to storing the new wake-up gesture pattern on the computing device, the wake-up event manager 130 also uploads the new wake-up gesture pattern to the pen device 102 for use in detecting future wake-up events.

[0034] In some examples, the wake-up event manager 130 is further configured to include a machine learning component configured to adjust the wake-up gesture pattern 110, the confidence threshold 126, and / or the method of calculating the confidence value 125 based on the user input associated with the wake-up event as feedback data. For example, the wake-up event manager 130 can analyze the pattern of intentional wake-up events, unintentional wake-up events, failed wake-up event attempts (e.g., when the user attempts to trigger a wake-up event and the pen device 102 and / or the computing device 104 fail to perform the wake-up event process, etc.) to tune the wake-up gesture pattern 110, the confidence threshold 126, and / or the method of calculating the confidence value 125 based on the pen device 102 sensor input to improve the efficiency of identifying and triggering wake-up events (e.g., adjusting the wake-up gesture pattern to reduce unintentional wake-up events and / or increase the accuracy of identifying intentional wake-up events, etc.). In addition, the wake-up event manager 130 can be configured to recommend adjustments to various aspects of the system 100 based on the user response to the wake-up event as feedback data. For example, the wake event manager 130 may identify wake gesture patterns 110 that are associated with a high frequency of unintentional wake events and / or wake gesture patterns 110 that cause the user to attempt to trigger a wake event multiple times in a row before the wake event is actually triggered (e.g., the wake gesture pattern 110 may have an error margin that is too narrow, or may be difficult to reproduce, etc.). Any and / or all of these aspects may cause the wake gesture pattern 110 and / or confidence threshold 126 to be flagged by the wake event manager 130 as potentially resulting in inefficient and / or inaccurate wake event performance. The wake event manager 130 may further notify the user of the flagged wake event gesture pattern 110 and encourage the user to consider changes and / or improvements to the flagged wake event gesture pattern 110 (e.g., the wake event manager 130 may send an automatic notification message to the user on the computing device 104, etc.).

[0035] In some examples, the machine learning module of the wake-up event manager 130 includes a trained regressor, such as a random decision forest, a directed acyclic graph, a support vector machine, a neural network, or other trained regressor. The trained regressor can be trained using the feedback data described above. Examples of trained regressors include convolutional neural networks and random decision forests. It should be further understood that in some examples, the machine learning module can operate according to machine learning principles and / or techniques known in the art without departing from the systems and / or methods described herein.

[0036] Alternatively or additionally, the machine learning module of the wake event manager 130 may also include a classifier as will be understood by one of ordinary skill in the art of machine learning technology.

[0037] In one example, the machine learning module of the wake-up event manager 130 can utilize training data pairs when applying machine learning techniques and / or algorithms. Millions of training data pairs (or more training data pairs) can be stored in a machine learning data structure. In some examples, the training data pairs include feedback data values ​​based on timestamps paired with value adjustment values. The pairing of these two values ​​demonstrates the relationship between feedback data values ​​and adjustment values ​​that can be used by the machine learning module to determine future value adjustments based on machine learning techniques and / or algorithms. The data used for training can be data collected from a single user or data collected from multiple users. The machine learning algorithm of the machine learning module can be applied to the data immediately, or it can be applied to an "offline" data set at defined intervals or based on other execution trigger conditions, so that the adjusted or tuned mode or other affected aspects of the system are updated in the system at a later time relative to the occurrence of the associated wake-up event.

[0038] It should be understood that in some examples, a match of gesture input data 108 to wake gesture pattern 110 is deemed sufficient to indicate a wake event, and computing device 104 does not perform any additional wake event validation (e.g., generating confidence value 125 and comparing value 125 to confidence threshold 126, etc.). Alternatively, pen device 102 may perform little or no gesture input data validation (e.g., comparing input data 108 to wake gesture pattern 110, etc.) and / or wake gesture pattern 110 may be configured to be relatively broad (e.g., coarse detection and / or validation decisions regarding wake events, etc.), such that most or all validation of wake events is performed at computing device 104 by wake message validator 124 (e.g., fine or more precise detection and / or validation decisions regarding wake events, etc.), as described herein. In a further example, the pen device 102 is configured to stream raw sensor data, compressed sensor data, and / or other gesture input data to the computing device 104 continuously or at short regular intervals, and the computing device 104 is configured to perform detection and / or verification of wake events based on the streamed data. However, such an implementation may result in undesirable power consumption levels of the pen device 102 and / or the computing device 104.

[0039] Figure 3A and 3B is an exemplary flow chart illustrating a pen device being used to generate a wake-up event message and transmit the wake-up event message to a computing device to cause the computing device to transition out of a low power state and / or detect an uplink signal from the computing device and send a signal to change an operating mode of the computing device according to an embodiment. Figure 3A300 is an exemplary flowchart illustrating a method in which a pen device (e.g., pen device 102, etc.) is used to generate a wake-up event message (e.g., wake-up event message 113, etc.) and transmit the wake-up event message to a computing device (e.g., computing device 104, etc.) to cause the computing device to transition out of a low-power state and into a non-low-power state (e.g., a high-power state or a higher-power state) according to an embodiment. At 302, the pen device obtains gesture input data via at least one sensor. Obtaining gesture input data can be performed continuously and / or at consistent intervals as long as the pen device is powered on and / or active. Alternatively or additionally, the pen device can be configured to activate and / or deactivate the acquisition of gesture input data depending on the state of the computing device. For example, upon entering a low-power state, the computing device can be configured to send a message to the pen device that causes the pen device to activate gesture input capture for the purpose of detecting a wake-up event, and upon entering another state from the low-power state, the computing device can be configured to send a message to the pen device that causes the pen device to deactivate gesture input capture for the purpose of detecting a wake-up event. Furthermore, such activation and / or deactivation may be based on defined time intervals (e.g., the computing device may disable gesture input capture for wake events by the pen device for a defined time period (e.g., 1 minute, 5 minutes, etc.) after transitioning out of a low-power state, etc.).

[0040] As described above, obtaining gesture input data may include obtaining data points from a plurality of sensors of the pen device (e.g., pressure sensors, IMUs, buttons, switches, capacitive sensors, microphones, etc.). In some examples, gesture input data capture may be limited to data from sensors that are included in a current set of wake-up gesture patterns (e.g., wake-up gesture patterns 110, etc.). For example, if none of the wake-up gesture patterns include data from a particular button on the pen device, obtaining gesture input data may not include collecting input data from the particular button because such input data is not useful for comparison with the wake-up gesture pattern.

[0041] At 304, the obtained gesture input data is compared to at least one stored wake-up gesture pattern. The pen device may include one or more wake-up gesture patterns that trigger a wake-up event as described herein. Comparing the obtained gesture input data to these patterns may include comparing specific data points of the obtained gesture input data to data points in these patterns, including accounting for any acceptable error margins that may be included in these patterns. In addition, the timing of the data points of the gesture input data may be compared to the timing requirements of these patterns in a similar manner, as described above.

[0042] At 306, if the gesture input data matches at least one wake-up gesture pattern, the process proceeds to 308. Alternatively, if the gesture input data does not match at least one wake-up gesture pattern, the process returns to 302 to obtain any additional gesture input data from the sensor of the pen device.

[0043] At 308, the pen device generates a wake event message. The wake event message may include an indicator and / or a code indicating that a wake event has been triggered. In addition, the wake event message may be generated to include some or all gesture input data (e.g., raw sensor data and / or gesture input data that may have been processed from the raw sensor data, etc.). Including gesture input data on the wake event message may enable the computing device to perform separate analysis and / or verification of the wake event as described herein with respect to calculating a confidence value for the wake event.

[0044] At 310, a wake-up event message is transmitted to the computing device via a communication protocol. The pen device is configured to transmit the wake-up event message using a communication protocol that the computing device can receive while in a low-power state. In some examples, the wake-up event message is transmitted to the computing device via a Bluetooth connection between the pen device and the computing device. The transmission of the wake-up event message can be timed based on a defined latency setting of the pen device and / or the computing device (e.g., the computing device can be configured to receive wake-up event messages only with a high latency, such as a one-second latency (receiving message(s) once per second), in order to maintain low power consumption of a low-power state of the computing device, etc.).

[0045] In some examples, operations 304 and 306 are performed on the computing device. The gesture input data is sent to the computing device as part of the wake-up event message, and the computing device compares the gesture input data to the wake-up gesture pattern stored on the computing device while the computing device remains in the low-power state. If the gesture input data matches the wake-up gesture pattern, the computing device can perform operations to transition from the low-power state to an active state or other non-low-power state.

[0046] Figure 3B 301 is an exemplary flow chart illustrating a method in which a pen device (e.g., pen device 102, etc.) wakes up from a low power state based on a detected gesture, detects the presence of a computing device (e.g., computing device 104, etc.) via an electrostatic communication channel, and sends a signal to the computing device via the same or another communication channel to change an operating mode. Operations 302-306 are similar to those described above with respect to Figure 3A Basically the same as described.

[0047] At 308, the pen device wakes up from the low power state based on the gesture input data matching at least one wake-up gesture pattern. In some examples, the pen device is configured to detect gesture input data despite being in the low power state, thereby implementing the described functionality. Waking up from the low power state by the pen device can include activating a network communication capability and / or an electrostatic signal detection and communication capability for use in the remainder of flowchart 301. Such capabilities can include, for example, Bluetooth communication capabilities and / or an electrostatic detection capability for detecting electrostatic signals from a digitizer of a computing device.

[0048] At 310, the pen device detects an uplink signal from the computing device via an electrostatic communication channel, such as when the computing device is unable to detect an electrostatic downlink signal from the pen device or when the computing device is capable of detecting a downlink signal but is not required to detect such a signal. In some examples, a pen device proximate to a computing device is capable of communicating with one or more digitizers of the computing device via electrostatic communication. The timing of the pen device and the computing device may be asynchronous (e.g., the timing of the pen device is not initially synchronized with the timing of the digitizer, etc.) or synchronous (e.g., the timing of the pen device is synchronized with the timing of the digitizer, etc.). In embodiments where the timing is initially asynchronous, the digitizer of the computing device may be configured to search for pen signals, and when such electrostatic signals are received from the pen device, the digitizer may be configured to change its timing to synchronize with the pen device, such that the digitizer is enabled to capture all signals from the pen device and send pen reports to the operating system and / or applications.

[0049] In some examples, the range at which the digitizer of the computing device can detect the pen device via electrostatic communication is different from the range at which the pen device can detect the digitizer of the computing device. For example, the range for the digitizer to detect the pen device (e.g., via a downlink signal from the pen device, etc.) may be about 15 mm from the screen of the computing device, while the range for the pen device to detect the digitizer (e.g., via an uplink signal from the digitizer, etc.) may be about 30-40 mm from the screen of the computing device. The effective range of the downlink signal and the uplink signal may be affected by the signal-to-noise ratio (SNR) of the signal and / or the amount of power used to generate the signal (e.g., the digitizer may use more power to generate the uplink signal than the power used by the pen device to generate the downlink signal, etc.). At 310, detection of the uplink signal by the pen device may occur when the pen device is within the range of detecting the uplink signal from the digitizer of the computing device, but is still outside the range of the digitizer of the computing device to detect the downlink signal from the pen device. In some examples, detection of the digitizer by the pen device within such an extended range may be used to more efficiently respond to potential interactions between the pen device and the computing device as described herein. It should be understood that in some examples, the computing device may be physically capable of detecting a downlink signal from a pen device, but is configured to attempt to detect a downlink signal in order to save power, processing resources, or for other reasons without departing from the description herein.

[0050] In some examples, the uplink signal detected by the pen device may include timing information associated with the digitizer of the computing device, thereby enabling the pen device to synchronize with the digitizer and / or the computing device upon detection of the uplink signal. In addition, the pen device may change its operating mode based on detection of the uplink signal (e.g., by increasing the strength of the transmitted downlink signal, increasing the transmission duration or frequency, etc.) to enable faster acquisition by the digitizer of the computing device. The uplink signal from the digitizer may be configured in such a way that the pen device calculates the position of the pen device relative to the digitizer and / or the "hover height" of the pen device above the digitizer. Such calculations may be based on the uplink signal strength, data provided in the uplink signal, and / or other aspects of the uplink signal. The calculated position may be communicated to the computing device, and changes to the operating mode of the computing device as described below may depend on the calculated position (e.g., when the pen device is within a defined distance from the digitizer, the UI of the computing device may change to become pen-friendly, etc.).

[0051] Alternatively or additionally, before detecting the electrostatic uplink signal, the pen device may communicate with the computing device via another communication channel (e.g., a Bluetooth channel, or other channel with a longer range than electrostatic, etc.) to request the computing device to transmit an uplink signal (if it has not already done so). Such a request may be sent based on the pen device itself detecting a gesture input to the pen device and / or based on another type of input to the pen device.

[0052] At 312, the pen device sends a signal via a communication channel other than the electrostatic communication channel to change the operating mode of the computing device after detecting the uplink signal. The communication channel can include a connection through a network interface, such as via a Bluetooth protocol as described herein. Other possible communication channels may be used without departing from this description, such as a low-power communication channel with a longer range than the electrostatic channel. In some examples, the signal from the pen device may include a pen identifier of the pen device and / or timing information for the computing device to synchronize with the signal of the pen device.

[0053] Changes to the operating mode of the computing device may include changes to the power state of the computing device (e.g., transitioning the computing device from a low-power state to an active state, etc. as described herein), changes to how a pen device is detected and / or how a pen device is signaled, and / or changes to interfaces, operating system operations, and / or application operations with respect to the pen device based on the detected proximity of the pen device. For example, the operation of the computing device may include changing the interface of a display screen to accommodate a user writing or drawing with a pen device and / or configuring a digitizer and / or other components of the computing device to capture ink (e.g., writing and / or drawing, etc.) input from use of the pen device. In some examples, because the range at which the pen device detects an uplink signal is greater than the range at which the digitizer detects a downlink signal, the computing device has more time to transition operations to capture pen device ink input, making it more likely that the computing device will capture all writing or drawing input rather than potentially failing to capture initial ink input based on waiting for detection of a downlink signal from the pen device. Other changes to the operating mode may include display of a pen-specific menu or other pen-related user interface components, waking the computing device or otherwise changing the power state of the computing device, launching or resuming a pen-related application, and the like.

[0054] Alternatively or additionally, the change to the operating mode of the computing device may include pairing or otherwise configuring the computing device and / or the pen device for further communication and / or interaction (e.g., Bluetooth pairing to enable further communication via a Bluetooth connection, etc.). Such pairing may be performed by the computing device and / or the pen device as an initial pairing process when the pen device and the computing device have not been previously paired or the associated pairing history data has been erased, etc. Alternatively, when the computing device and the pen device have been previously paired, the change to the operating mode of the computing device may include restoring the pairing state of the computing device and the pen device.

[0055] In some examples, the process described by 310-312 may be performed independently of the process described by 302-308, as long as the computing device and the pen device are capable of communicating via electrostatic signals and another communication channel as described. For example, if the computing device and the pen device have previously interacted to exchange gesture data, the process described by 310-312 may be performed later using (all) electrostatic signals and one or more previously established communication channels. In some embodiments, the pen device may not be configured to actively scan for electrostatic uplink signals until gestures and / or other inputs are detected by the pen device. For example, the pen device may be configured to remain in an inactive state (e.g., not scanning for uplink signals, reducing signal transmission frequency, etc.) to save battery power until gesture input is detected. After detecting the gesture, the pen device can transition to an active state and detect an electrostatic uplink signal as described herein. In this way, the present disclosure utilizes different communication paths between the pen device and the computing device to obtain benefits such as fast acquisition times and inaccurate in-range reporting. The first acquisition time between the pen device and the computing device determines and affects the pen device performance. For example, if the acquisition time is long, and the pen device is acquired only after it contacts the screen of the computing device, the effect may be to lose the first stroke of input from the pen device, resulting in a poor user experience (e.g., degraded handwriting or drawing quality in an inking application) or a degraded navigation experience through one or more applications (e.g., if the pen device is used for such navigation). Additionally, knowing that the pen device is in range (e.g., even if the exact location of the pen device is unknown) is advantageous and facilitates certain features on the computing device. For example, a pen-specific menu is displayed in anticipation that the pen device is to be used.

[0056] Figure 44 is an exemplary flowchart 400 illustrating a computing device (e.g., computing device 104, etc.) receiving a wake-up event message (e.g., wake-up event message 113, etc.) according to an embodiment and transitioning out of a low-power state when the wake-up event message is verified. The process of flowchart 400 may be performed by a computing device that is currently in a low-power state (e.g., a "sleep" state, a "suspend" state, etc.). The following verification process may enable the computing device to make a more accurate verification of the wake-up event, due to the computing device having access to more and / or different data, greater processing resources and capabilities, and / or more complex algorithms than a pen device. If the computing device is already in a state other than a low-power state (e.g., an active state, a default state, etc.), the computing device may be configured to ignore the received wake-up event message or otherwise be configured not to receive the wake-up event message (e.g., using an associated radio network connection or an electrostatic uplink to "time out" a gesture detection mechanism, etc.). In addition, as described above, the computing device may be configured to disable the generation of a wake-up event by the pen device when the computing device is not in a low-power state. At 402 , the computing device receives a wake event message from a pen device via a communication protocol usable by the computing device in a low-power state (eg, via a Bluetooth connection, etc.).

[0057] At 404, the computing device generates a confidence value for the wake event message. The confidence value can be a continuous number that is calculated by a regression or soft decision engine based on factors associated with data included in the wake event message (e.g., gesture input data, such as raw or compressed sensor data, etc.), sensor data collected by the computing device (e.g., from sensor 121, etc.), and / or context data associated with the wake event message (e.g., when the message is received, results of other messages that have been recently received, etc.). Each factor used to calculate the confidence value can include a defined weight indicating the degree to which the factor affects the calculated confidence value. For example, a factor with a weight of 0.1 may have less impact on the confidence value than a factor with a weight of 0.5 (e.g., the weight value can be applied to the associated factor value by multiplication, and the resulting weighted factor value can be combined by summing, etc., etc.).

[0058] In some examples, the confidence value is calculated based on the degree to which the input data from the wake-up event message and / or the sensor data from the sensor of the computing device (e.g., sensor 121, etc.) matches the wake-up gesture pattern stored on the computing device (e.g., these wake-up gesture patterns may be the same as the wake-up gesture patterns stored on the pen device, they may include additional or different patterns based on the sensor data of the computing device, or they may include patterns based on a combination or fusion of data from the pen device and the sensor data of the computing device, etc.). The degree of difference between the input data and the wake-up gesture pattern may inversely affect the confidence value (e.g., if the input data differs from the wake-up gesture pattern by 10% or less, a factor value of 10 may be included in the confidence value, and if the input data differs from the wake-up gesture pattern by 30% or more, a factor value of 3 may be included in the confidence value, etc.). The factor values ​​included in the confidence value may be configured as a range of possible values ​​(e.g., factor values ​​from 0-10, etc.), binary values ​​(e.g., factor value 0 or factor value 10, etc.), and / or any other combination of value types without departing from the description herein.

[0059] In addition, the confidence value can be calculated based on other contextual data associated with the received wake event message. For example, if the wake event message is received within a time frame when the computing device is typically in use by a user, the confidence value may be higher than if the message is received within a time frame when the computing device is typically maintained in a low power state for an extended period of time (e.g., the computing device is in a "sleep" state overnight, etc.). In another example, if the computing device has recently received consistent false wake event messages (e.g., the pen device was carried while the user was walking, resulting in motion that could cause false wake event messages to be generated, etc.), the confidence value may be lower than if the message was received when the pen device was otherwise relatively stationary.

[0060] At 406, the computing device verifies that the confidence value exceeds a defined confidence threshold to generate a hard decision as to whether the wake event is true or false. The confidence threshold can be a single value defined in the context of a method for calculating confidence values ​​as described above (e.g., if confidence values ​​can be generated in a range of 0 to 100, the confidence threshold can be defined as 85 on the confidence value scale, etc.). The confidence threshold can be a default value and / or it can be set by a user of the computing device. In some examples, the confidence threshold and / or other aspects of calculating the confidence value as described above can be adjusted based on feedback as described herein with respect to machine learning.

[0061] At 408, if the confidence value exceeds the confidence threshold, the process proceeds to 410. Alternatively, if the confidence value does not exceed the confidence threshold, the process returns to 402. When the confidence value does not exceed the confidence threshold, the wake event message may be ignored by the computing device. Alternatively, the computing device, in response, may provide a message to the pen device notifying the pen device that the most recent message may be false. Such a notification may cause the pen device to return to obtaining gesture input data, rather than disabling the collection of gesture input data, for example, when the computing device has transitioned from a low-power state.

[0062] At 410, the computing device transitions from the low-power state to the active state. Transitions between low-power states and other states are well known in the art (e.g., "waking up" a computing device that is in a "sleep" state or a "suspended" state, etc.). The transition may include activating components and / or applications that were disabled in the low-power state and / or changing components and / or applications that are in a low-power mode to an active mode, a default mode, or other mode that causes the components and / or applications to consume additional power (e.g., in exchange for improved performance, etc.).

[0063] In some examples, the transition may also include executing one or more applications, such as applications that may vary depending on the method of detecting the wake event (e.g., if the wake event is detected by a sensor of the pen device as described herein, the computing device may be configured to execute an application that enables a user to write or draw with the pen device, such as a document generation application or an image editing application, etc.).

[0064] Figure 5 5 is an exemplary sequence diagram 500 illustrating interaction between a pen device 502 and a computing device 504 including adjusting a wake-up gesture pattern based on user feedback according to an embodiment. It should be understood that the computing device 504 is in a low power state as described herein at the beginning of the time period described by the diagram 500. At 506, the pen device 502 obtains gesture input data matching the wake-up gesture pattern, and at 508, the pen device 502 transmits a wake-up event message associated with the gesture input data to the computing device 504. The wake-up gesture pattern may be adjusted as described above with respect to FIG. Figure 3A and 3B The processes at 506 and 508 may be performed in substantially the same manner or in a similar manner as described.

[0065] At 510, the computing device receives and verifies the wake event message, and at 512, as a result of the wake event message being verified, the computing device transitions to an active state. Figure 4 The processes at 510 and 512 may be performed in substantially the same manner as described or in a similar manner.

[0066] At 514, the computing device 504 obtains user feedback about the wake-up event. The user feedback can be obtained by the computing device 504 prompting the user of the computing device 504 to confirm or deny whether the wake-up event is intentional. In some cases, the user may have inadvertently triggered a wake-up event based on the sensor data provided to the pen device 502, causing the computing device 504 to wake up at an unintentional time. In response to the prompt of the computing device, the user can indicate that the wake-up event is unintentional, and the computing device 504 can be configured to record the user feedback response and then return to a low-power state. Alternatively, the user may have intended to trigger the wake-up event, and in response to the prompt, it can be confirmed that the wake-up event is intentional. In this case, the computing device 504 can be configured to record the user feedback response and continue to operate in the current active state.

[0067] Alternatively or additionally, the user's immediate interaction with the computing device 504 upon transitioning to the active state may be obtained as user feedback about the wake event. For example, if after transitioning to the active state at 512, the user of the computing device 504 immediately chooses to transition the computing device 504 back to the low-power state, the user's selection may be considered as user feedback that the transition to the active state was triggered in error. Furthermore, if the user immediately begins using the computing device 504, the user's action may be considered as user feedback that the transition to the active state was triggered correctly.

[0068] In some examples, computing device 504 is configured to obtain user feedback only after a subset of wake events. For example, if during verification of the wake event message at 510, the calculated confidence value of the wake event is found to significantly exceed a defined confidence threshold (e.g., the confidence value exceeds the confidence threshold by a defined amount and / or a defined percentage of the confidence threshold, etc.), user feedback may not be obtained because the wake event is likely to have been intentionally caused by the user. Alternatively, if the calculated confidence value is found to be close to a defined confidence threshold (e.g., the confidence value is within a defined amount of the confidence threshold and / or a defined percentage of the confidence threshold, etc.), the user may be prompted to provide feedback or feedback may be obtained from the user in other ways because the wake event may have been unintentional.

[0069] At 516, computing device 504 generates an adjusted wake-up gesture pattern based on the user feedback. In some examples, adjustments are made to the wake-up gesture pattern that causes the wake-up event to be triggered. The adjustments made may include changes to the error margin of the wake-up gesture pattern to make the gesture input more or less likely to match the pattern. For example, if the wake-up event is determined to be unintentional based on user feedback, the error margin of the associated wake-up gesture pattern may be reduced (e.g., the range of values ​​that match to the pattern value may be reduced, the time interval for matching to the pattern may be shortened, etc.). Alternatively or additionally, if user feedback indicates that the user attempted to trigger a wake-up event but did not trigger the wake-up event due to the failure of the sensor input to match the wake-up gesture pattern, the error margin of the associated wake-up gesture pattern may be extended.

[0070] In some examples, user feedback is used to adjust confidence threshold(s) associated with computing device 504 and / or the manner in which confidence values ​​are calculated as described herein. Similar to adjusting the wake-up gesture mode, the confidence thresholds and / or confidence value calculations can be adjusted to make it more or less likely that a wake-up event will be triggered.

[0071] At 518, the computing device 504 transmits the adjusted wake-up gesture pattern 518 to the pen device 502. The transmission may be over the same network connection as the transmission of the wake-up event message at 508. Alternatively or additionally, because the computing device 504 has transitioned to the active state, the computing device 504 may be configured to transmit the adjusted wake-up gesture pattern to the pen device 502 via a different network protocol, such as a protocol that may have been disabled when the computing device 504 was in the low-power state.

[0072] At 520, the pen device 502 receives the adjusted wake-up gesture pattern and stores it along with any other wake-up gesture patterns associated with the pen device 502. The adjusted wake-up gesture pattern may overwrite a previous version of the adjusted wake-up gesture pattern so that the adjusted wake-up gesture pattern is used to identify a wake-up event based on future gesture input data.

[0073] Alternatively or additionally, the computing device 504 may be configured to send feedback data associated with the wake-up event message to the pen device 502, and the pen device 502 may be configured to adjust the wake-up gesture mode based on the sent feedback data. The feedback data sent back to the pen device 502 may include a tag or identifier associated with the wake-up event message, the computing device 504, and / or associated sensor data or other data used by the computing device 504 during verification. For example, a wake-up event found to be false at the computing device 504 may cause a false wake-up event message associated with the wake-up event message to be sent to the pen device 502. The pen device 502 may then tune or adjust the associated wake-up gesture mode based on the machine learning principles and / or techniques described herein. A positive result (e.g., a wake-up event message found to be true and / or accurate, etc.) may be sent to the pen device 502 to be applied to the wake-up gesture mode as well. Although some pen devices may not include the processing resources necessary for such an implementation, in a system with a pen device capable of such processing, the reliance on the computing device for mode tuning may be reduced.

[0074] Additional example scenes

[0075] Aspects of the present disclosure enable various additional scenarios, such as those described next.

[0076] In one example, a user has a computing device in a sleeping state with a pen device. The user picks up the pen device and taps the tip of the pen device twice in rapid succession on a nearby surface. The pen device includes a pressure sensor on the tip of the pen that detects the two taps as gesture input data. The pen device compares the detected taps to a wake-up gesture pattern stored on the pen device. The wake-up gesture pattern includes two taps separated by 0.5 seconds and includes a 20% error margin regarding the timing of the taps. The pen device determines that the detected taps match the wake-up gesture pattern and are within a defined error margin. A wake-up event message is generated and transmitted to the computing device using a Bluetooth network connection.

[0077] The computing device receives the wake-up event message and calculates a confidence value based on the data included in the message and other contextual data associated with the receipt of the message. The computing device verifies the message when the confidence value exceeds a defined confidence threshold, and based on the verification, the computing device transitions from the sleep state to the active state. Once the computing device transitions to the active state, the user begins working on the computing device.

[0078] In another example, a user defines a new wake-up gesture pattern for use with a pen device to wake up a computing device. The new wake-up gesture pattern includes pressing a tip of the pen against a surface and moving the pen in a circular motion, as if drawing a circle on the surface. Data associated with the wake-up gesture pattern includes input data from a pressure sensor in the tip of the pen device and movement data from an IMU in the pen device. The user further defines that when the new wake-up gesture pattern is used to wake up the computing device, the computing device should also activate a user's favorite note-taking application, thereby enabling the user to quickly and intuitively access the application from a sleeping computing device.

[0079] Another example contemplates that a user's computing device is in a low-power state and the user wants to use a note-taking application. The user picks up a pen device and attempts to execute a new wake-up gesture pattern by pressing the pen device tip against the desktop and moving it in a circular pattern. The pen device obtains input data from the pressure sensor and the IMU, but based on the error margin(s) associated with the new wake-up gesture pattern, the pen device determines that the movement input does not sufficiently match the defined pattern. Upon seeing that the computing device does not wake up, the user attempts to execute the pattern again. This time, the input data matches the defined pattern, and a wake-up event message is transmitted to the computing device, including the input data and an identifier of the matched pattern. The computing device verifies the message and transitions to an active state. Based on the matched pattern, the computing device then launches the note-taking application associated with the wake-up gesture pattern for use by the user. The user immediately begins using the note-taking application.

[0080] The computing device collects contextual data associated with a most recent wake event, including data indicating that a user had previously attempted and failed to cause a wake event immediately prior to the most recent wake event. A machine learning component of the computing device determines that the failed attempt prior to the wake event indicates feedback that the error margin(s) associated with a new wake gesture pattern are too narrow, making execution of the pattern more difficult than it should be. The machine learning component adjusts the error margin(s) of the new pattern to widen the range of values ​​that satisfy the pattern, and the adjusted pattern is provided to the pen device for use during future wake gesture detections.

[0081] In other examples, other types of sensor data may be used to detect a wake-up event. For example, a microphone sensor on a pen device may be configured to detect a defined sound (such as a spoken word or phrase) that causes the pen device to initiate a wake-up event. In addition, the described gesture types may be detected by multiple sensors of the pen device (e.g., a tap of the pen device is detected by a pressure sensor on the tip of the pen device, by an IMU embedded in the device, and / or by a microphone sensor of the pen device, etc.). The types of sensors may include a piezoelectric sensor within the pen device that is sensitive to force applied to the pen device to detect the level of grip force used by the user on the pen device (which may be used alone or in combination with other sensor data to trigger a wake-up event), and the like.

[0082] In still other examples, the system includes detecting pen device gestures for other purposes, such as writing, erasing, activating an application, navigating in an interface, etc. The pen device and / or computing device of the system is configured to trigger a wake event upon detection of at least one of these other device gestures.

[0083] In addition, the system can be configured to trigger a wake-up event based on detecting a wake-up gesture and / or other gesture at the pen device using a wide margin of error, so that virtually any detected sensor input data or a detected change in sensor input data causes the pen device to transmit a wake-up event message to the computing device. Thus, the computing device is relatively easy for the user to wake up. However, as the user moves around, carries the computing device and / or pen device, etc., many unintentional wake-up events may be triggered at the pen device level, so the verification at the computing device can be configured and / or tuned to control inadvertent events.

[0084] Exemplary Operating Environment

[0085] The present disclosure can be implemented by Figure 6 600 in the functional block diagram 600. In one embodiment, the components of the computing device 618 may be implemented as part of an electronic device according to one or more embodiments described in this specification. The computing device 618 includes one or more processors 619, which may be a microprocessor, a controller, or any other suitable type of processor for processing computer executable instructions to control the operation of the electronic device. Alternatively or additionally, the processor 619 is any technology (such as a hard-coded or hardware machine) capable of executing logic or instructions. Platform software including an operating system 620 or any other suitable platform software may be provided on the device 618 to enable application software 621 to be executed on the device. According to one embodiment, transitioning the computing device from a low power state based on gesture input obtained by a pen device as described herein may be achieved by software.

[0086] Computer executable instructions may be provided using any computer readable medium accessible to computing device 618. Computer readable media may include, for example, computer storage media such as memory 622 and communication media. Computer storage media (such as memory 622) include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules, etc. Computer storage media include, but are not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device. In contrast, communication media can embody computer readable instructions, data structures, program modules, etc. in modulated data signals such as carrier waves or other transmission mechanisms. As defined herein, computer storage media do not include communication media. Therefore, computer storage media should not be understood to be propagation signals themselves. Propagation signals themselves are not examples of computer storage media. Although computer storage media (memory 622) is shown as being within computing device 618, those skilled in the art will appreciate that the storage may be distributed or located remotely and accessed via a network or other communications link (eg, using communications interface 623).

[0087] The computing device 618 may include an input / output controller 624 configured to output information to one or more output devices 625 (e.g., a display screen or a speaker) that may be separated from or integrated with the electronic device. The input / output controller 624 may also be configured to receive and process input from one or more input devices 626 (e.g., a keyboard, a microphone, or a touch pad). In one embodiment, the output device 625 may also serve as an input device. An example of such a device may be a touch-sensitive display. The input / output controller 624 may also output data to a device other than the output device (e.g., a locally connected printing device). In some embodiments, a user may provide input to (or all) the input device 626 and / or receive output from (or all) the output device 625.

[0088] The functionality described herein may be performed, at least in part, by one or more hardware logic components. According to one embodiment, the computing device 618 is configured by program code of various embodiments that perform the described operations and functionality when executed by the processor 619. Alternatively or additionally, the functionality described herein may be performed, at least in part, by one or more hardware logic components. As an example and not limitation, illustrative types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), program-specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and graphics processing units (GPUs).

[0089] At least some of the functionality of the various elements in the figures may be performed by other elements in the figures or entities not shown in the figures (eg, processors, web services, servers, applications, computing devices, etc.).

[0090] Although described in connection with an exemplary computing system environment, examples of the disclosure are implementable with numerous other general purpose or special purpose computing system environments, configurations, or devices.

[0091] Examples of known computing systems, environments, and / or configurations that may be suitable for various aspects of the present disclosure include, but are not limited to: mobile or portable computing devices (e.g., smart phones), personal computers, server computers, handheld devices (e.g., tablets) or laptops, multiprocessor systems, game consoles or controllers, microprocessor-based systems, set-top boxes, programmable consumer electronics, mobile phones, mobile computing and / or communication devices with wearable or accessory form factors (e.g., watches, glasses, head-mounted devices or headphones), network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. In general, the present disclosure may be operated by any device having processing capabilities that enable it to execute instructions such as those described herein. Such systems or devices may accept input from a user in any manner, including from an input device such as a keyboard or pointing device, via gesture input, proximity input (such as by hovering), and / or via voice input.

[0092] Each example of the present disclosure may be described in the general context of computer executable instructions (such as program modules) executed by one or more computers or other devices in software, firmware, hardware, or a combination thereof. Computer executable instructions may be organized into one or more computer executable components or modules. In general, program modules include, but are not limited to, routines, programs, objects, components, and data structures that perform specific tasks or implement specific abstract data types. Any number of such components or modules and any organization thereof may be utilized to implement various aspects of the present disclosure. For example, various aspects of the present disclosure are not limited to specific computer executable instructions or specific components or modules illustrated in the accompanying drawings and described herein. Other examples of the present disclosure may include different computer executable instructions or components having more or less functionality than that illustrated and described herein.

[0093] In examples involving a general purpose computer, aspects of the disclosure transform the general purpose computer into a special purpose computing device when configured to execute the instructions described herein.

[0094] As an alternative or in addition to other examples described herein, each example includes any combination of the following:

[0095] -A system for transitioning a computing device from a low power state using a pen device, the system comprising:

[0096] - at least one processor of the device;

[0097] - at least one network interface of the device;

[0098] - at least one sensor of the pen device; and

[0099] - at least one memory of the device including computer program code, the at least one memory and the computer program code being configured to, together with the at least one processor, cause the at least one processor to:

[0100] - obtaining gesture input data via the at least one sensor;

[0101] - comparing the acquired gesture input data with at least one gesture pattern;

[0102] - generating a wake-up event message based on the gesture input data matching the at least one gesture pattern; and

[0103] - transmitting the wake event message to the computing device via the network interface, wherein the wake event message is transmitted using a communication protocol that the computing device is configured to receive in the low power state.

[0104] - further including:

[0105] - at least one processor of the computing device;

[0106] - at least one network interface of the computing device; and

[0107] - at least one memory of the computing device including computer program code, the at least one memory of the computing device and the computer program code being configured to, together with at least one processor of the computing device, cause the at least one processor of the computing device to:

[0108] - receiving the wake-up event message from the device via the network interface;

[0109] - generating a confidence value of the wake-up event message;

[0110] - comparing the confidence value to a defined confidence threshold; and

[0111] -transitioning the computing device from the low-power state to an active state based on the confidence value exceeding the confidence threshold.

[0112] - wherein the at least one memory of the computing device and the computer program code are configured to, together with the at least one processor of the computing device, further cause the at least one processor of the computing device to:

[0113] -Disabling an ability to receive wake-up event messages from the pen device for a defined time interval based on transitioning from the low-power state to the active state.

[0114] - wherein the at least one memory of the computing device and the computer program code are configured to, together with the at least one processor of the computing device, further cause the at least one processor of the computing device to:

[0115] - Ignoring the wakeup event message based on the confidence value failing to exceed the confidence threshold, whereby the wakeup event message is deemed to be an unintentional wakeup event.

[0116] - wherein the at least one memory of the computing device and the computer program code are configured to, together with the at least one processor of the computing device, further cause the at least one processor of the computing device to:

[0117] - prompting a user of the computing device via a user interface of the computing device to provide wake-up event feedback; and

[0118] -adjusting a wake-up gesture pattern associated with the wake-up event message based on machine learning based on receiving the wake-up event feedback; and

[0119] The adjusted wake-up gesture pattern is uploaded to the pen device.

[0120] - wherein the at least one gesture pattern comprises tapping the pen device on a surface at least once, wherein the tap is detected by at least one of an IMU and a pressure sensor in at least one sensor of the pen device.

[0121] -Wherein the at least one gesture pattern comprises moving the pen device in a pattern of one or more movements, wherein the one or more movements are detected by an IMU of the pen device.

[0122] - Wherein obtaining the posture input data comprises:

[0123] - detecting that a first sensor of the pen device is pressed, wherein the first sensor comprises at least one of: a button, a switch, or a pressure sensor; and

[0124] - based on the detected pressing of a first sensor of the pen device, obtaining gesture input data from at least one sensor of the pen device other than the pressed first sensor.

[0125] -Wherein the at least one sensor of the pen device comprises at least one of: an inertial measurement unit (IMU), a button, a switch, a pressure sensor, or a microphone.

[0126] -A computerized method for transitioning a computing device from a low power state using a pen device, the method comprising:

[0127] - obtaining, by the pen device, gesture input data from at least one sensor of the pen device;

[0128] - comparing, by the pen device, the acquired gesture input data with at least one gesture pattern;

[0129] - generating a wake-up event message by the pen device based on the gesture input data matching the at least one gesture pattern; and

[0130] - The pen device transmits the wake-up event message to the computing device via the network interface of the pen device, wherein the wake-up event message is transmitted using a communication protocol that the computing device is configured to receive in a low power state.

[0131] - further including:

[0132] - receiving, by the computing device, the wake-up event message from the pen device via a network interface of the computing device;

[0133] - generating, by the computing device, a confidence value for the wake-up event message;

[0134] - comparing, by the computing device, the confidence value to a defined confidence threshold; and

[0135] - transitioning, by the computing device, from the low-power state to an active state based on the confidence value exceeding the confidence threshold.

[0136] - further including:

[0137] - based on the transition from the low-power state to the active state, disabling, by the computing device, an ability to receive a wake-up event message from the pen device for a defined time interval.

[0138] - further including:

[0139] - Ignoring the wake event message by the computing device based on the confidence value failing to exceed the confidence threshold, whereby the wake event message is deemed to be an unintentional wake event.

[0140] - further including:

[0141] - prompting, by the computing device, a user of the computing device to provide wake-up event feedback via a user interface of the computing device; and

[0142] -Adjusting the confidence threshold by the computing device based on machine learning based on receiving wake event feedback.

[0143] - further including:

[0144] - receiving, by the computing device, a request to define a new wake-up gesture pattern;

[0145] - obtaining new gesture input data from at least one sensor of the pen device based on the request;

[0146] - defining, by the computing device, a new wake-up gesture pattern based on the obtained new gesture input data; and

[0147] -The computing device uploads the new wake-up gesture pattern to the pen device.

[0148] - wherein the at least one gesture pattern comprises tapping the pen device on a surface at least once, wherein the tap is detected by at least one of an IMU and a pressure sensor in at least one sensor of the pen device.

[0149] -Wherein the at least one gesture pattern comprises moving the pen device in a pattern of one or more movements, wherein the one or more movements are detected by an IMU of the pen device.

[0150] - Wherein obtaining the posture input data comprises:

[0151] - detecting that a first sensor of the pen device is pressed, wherein the first sensor comprises at least one of: a button, a switch, or a pressure sensor; and

[0152] - based on the detected pressing of a first sensor of the pen device, obtaining gesture input data from at least one sensor of the pen device other than the pressed first sensor.

[0153] - One or more computer storage media having computer executable instructions for causing a computing device to transition from a low power state using a pen device, the computer executable instructions, when executed by a processor of the pen device, causing the processor of the pen device to at least:

[0154] - obtaining gesture input data from at least one sensor of the pen device;

[0155] - comparing the acquired gesture input data with at least one gesture pattern;

[0156] - generating a wake-up event message based on the gesture input data matching the at least one gesture pattern; and

[0157] - transmitting the wake-up event message to the computing device via the network interface of the pen device, wherein the wake-up event message is transmitted using a communication protocol that the computing device is configured to receive in the low-power state.

[0158] - wherein the computer executable instructions include instructions that, when executed by a processor of the computing device, cause the processor of the computing device to perform at least the following operations:

[0159] - receiving the wake-up event message from the device via the network interface;

[0160] - generating a confidence value of the wake-up event message;

[0161] - comparing the confidence value to a defined confidence threshold; and

[0162] -transitioning the computing device from the low-power state to an active state based on the confidence value exceeding the confidence threshold.

[0163] As will be apparent to one skilled in the art, any range or device value given herein may be expanded or altered without losing the effect sought.

[0164] Although aspects of the invention do not track any personally identifiable information, examples have been described with reference to data monitored and / or collected from users. In some examples, the user may be provided with a notification about the data collection (e.g., via a dialog box or preference setting) and given an opportunity to grant or deny consent for the monitoring and / or collection. The consent may take the form of an opt-in consent or an opt-out consent.

[0165] Although the subject matter of the present invention is described in language specific to structural features and / or method actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as example forms of implementing the claims.

[0166] It will be understood that the benefits and advantages described above may relate to one embodiment or may relate to several embodiments. The embodiments are not limited to those embodiments that solve any or all of the problems set forth or those embodiments that have any or all of the benefits and advantages set forth. It will be further understood that reference to "an" item refers to one or more of those items.

[0167] The embodiments illustrated and described herein, as well as embodiments not specifically described herein but within the scope of aspects of the claims, constitute exemplary means for obtaining gesture input data from at least one sensor of a pen device, for comparing the obtained gesture input data to at least one gesture pattern, for generating a wake event message based on matching the gesture input data to the at least one gesture pattern, and for transmitting the wake event message to a computing device via a network interface of the pen device, wherein the wake event message is transmitted using a communication protocol that the computing device is configured to receive in a low power state. The illustrated one or more processors 619 together with computer program code stored in memory 622 constitute exemplary processing means for obtaining and processing sensor data, comparing the data to a gesture pattern, calculating a confidence value and comparing the value to a confidence threshold, and causing the computing device to transition to an active state, as described herein.

[0168] The term “comprising” is used in this specification to mean including the feature(s) or action(s) that follows thereafter, but does not exclude the existence of one or more additional features or actions.

[0169] In some examples, the operations illustrated in the various figures may be implemented as software instructions encoded on a computer-readable medium, implemented in hardware that is programmed or designed to perform the operations, or both. For example, various aspects of the present disclosure may be implemented as a system on a chip or other circuit system including a plurality of interconnected conductive elements.

[0170] The order of execution or completion of the operations in each example of the present disclosure illustrated and described herein is not necessary unless otherwise specified. That is, unless otherwise specified, the operations can be performed in any order, and each example of the present disclosure may include additional operations or operations less than the operations disclosed herein. For example, it is contemplated that performing or completing another operation before, simultaneously, or after a particular operation is also within the scope of the various aspects of the present disclosure.

[0171] When introducing elements of aspects of the present disclosure or examples thereof, the articles "a," "an," "the," and "said" are intended to mean one or more of such elements. The terms "comprising," "including," and "having" are intended to be inclusive and mean that additional elements may be present in addition to the listed elements. The term "exemplary" is intended to mean "an example of..." The phrase "one or more of: A, B, and C" means "at least one of A and / or at least one of B and / or at least one of C."

[0172] Having described various aspects of the disclosure in detail, it will be apparent that various modifications and variations may be made without departing from the scope of the various aspects of the disclosure as defined in the appended claims. As various changes may be made to the above constructions, products, and methods without departing from the scope of the various aspects of the disclosure, it is intended that all subject matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative rather than restrictive.

Claims

1. A system for transitioning a computing device from a low power state using a pen device, the system comprising: at least one processor of the pen device; at least one network interface of the pen device; at least one sensor of the pen device; as well as at least one memory of the pen device including computer program code, the at least one memory and the computer program code being configured to, together with the at least one processor, cause the at least one processor to: obtaining gesture input data via the at least one sensor; comparing the obtained gesture input data to at least one gesture pattern; generating a wake event message based on the gesture input data matching the at least one gesture pattern; as well as transmitting the wake event message to the computing device via the network interface to cause the computing device to transition out of the low power state, wherein the wake event message is transmitted using a communication protocol that the computing device is configured to receive in the low power state; The system further comprises: at least one processor of the computing device; at least one network interface of the computing device; and at least one memory of the computing device including computer program code, the at least one memory of the computing device and the computer program code being configured to, together with at least one processor of the computing device, cause the at least one processor of the computing device to: The wake-up event message is received from the pen device via the network interface.

2. The system of claim 1, wherein the at least one memory of the computing device and the computer program code are configured to, together with at least one processor of the computing device, further cause the at least one processor of the computing device to: generating a confidence value for the received wake-up event message; comparing the generated confidence value to a defined confidence threshold; and The computing device is transitioned from the low-power state to an active state based on the confidence value exceeding the confidence threshold.

3. The system of claim 2, wherein the at least one memory of the computing device and the computer program code are configured to, together with the at least one processor of the computing device, further cause the at least one processor of the computing device to: An ability to receive a wake event message from the pen device is disabled for a defined time interval based on transitioning from the low-power state to the active state.

4. The system of claim 2, wherein the at least one memory of the computing device and the computer program code are configured to, together with the at least one processor of the computing device, further cause the at least one processor of the computing device to: The wake-up event message is ignored based on the confidence value failing to exceed the confidence threshold, whereby the wake-up event message is deemed to be an unintentional wake-up event.

5. The system of claim 2, wherein the at least one memory of the computing device and the computer program code are configured to, together with the at least one processor of the computing device, further cause the at least one processor of the computing device to: prompting a user of the computing device to provide wake-up event feedback via a user interface of the computing device; as well as adjusting a wake-up gesture pattern associated with the wake-up event message based on machine learning based on receiving the wake-up event feedback; as well as The adjusted wake-up gesture pattern is uploaded to the pen device.

6. The system of claim 1, wherein: The at least one gesture pattern includes tapping the pen device against a surface at least once, wherein the tap is detected by at least one of an inertial measurement unit (IMU) and a pressure sensor in at least one sensor of the pen device.

7. The system of claim 1, wherein: The at least one gesture pattern includes moving the pen device in a pattern of one or more movements, wherein the one or more movements are detected by an IMU of the pen device.

8. The system of claim 1, wherein: Obtaining the posture input data includes: detecting that a first sensor of the pen device is pressed, wherein the first sensor comprises at least one of: a button, a switch, or a pressure sensor; and Gesture input data is obtained from at least one sensor of the pen device other than the pressed first sensor based on the detected pressing of the first sensor of the pen device.

9. The system of claim 1, wherein: The at least one sensor of the pen device includes at least one of: an IMU, a button, a switch, a pressure sensor, a capacitive sensor, or a microphone.

10. A computerized method for transitioning a computing device from a low power state using a pen device, the method comprising: obtaining, by the pen device, gesture input data from at least one sensor of the pen device; comparing, by the pen device, the obtained gesture input data with at least one gesture pattern; generating, by the pen device, a wake-up event message based on the gesture input data matching the at least one gesture pattern; transmitting, by the pen device through a network interface of the pen device, the wake-up event message to the computing device to cause the computing device to transition out of the low-power state, wherein the wake-up event message is transmitted using a communication protocol that the computing device is configured to receive in the low-power state; as well as The wake event message is received by the computing device from the pen device via a network interface of the computing device.

11. The computerized method of claim 10, further comprising: generating, by the computing device, a confidence value for the wake-up event message; comparing, by the computing device, the confidence value to a defined confidence threshold; as well as Transitioning, by the computing device, from the low-power state to an active state based on the confidence value exceeding the confidence threshold.

12. The computerized method of claim 11, wherein: Generating a confidence value for the wake event message includes calculating the confidence value based on a combination of a first sensor data set included in the wake event message and a second sensor data set from at least one sensor of the computing device.

13. The computerized method of claim 11, further comprising: Based on the transition from the low-power state to the active state, an ability to receive a wake event message from the pen device is disabled, by the computing device, for a defined time interval.

14. The computerized method of claim 11, further comprising: The wake event message is ignored by the computing device based on the confidence value failing to exceed the confidence threshold, whereby the wake event message is deemed to be an unintentional wake event.

15. The computerized method of claim 11, further comprising: Prompting, by the computing device, a user of the computing device to provide wake-up event feedback via a user interface of the computing device; as well as The confidence threshold is adjusted by the computing device based on machine learning based on receiving the wake event feedback.

16. The computerized method of claim 11, further comprising: Receiving, by the computing device, a request to define a new wake-up gesture pattern; obtaining new gesture input data from at least one sensor of the pen device based on the request; defining, by the computing device, a new wake-up gesture pattern based on the obtained new gesture input data; as well as The new wake-up gesture pattern is uploaded to the pen device by the computing device.

17. The computerized method of claim 10, wherein: The at least one gesture pattern includes tapping the pen device against a surface at least once, wherein the tap is detected by at least one of an IMU and a pressure sensor of at least one sensor of the pen device.

18. The computerized method of claim 10, wherein: Obtaining the posture input data includes: detecting that a first sensor of the pen device is pressed, wherein the first sensor comprises at least one of: a button, a switch, or a pressure sensor; and Gesture input data is obtained from at least one sensor of the pen device other than the pressed first sensor based on the detected pressing of the first sensor of the pen device.

19. One or more computer storage media having computer executable instructions for transitioning a computing device from a low power state using a pen device, the computer executable instructions, when executed by a processor of the pen device, causing the processor of the pen device to at least: obtaining gesture input data from at least one sensor of the pen device; generating a wakeup event message based on the obtained gesture input data, the wakeup event message including at least a portion of the obtained gesture input data; as well as transmitting the wake event message to the computing device via a network interface of the pen device to cause the computing device to transition out of the low power state, wherein the wake event message is transmitted using a communication protocol that the computing device is configured to receive in the low power state; The computer executable instructions include instructions that, when executed by the processor of the computing device, cause the processor of the computing device to perform at least the following operations: The wake-up event message is received from the pen device via the network interface.

20. The one or more computer storage media having computer executable instructions as recited in claim 19, wherein: The computer executable instructions further include instructions that, when executed by the processor of the computing device, cause the processor of the computing device to at least perform the following operations: generating a confidence value of the wake-up event message; comparing the confidence value to a defined confidence threshold; and The computing device is transitioned from the low-power state to an active state based on the confidence value exceeding the confidence threshold.

21. A computer-readable storage medium having instructions, which, when executed, cause a machine to perform the method of any one of claims 10-18.