Authentication management via IMU and radar

Abstract

To provide a technique and a system for authentication management via an IMU and a radar.SOLUTION: A technique and a system use inertia sensor data and / or radar data from an inertial measurement unit (IMU) in order to manage authentication for a computing device. Accordingly, the technique saves electric power, improves accuracy and reduces a waiting time compared to a number of general techniques and systems for authentication for the computing device.SELECTED DRAWING: Figure 1

Description

[Background technology]

[0001] background User equipment such as smartphones, wearable computers, and tablets often require user authentication before the user has access to the device. However, current authentication techniques often cannot authenticate users quickly or accurately. Many techniques also cannot authenticate users without expending excessive power to authenticate, which is particularly problematic for battery-powered devices.

[0002] Additionally, users are interacting with their devices more and more frequently, with some users authenticating themselves to their devices dozens or even hundreds of times a day. Because of this need to authenticate users and the number of times users are authenticated, the time, power, and ease of doing so become increasingly important. Summary of the Invention

[0003] overview This document describes a method and system for authentication management via IMU and radar. The method and system utilize inertial sensor data from an inertial measurement unit (IMU) to manage authentication for computing devices. In doing so, the technique saves power, improves accuracy, or reduces latency compared to many common techniques and systems for computing device authentication.

[0004] For example, a method is described for determining a user's intent to engage based on radar data and by user equipment. In response to the determination, the method changes a power state of a power-consuming component of an authentication system from a first power state to a second power state, the second power state consuming more power than the first power state. The method then performs an authentication process by the authentication system, where the authentication system uses the power-consuming component in the second power state or a third, higher power state.

[0005] This document also describes computer readable media having instructions for performing the methods outlined above and other methods described herein, as well as systems and means for performing these methods.

[0006] This summary is provided to introduce simplified concepts for authentication management via IMU and radar, which are further described below in the detailed description and drawings. This summary is not intended to identify essential features of the claimed subject matter, nor is it intended for use in determining the scope of the claimed subject matter.

[0007] BRIEF DESCRIPTION OF THE DRAWINGS In this document, details of one or more aspects of authentication management via IMU and radar are described with reference to the following drawings, in which the same numbers are used throughout to refer to like features and components. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an exemplary environment in which an approach for authentication management via IMU and radar may be implemented. [Figure 2] FIG. 2 illustrates an example of the authentication system described in FIG. 1. [Figure 3] FIG. 3 illustrates an exemplary user authenticated by the authentication system of FIG. 2. [Figure 4]2 illustrates an example implementation of the user equipment of FIG. 1 that can change state, including power state, of the authentication system in response to determining a user's intent to engage the user equipment. [Figure 5] FIG. 1 illustrates exemplary information, power, and access states of a user equipment. [Figure 6-1] FIG. 1 illustrates an exemplary radar system as part of a computing device. [Figure 6-2] FIG. 1 illustrates an exemplary transceiver and processor. [Figure 6-3] FIG. 1 illustrates an exemplary relationship between power consumption, gesture frame update rate, and response delay. [Figure 6-4] FIG. 2 illustrates an exemplary framing structure. [Figure 7] FIG. 6-2 illustrates an exemplary arrangement of receive antenna elements for the radar system of FIG. 6-1. [Figure 8] FIG. 6-2 illustrates additional details of an example implementation of the radar system of FIG. 6-1. [Figure 9] FIG. 6 illustrates an exemplary scheme that may be implemented by the radar system of FIG. 6-1. [Figure 10] FIG. 1 illustrates an exemplary method for authentication management via an IMU and / or radar. [Figure 11] FIG. 1 illustrates an exemplary scenario for authentication management via IMU and radar. [Figure 12] FIG. 1 illustrates an exemplary method for reducing a state of a user equipment. [Figure 13] FIG. 1 illustrates an exemplary scenario for degrading a user equipment state. [Figure 14] FIG. 1 illustrates an exemplary method for maintaining an authenticated state. [Figure 15] FIG. 1 illustrates an example scenario for maintaining an authenticated state. [Figure 16] FIG. 10 illustrates another exemplary scenario for maintaining an authenticated state. DETAILED DESCRIPTION OF THE INVENTION

[0009] Detailed Description Overview This document describes a method and system for authentication management using an inertial measurement unit (IMU) and radar. The method and system use inertial sensor data from the IMU and radar data to manage authentication for user equipment. By doing so, the method saves power or increases speed in the authentication system.

[0010] For example, assume a user leaves their smartphone on their desk, has a 20-minute conversation with a colleague, and then returns to their desk and smartphone. When they return, the smartphone will be de-authenticated, which is typical for many computing devices due to timeout periods that are well under 20 minutes. Therefore, when the user picks up the phone, the user will need to re-authenticate. To do so, the user will press a button, after which the smartphone will turn on the display and present authentication options such as entering a password through the display or orienting the device's front camera to perform facial recognition. After the user picks up the phone, presses a button, and is authenticated through the user's touch input on the display or use of the smartphone's front camera, the smartphone's state is authenticated and unlocked, moving from a state of no or low access rights to a state of high or similar access rights (e.g., an unlocked state). While users may not be aware of how smartphones work, in many cases, smartphones power up the display or front camera to re-authenticate. The user must be authenticated, which takes both time and power, and only then can the user be authenticated. This slows down the authentication process.

[0011] In contrast, consider a disclosed technique that uses radar data to determine that a user is reaching out toward their smartphone, and then uses IMU data to determine that the user is picking up (e.g., lifting) their smartphone. Using either of these data, the technique can more quickly instantiate the power-up of the smartphone's display, front camera, or other authentication component, thereby reducing the amount of time required for the user to be authenticated.

[0012] This is just one example of how the described techniques and devices can be used to manage authentication via IMUs and radar. Other examples and implementations are described throughout this document. The document will now turn to an example operating environment, after which example devices, methods, and systems are described.

[0013] Operating environment 1 illustrates an example environment 100 in which an approach for authentication management via IMU and radar may be implemented. The example environment 100 includes a user equipment (UE) 102 (e.g., a smartphone) that includes or is associated with a radar system 104, a radar manager 106, an inertial measurement unit (IMU) 108, a motion manager 110, a state manager 112, an authentication system 114, and a display 116.

[0014] In the exemplary environment 100, the radar system 104 provides a radar field 118 by transmitting one or more radar signals or waveforms as described below with reference to FIGS. 7-9. The radar field 118 is a volume of space from which the radar system 104 can detect reflections of radar signals and waveforms (e.g., radar signals and waveforms reflected from objects within the volume of space, also generally referred to herein as radar data). The radar system 104 also enables the UE 102 or other electronic devices to sense and analyze this radar data from reflections within the radar field 118. The radar field 118 may take any of a variety of shapes and forms. For example, the radar field 118 may have a shape as described with reference to FIGS. 1 and 7. In other cases, the radar field 118 may take a shape such as a radius extending from the radar system 104, a volume around the radar system 104 (e.g., a sphere, a hemisphere, a portion of a sphere, a beam, or a cone), or a non-uniform shape (e.g., to accommodate interference from obstacles within the radar field 118). The radar field 118 may extend any of a variety of distances from the radar system 104, such as from a few inches to 12 feet (less than 1 / 3 meter to 4 meters). The radar field 118 may be predefined, may be user selectable, or may be determined through another method (e.g., based on power requirements, remaining battery life, or another factor).

[0015] Reflections from the user 120 within the radar field 118 allow the radar system 104 to determine various information about the user 120, such as the body position or posture of the user 120. Such information may be indicative of a variety of different non-verbal body language cues, body positions, or body postures. These cues, positions, and postures may indicate the absolute position or distance of the user 120 relative to the UE 102, changes in the position or distance of the user 120 relative to the UE 102 (e.g., whether the user 120, or the user's hands, or an object held by the user 120, is moving closer to or further away from the UE 102), or movements of the user 120 toward or away from the UE 102. The reflections may include the velocity of the user 120 (e.g., hand, or non-user object) as it moves away from the UE 102, whether the user 120 is facing or facing away from the UE 102, whether the user 120 is leaning toward the UE 102, waving, reaching out, or pointing at the UE 102, etc. These reflections may also be analyzed to determine or add confidence to authentication, such as human identity (e.g., scattering center of the user's face) via analysis of radar data.

[0016] The radar manager 106 is configured to determine the user's intent to engage, disengage, or maintain engagement with the UE 102 based on radar data from the radar system 104. The user's intent can be inferred from the various cues, position, posture, and distance / velocity described above, such as based on extending a hand or arm toward the UE 102, eye movement toward the UE 102, or head or face movement directed toward the UE 102. For a hand or arm outstretched, the radar manager 106 determines that the user is extending their hand or orienting their arm in a manner that indicates a possible intent to touch or pick up the UE 102. Examples include the user reaching toward the volume buttons on a wirelessly attached speaker, reaching toward a wireless or wired mouse associated with a tablet computer, or reaching toward the UE 102 itself. The outstretched hand can be determined based on hand movement alone, arm and hand movement, or bending and straightening the arm in a manner that allows the hand of the arm to touch or grasp the UE 102. As described below in FIGS. 14-16, this determination of intent to engage may be for a non-user or for a user, whether authenticated or not.

[0017] A user's intent to engage may also be inferred based on the user's head or eye movements to look at or orient their face toward the UE 102 or, possibly, associated peripheral devices of the UE 102. For a user's eye movements looking toward the UE 102, the radar manager 106 determines, such as through tracking the user's eyes, that the user's eyes are looking in the direction of the UE 102. For a user's head movements that orient their face toward the UE 102 (e.g., face orientation), the radar manager 106 determines that various points (e.g., scattering centers, as described below) are currently oriented such that the user's face is pointing toward the UE 102. Thus, the user does not need to perform any action designed to control or activate the UE 102, such as activating (pressing) a button on the UE 102 or performing a touch-dependent gesture (e.g., on a touchpad or screen) or a touch-independent gesture (e.g., using the radar system 104), for the radar manager 106 to determine that the user intends to engage with the UE 102 (or to disengage or maintain engagement with the UE 102).

[0018] As described above, the radar manager 106 is also configured to determine the user's intent to disengage from the UE 102. The radar manager 106 determines the user's intent to disengage similarly to the user's intent to engage, but the user's intent to disengage is inferred from the user's hand or arm moving away from the UE 102 (e.g., retracting), eye movement away from the UE 102, or head or facial movement away from the UE 102 (e.g., a change in facial orientation to avoid looking at the UE 102). Additional aspects for determining the user's intent to disengage include not only the reversal or cessation of engagement described above, but also radar data indicating that the user has walked away, moved their body away, or engaged with another unrelated object or device. Thus, the radar manager 106 determines the user's intent to engage with some other object, device, or user equipment. An intent to disengage from the UE 102 may be determined based on determining whether the user is looking at or interacting with the smartphone. For example, assume a user is looking at and interacting with a smartphone. Examples of intent to engage that indicate an intent to disengage from the smartphone include the user looking at a television screen instead of looking at the smartphone, starting to talk to someone physically nearby, or reaching for another device, such as an e-book or media player, whose engagement would replace engagement with the smartphone.

[0019] The radar manager 106 is also configured to determine the user's intent to maintain engagement with the UE 102. This maintenance of engagement can be active or passive. For active engagement, the radar manager 106 may determine, based on radar data, that the user is interacting through touch-independent gestures, etc. The radar manager 106 may also, or instead, determine active engagement through non-radar data (e.g., performed with assistance from other components of the UE 102). This non-radar data includes indications that the user is entering data into or controlling the UE 102 or a peripheral device. Thus, through touch, typing, or voice data, the user may be determined to be touching (e.g., tapping or gesturing on a soft keyboard) through touchscreen input on the display 116, typing on a peripheral keyboard, or dictating voice input. For maintaining passive engagement, the radar manager 106 determines, alone or through the assistance of other components of the UE 102, that the user is consuming content or offering the UE 102 to another person for content consumption, for example, by facing the UE 102, looking at the display 116, or holding the UE 102 in a manner that orients the UE 102's display so that it is viewable by the user or a third party. Other examples of maintaining passive engagement include user presence, such as via the radar manager 106 determining that the user 120 is near the UE 102 (e.g., within 2 meters, 1.5 meters, 1 meter, or 0.5 meters of the UE 102). Details of exemplary ways in which the radar manager 106 passively and actively determines a user's intent to engage, disengage, or remain engaged are described below.

[0020] Additionally, the radar manager 106 may also use radar data from the radar system 104 to determine gestures made by the user. These gestures may involve the user touching some surface, such as a table, the display 116, or their shirt sleeve, or touch-independent gestures. Touch-independent gestures may be made in the air, in three dimensions, and / or without requiring the hand or fingers to touch an input device, but are not prevented from touching some object. These gestures may be determined based on the radar data and then used as inputs to the UE 102 or to indicate engagement with the UE 102. Exemplary gestures include those similar to sign language (e.g., ASL (American Sign Language)), which may be various complex one- or two-handed gestures, or simple two- or one-handed gestures, such as swiping left, right, up, or down, raising or lowering a palm (e.g., to raise or lower the volume of the UE 102 or the volume of a television or stereo controlled through the UE 102), or swiping forward or backward (e.g., from left to right or right to left) to change music and video tracks, snooze alarms, hang up a phone call, or even play games. These are just a few of the many exemplary gestures and functions that can be controlled by these gestures and enabled via the radar system 104 and radar manager 106. For this reason, this document is intended to be a guide only to those skilled in the art. Although directed to gesture recognition and state management, nothing in this document should be misconstrued to indicate that the radar system 104 and radar manager 106 cannot be configured for gesture recognition.

[0021] The IMU 108 may be any of a variety of devices configured to measure motion, which is defined herein to include specific forces, angular rates, orientations, vibrations, accelerations, velocities, and positions, including pitch, roll, and yaw about each of three axes (e.g., X, Y, and Z). The IMU 108 may be one or more devices within the UE 102, such as an accelerometer, a gyroscope, and / or a magnetometer.

[0022] The motion manager 110 is configured to determine the motion of the UE 102 based on the inertial data from the IMU 108. Exemplary motions include the UE 102 being picked up (e.g., picked up), being oriented toward or away from the user 120, and vibration. Exemplary motions may indicate the UE 102 ceasing to be in physical contact with the user 120, the UE 102 being placed on an inanimate object (e.g., a table, an automobile console, the arm of a couch, a pillow, a floor, a docking station), and the UE 102 being placed within an enclosed container, such as a pocket, a bag, or a purse.

[0023] These movements may indicate a user's potential disengagement, disengagement, or continued engagement with the UE 102. For example, the movements of the UE 102 may indicate that the user equipment is moving or pointing toward the user 120 or being moved / pointing away from the user 120, that the user equipment is moving too quickly or changing its movement too quickly to be interacted with for many possible types of user engagement, that the user equipment is being held by the user 120 (via natural human movements, breathing, heartbeat), or vibrating due to mechanical or non-user sources (e.g., vehicle vibration, ambient sounds shaking the UE 102, music vibrating the UE 102). Thus, turning away indicates a potential disengagement with the UE 102, but may also include a change in orientation of the UE 102 such that the user 120 is no longer looking at the display 116 from a previous orientation. A user 120 typing or reading in one orientation and then flipping the phone over or onto its side, or placing it in a pocket, are just some examples of movements that may indicate a farther orientation and thus potential disengagement. Exemplary movements that may indicate maintained engagement include vibrations that indicate the user is maintaining a hold or position of the UE 102 or maintaining an orientation relative to the UE 102 that previously indicated or was consistent with engagement with the UE 102.

[0024] Display 116 may include any suitable display device, such as a touchscreen, a liquid crystal display (LCD), a thin film transistor (TFT) LCD, an in-place switching (IPS) LCD, a capacitive touchscreen display, an organic light emitting diode (OLED) display, an active-matrix organic light-emitting diode (AMOLED) display, a Super AMOLED display, etc. As mentioned, display 116 may be powered at various levels, such as full color saturation with touch input powered, reduced color saturation with touch input not powered, and low color saturation and low power (e.g., a gray clock) or no power.

[0025] The state manager 112 manages authentication for the UE 102, but may also manage the state of the UE 102, such as its power state and information state. This management is based on decisions made by the radar manager 106 and the motion manager 110. For example, the state manager 112 can manage power to components of the authentication system 114 by modifying the display 116 of the UE 102 to power up in anticipation of receiving touch input from the user 120 to enter a password, modifying a computer processor to perform calculations used in authentication, modifying an imaging system to perform image-based facial recognition, modifying a radar (e.g., radar system 104), or other components used to authenticate the user 120, etc.

[0026] As will be described, this management of the UE 102 is based on determinations by the radar manager 106 and the motion manager 110, which determine an intent to engage with the UE 102, an intent to disengage from the UE 102, or an intent to maintain that engagement and the motion of the UE 102, respectively. The state manager 112 may do so based solely on these determinations or based on other information, such as the current state, current engagement, running applications, and content exhibited by those applications. Also, while the radar manager 106 may determine the user's intent and the motion manager 110 can determine motions, some of which are determined to indicate the user's intent to engage with the UE 102, the state manager 112 can use both of these determinations to improve the accuracy, robustness, and / or speed of its overall determination that the user's intent is to engage with the UE 102 and thus authenticate themselves to the UE 102.

[0027] This use of both the radar manager 106 and motion manager 110 determinations may occur together or in stages as part of the management of the authentication system 114, or only one of them may be used. For example, assume the UE 102 is in a low power state for components used to authenticate. The radar manager 106 may determine that the user 120 intends to authenticate with the UE 102 based on movement toward the UE 102 or reaching out toward the UE 102. In some cases, this alone may be deemed insufficient by the state manager 112 to cause the UE 102 to change to a higher power state. Thus, the state manager 112 may power up some of the authentication components to an intermediate state rather than a higher power state (e.g., high power state 504-1 of FIG. 5). For example, if the authentication system 114 uses an infrared sensor to perform facial recognition, the state manager 112 can power these sensors and the display 116 to a higher power in anticipation of authenticating the user and, for the display 116, indicating to the user that the UE 102 is "awake" and therefore increasingly responsive. As an additional step, the state manager 112 can wait until the motion manager 110 determines that the user has moved, picked up, lifted, etc. the UE 102 before fully powering the authentication component, here the infrared sensor. Although not required, the state manager 112 may allow authentication to be attempted by the component without further input from the user, thereby making authentication seamless to the user 120.

[0028] However, in some cases, the state manager 112 powers up or otherwise prepares the state of the UE 102 and various authentication components in response to both inertial data and radar data, for example, the radar manager 106 determining that the user intends to authenticate and the motion manager 110 determining that the user is picking up the UE 102.

[0029] In other words, the state manager 112 can wait until it has a higher level of confidence that the user's intent is to authenticate by picking up the UE 102, such as an indication by the motion manager 110 that the user has just begun to touch the UE 102. In such a case, the state manager 112 may increase power based solely on the determination of the radar manager 106, but instead of waiting until the motion manager 110 indicates contact by the user to fully power the display or authentication system 114 or components thereof, it may increase power only to an intermediate power level for these components. However, as will be described, the state manager 112 may change state to a higher power level based solely on a determination of intent to engage.

[0030] One of many example ways in which the state manager 112 can manage the authentication of the UE 102 is illustrated by the example environments 100-1, 100-2, and 100-3 of FIG. 1 . In environment 100-1, a user 120 is reading a book with the UE 102 resting on a table. Assume that the user 120 places their book on the edge of the table, or speaks, or some other vibration source causes the UE 102 to vibrate. Based on this vibration, the motion manager 110 might determine that the UE 102 is being moved or might ignore it as an ambient vibration. However, assume that the vibration is of sufficient amplitude or characteristics to cause some types of IMUs and their associated applications to determine that the movement likely indicates user engagement. In contrast to these types of IMUs and conventional state management systems, the state manager 112 also or instead receives information from the radar manager 106 indicating that the user 120 is maintaining engagement or is currently behaving in a manner that indicates a lack of engagement. The radar manager 106 may do so based on the user's 120 body orientation, eye contact, or facial orientation, which is toward the book rather than the UE 102. Thus, the state manager 112 may refrain from increasing power, illuminating the display, powering up the authentication system 114, and otherwise wasting resources or annoying the user 120 by disrupting the user's 120 engagement with the book. An exemplary benefit of temporarily moving away from this scenario can be seen in the context of a moving automobile, where the user is driving with their face forward and their hands on the steering wheel, and the UE 102 includes a smartphone resting on the passenger seat. Often, a bump in the road, in the absence of radar-based input, may cause the IMU and its accompanying application to determine that the user is likely engaged, when in fact the user did not intend to engage. This creates a false positive condition in which authentication power-up resources are wasted.However, if radar-based input is taken into account, the radar-based input may indicate that the user's arm is not extended towards the smartphone, thus avoiding the false positive condition.

[0031] Continuing with this scenario, consider example environment 100-2 in which radar manager 106 determines that a user intends to engage with UE 102. This intent to engage may be determined based on radar data from radar system 104 indicating that user 120 is looking toward UE 102, orienting their body toward UE 102, or extending their hand toward UE 102 (all three are illustrated, but any one of these three conditions may be sufficient to indicate intent to engage). As described in more detail below, state manager 112 may determine to change the state of UE 102, such as authentication system 114, based on the intent to engage determined by radar manager 106. However, other or additional components may also be changed.

[0032] To conclude this scenario, consider environment 100-3. State manager 112 quickly and seamlessly transitions authentication system 114 of UE 102 to a higher power state. As will be described, the state manager 112 does so based on the radar manager 106's indication that the user 120 intends to engage with the UE 102. For environment 100-3, it is assumed that the user 120 is authenticated when the user is just touching the UE 102. This authentication and unlocking is indicated to the user 120 through a change in the display 116 from a low-saturation, low-light star symbol (indicated at 122 in environment 100-2) to a high-saturation, high-light star symbol (indicated at 124 in environment 100-3).

[0033] More particularly, consider the example of an authentication system 114 shown in Figure 2. This is only an example, as there are other possible authentication systems that may be controlled by the state manager 112, such as password entry via a touch-sensitive display, radar authentication using the radar system 104, or a fingerprint reader, to name just a few.

[0034] This example of an authentication system 114 is illustrated showing the interior 200 of a UE 102 (shown as a smartphone). In the illustrated configuration, the UE 102 includes a radar integrated circuit 202 of the radar system 104, a speaker 204, a front-facing camera 206, a proximity sensor 208, and an ambient light sensor 210. The UE 102 also includes a face recognition unlock sensor 212, which includes a near-infrared (NIR) flood illuminator 214 and a near-infrared (NIR) flood illuminator 216. The facial recognition unlock sensor 212 also includes two NIR cameras 218-1 and 218-2 positioned on either side of the UE 102. The NIR cameras 218-1 and 218-2 sense infrared and near-infrared light reflected by the user. This reflected near-infrared light can be used to determine facial features and, using these features, determine whether the user is authentic based on a comparison with previously stored facial feature information. The NIR flood illuminator 214, for example, "floods" NIR light into the environment, which, upon reflection from the user (and other objects), provides an image. This image includes the user's face, even in low or no ambient light conditions, and can therefore be used to determine facial features. The NIR dot projector 216 provides NIR light reflections that can be analyzed to determine the depth of objects containing the user's facial features. To this end, a depth map (e.g., a spectral depth map) may be created for the user (e.g., beforehand when setting up face authentication), and a current depth map may be determined and compared to a previously created, stored depth map. This depth map helps prevent authentication of a photograph or other two-dimensional rendering of the user's face (rather than the person's actual face).

[0035] This mapping of the user's facial features can be securely stored on the UE 102 and, based on the user's preferences, can be both secure on the UE 102 and prevented from being made available to external entities.

[0036] The authentication system 114 includes a facial recognition unlock sensor 212, but may also include other components such as a front camera 206, a proximity sensor 208, and an ambient light sensor 210, as well as a processor for analyzing data, memory (which may also have multiple power states) for storing, caching, or buffering sensor data, etc.

[0037] The face recognition unlock sensor 212 senses IR (infrared) and NIR (near-infrared) data to perform face recognition, which is one way the technique authenticates a user and therefore changes the access state (e.g., unlocks the UE 102), as described in the methods described below. To conserve power, the face recognition unlock sensor 212 operates in a low-power state when not in use (it can also simply be turned off). In particular, the NIR flood illuminator 214 and the NIR dot projector 216 , and do not emit in the off state. However, a warm-up sequence associated with transitioning from a low or no-power state to an intermediate power state and / or a high-power state may be used for the NIR flood illuminator 214 and the NIR dot projector 216. Powering up one or both of these components can reduce the latency in authenticating a user, sometimes by half a second or more. Considering that many users authenticate their devices dozens or even hundreds of times each day, this can save the user time and improve the user experience. As described herein, this time delay is reduced by the radar manager 106 determining, based on radar data provided by the radar system 104, that the user intends to engage their device. This is managed by the state manager 112. In practice, a technique proactively detects the user's intent to engage and initiates a warm-up sequence. The technique may do so before the user touches the UE 102, but this is not required. Thus, the technique allows the NIR flood illuminator 214 and the NIR dot projector 216 to be fully powered for use in authenticating the user, which reduces the time spent by the user waiting for facial recognition to complete.

[0038] Before proceeding to other components of the UE 102, consider the aspect of the facial recognition unlock sensor 212. This exemplary component of the authentication system 114 can authenticate a user using facial recognition at only 10 degrees relative to the plane of the display 116. Thus, the user does not need to pick up the phone and point the sensor at an angle of 70-110 degrees or 80-100 degrees toward their face; instead, the authentication system 114 is configured to use the facial recognition unlock sensor 212 to authenticate the user even before the user picks up the UE 102. This is illustrated in FIG. 3, which shows a user 120 with the portion of their face used for facial recognition (e.g., their chin, nose, or cheekbones) positioned at an angle 302 that may be only 10 degrees relative to the plane 304 of the display 116. The user 120 is also shown to be authenticated while their face is positioned more than one meter away from the facial recognition unlock sensor 212, as indicated by face distance 306. By doing so, the technique allows for nearly seamless and rapid authentication even when the UE 102 is oriented upside down or at an odd angle.

[0039] More particularly, consider FIG. 4 , which illustrates an example implementation 400 of a UE 102 (including a radar manager 106, a motion manager 110, and a state manager 112) in which an approach for authentication management via IMU and radar can be implemented. The UE 102 of FIG. 4 is illustrated with various example devices, including a UE 102-1, a tablet 102-2, a laptop 102-3, a desktop computer 102-4, a computing watch 102-5, computing glasses 102-6, a gaming system 102-7, a home automation and control system 102-8, and a microwave oven 102-9. The UE 102 may also include other devices, such as a television, an entertainment system, an audio system, an automobile, a drone, a trackpad, a drawing pad, a netbook, an e-reader, a home security system, and other home appliances. Note that the UE 102 may be wearable, non-wearable but mobile, or relatively non-mobile (e.g., desktop and appliance).

[0040] Exemplary overall lateral dimensions of the UE 102 may be, for example, approximately 8 centimeters by approximately 15 centimeters. An exemplary footprint of the radar system 104 may be even more limited, such as approximately 4 millimeters by 6 millimeters, including the antenna. Such a limited footprint requirement for the radar system 104, combined with the power and processing limitations needed to accommodate many other desirable features of the UE 102 within such a space-limited package, can result in significant limitations in the accuracy and effectiveness of radar gesture detection. Although this may result in compromises, at least some of the compromises may be overcome in light of the teachings herein.

[0041] The UE 102 also includes one or more computer processors 402 and one or more computer-readable media 404, which include memory media and storage media. Applications and / or an operating system (not shown) embodied as computer-readable instructions on the computer-readable media 404 may be executed by the computer processor 402 to provide some or all of the functionality described herein, such as some or all of the functions of the radar manager 106, the motion manager 110, and the state manager 112 (shown within the computer-readable media 404, but this is not required).

[0042] The UE 102 may also include a network interface 406. The UE 102 can use the network interface 406 to communicate data over a wired network, a wireless network, or an optical network. By way of example and not limitation, the network interface 406 may communicate data over a local-area-network (LAN), a wireless local-area-network (WLAN), a personal-area-network (PAN), a wide-area-network (WAN), an intranet, the Internet, a peer-to-peer network, a point-to-point network, or a mesh network.

[0043] In an aspect, the radar system 104 is implemented at least partially in hardware. Various implementations of the radar system 104 may include a system-on-chip (SoC), one or more integrated circuits (ICs), a processor with embedded processor instructions or configured to access processor instructions stored in memory, hardware with embedded firmware, a printed circuit board with various hardware components, or any combination thereof. The radar system 104 operates as a monostatic radar by transmitting and receiving its own radar signal. In some implementations, the radar system 104 may also cooperate with other radar systems 104 in an external environment to implement bistatic radar, multistatic radar, or network radar. However, constraints or limitations of the UE 102 may affect the design of the radar system 104. The UE 102 may have, for example, limited power available to operate the radar, limited computing power, size constraints, layout constraints, an external housing that attenuates or distorts the radar signal, etc. The radar system 104 includes several features that enable a high degree of radar functionality and performance to be achieved in the presence of these constraints, as further described below.

[0044] Before describing additional example ways in which the state manager 112 may act, consider FIG. 5, which illustrates a number of information, power, and access states in which the UE 102 may operate.

[0045] 5 illustrates the access states, information states, and power states in which the UE 102 may operate, each of which may be managed by the described techniques. These example levels and types of device state 500 are shown at three levels of granularity for visual simplicity, although many levels of each of the access state 502, power state 504, and information state 506 are possible. The access state 502 is shown at three example levels of granularity: a high access state 502-1, a medium access state 502-2, and a low access state 502-3. Similarly, the power state 504 is shown at three example levels of granularity: a high power state 504-1, a medium power state 504-2, and a low access state 502-3. and low power state 504-3. Similarly, information state 506 is shown at three exemplary levels of granularity: high information state 506-1, medium information state 506-2, and low information state 506-3.

[0046] More specifically, the access state 502 pertains to the access rights to data, applications, features, accounts, or components of the UE 102 that are available to the user of the device. This access may be elevated and is sometimes referred to as an “unlocked” state for the UE 102. This elevated access level may include simply the device's applications and features, or may also include access to various accounts, such as bank accounts, social media accounts, etc., that are accessible through the UE 102. Many computing devices, such as the UE 102, require authentication to provide elevated access, such as the elevated access state 502-1.

[0047] Various intermediate access levels (e.g., 502-2) may be granted by the UE 102, where such states allow the user to access some, but not all, of the UE 102's accounts, services, or components. An example includes allowing the user to take photos but not access previously captured photos. Another example includes allowing the user to answer calls but not access the contact list when making calls. These are just a few of the many intermediate rights, depicted by intermediate access states 502-2, that the UE 102 can grant. As depicted by the dashed boxes in FIG. 5, the high access level 502-1, or both the high and intermediate access levels 502-2, may be an authenticated state 508. The authenticated state 508 is an example of a state in which the user is authenticated, and therefore access (but not necessarily full access) is granted by the UE 102. Thus, the user is authenticated and then granted access. As noted throughout this document, the described techniques and systems enable greater security, where access to the UE 102 is easier for the user and access is more likely to be given to authorized users rather than third parties. Finally, the access state 502 may refrain from granting access, as shown as low access state 502-3. In this case, the device may be turned on, may send notifications such as alarms to wake the user, etc., but may not allow access to the functions of the UE 102 (or the UE 102 may simply be turned off, thereby not allowing access).

[0048] The power states 504 are illustrated at three exemplary levels of granularity: a high power state 504-1, an intermediate power state 504-2, and a low power state 504-3. The power states 504 relate to the amount of power to one or more components of the UE 102, such as the radar system 104, the display 116, or other power-consuming components, such as a processor, camera, microphone, voice assistant, touchscreen, sensors, radar, and components that are part of the authentication system 114 (which may also include the aforementioned components listed above). Terms such as power up, power up, increase power, and decrease power generally refer to the conditions for powering up a component, and to the power states 504, to the power management integrated circuit (PMIC) controlling the PMIC. managing power rails extending from the PMIC; opening and closing switches between the power rails, the PMIC, and one or more circuit components (e.g., the mentioned NIR components, camera, display, and radar); and providing supply voltages to operate the components correctly and safely, which may include increasing, decreasing, or distributing applied voltages or managing current inrush.

[0049] For the radar system 104, the power states 504 may be configured with different duty cycles. The power consumption can be reduced by collecting radar data (e.g., lower frequencies may use less power, higher frequencies may use more power), turning off various components when they are not active, or by adjusting power amplification levels. By doing so, the radar system 104 may use approximately 90 mW of power in a high power state 504-1, 30-60 mW in a medium power state 504-2, or less than 30 mW in a low power state 504-3 (e.g., the radar system 104 can operate at 2-20 mW while still providing some usable radar data, such as user presence). Each of these levels of power usage allows different resolution and range. Additional details regarding power management of the radar system 104 (and the UE 102) are described with reference to FIG. 6-1.

[0050] In the above-described state-changing situations, the state manager 112 may increase power to various components of the UE 102, such as the authentication system 114 or the display 116, from a lower power state (e.g., from the low power state 504-3 to the intermediate power state 504-2, or from either of these to the high power state 504-1), based on a determination by the radar manager 106 or the motion manager 110. By doing so, the UE 102 may engage or authenticate a user more quickly or easily. To this end, the state manager 112 may change the power state 504 to a higher or lower power than the current power for the UE 102's systems or for particular power-consuming entities associated with the UE 102. Example components are further described as part of FIG. 2 above and include powering up (or down) the face recognition unlock sensor 212 and its components, the NIR flood illuminator 214 and NIR dot projector 216, and the NIR cameras 218-1 and 218-2.

[0051] A third exemplary state of the UE 102 is the information state 506, which is illustrated by a high information state 506-1, a medium information state 506-2, and a low information state 506-3. More specifically, the information state 506 relates to the amount of information provided to a user, e.g., the user 120 of FIG. 1. In the context of notifications, the high information state 506-1 provides the highest level of information, generally assuming the UE 102 is unlocked or otherwise authenticated, or has a user preference for providing a high level of information even without authentication. Examples for the high information state 506-1 include showing the caller's name, number, and even an associated image when a call is received. Similarly, when a text or email or other type of message is received, the content is automatically presented through the display 116 or an audio speaker, a peripheral device, etc. This represents a high level of engagement, but the user's preferences can determine what level of engagement is required. It is assumed that there is some correlation between user engagement and the amount of information provided, and therefore, the technique can determine engagement and tailor the information presented accordingly. Examples of reduced information, such as intermediate information state 506-2, include presenting a ringtone when a call is received but not presenting the caller's name / identification information, or indicating that a text message or email has been received but not the entire message, showing only the subject, address, or some of the body content. Low information state 506-3 presents little or no information personally associated with user 120 but may include information that is generic or widely considered common knowledge or non-sensitive, such as the display 116 indicating the current date, time, weather conditions, battery power status, or that UE 102 is on. Other examples of low information state 506-3 include when a text message is received with simply an audible "ping" indicating that a message has been received, or a blank or black screen when a ringtone is received for a call but no caller's name, number, or other information is received.

[0052] 6-1 shows an example implementation 600 of a radar system 104. In the example 600, the radar system 104 includes at least one of each of the following components: a communication interface 602, an antenna array 604, a transceiver 606, a processor 608, and a system medium 610 (e.g., one or more computer-readable storage media). The processor 608 may be implemented as a digital signal processor, a controller, an application processor, another processor (e.g., the computer processor 402 of the UE 102), or some combination thereof. The system medium 610, which may be included within the computer-readable medium 404 of the UE 102 or separate from the computer-readable medium 404, includes one or more of the following modules: an attenuation mitigator 614, a digital beamformer 616, an angle estimator 618, or a power management module 620. These modules can compensate for or mitigate the effects of integrating the radar system 104 into the UE 102, thereby enabling the radar system 104 to recognize small or complex gestures, distinguish between different user orientations (e.g., "reach"), continuously monitor the external environment, or achieve a target false alarm probability. With these features, the radar system 104 can be implemented in a variety of different devices, such as the device shown in FIG.

[0053] Using the communication interface 602, the radar system 104 can provide radar data to the radar manager 106. The communication interface 602 can be a wireless or wired interface, depending on whether the radar system 104 is implemented remotely from the UE 102 or integrated within the UE 102. Depending on the application, the radar data can be raw data or minimally processed data, in-phase and quadrature (I / Q) data, range-Doppler data, or other data. , processed data including target location information (e.g., range, azimuth, elevation), clutter map data, etc. Generally, the radar data includes information usable by the radar manager 106 to provide the state manager 112 with the user's intent to engage, disengage, or remain engaged.

[0054] The antenna array 604 includes at least one transmit antenna element (not shown) and at least two receive antenna elements (as shown in FIG. 7 ). In some cases, the antenna array 604 may include multiple transmit antenna elements to implement a multiple-input multiple-output (MIMO) radar that can transmit multiple separate waveforms at once (e.g., a different waveform for each transmit antenna element). The use of multiple waveforms may increase the measurement accuracy of the radar system 104. For implementations that include three or more receive antenna elements, the receive antenna elements may be positioned in a one-dimensional shape (e.g., a line) or a two-dimensional shape. A one-dimensional shape allows the radar system 104 to measure one angular dimension (e.g., azimuth or elevation), while a two-dimensional shape allows two angular dimensions (e.g., both azimuth and elevation) to be measured. An exemplary two-dimensional arrangement of receive antenna elements is further described with respect to FIG. 7 .

[0055] 6-2 illustrates an example transceiver 606 and processor 608. The transceiver 606 includes multiple components that can be individually turned on or off via a power management module 620 according to the operational state of the radar system 104. Note that the power management module 620 may be separate, integrated, or under the control of the state manager 112, such as when the state manager 112 powers up or down components used to authenticate a user (e.g., the authentication system 114). The transceiver 606 includes active components 622, a voltage-controlled oscillator (VCO) and voltage-controlled buffer 624, a multi-channel oscilloscope (MACS) 626, a synchronous oscilloscope (SOS) 628 ... Plexor 626, analog-to-digital converter (ADC) 628, a phase lock loop (PLL) 630, and a crystal oscillator 63 6. The active components 622 are shown to include at least one of the following components: active component 622, active component 624, and active component 626. When turned on, each of these components consumes power even if the radar system 104 is not actively using these components to transmit or receive radar signals. Active components 622 may include, for example, an amplifier or a filter coupled to a supply voltage. The VCO 624 generates a frequency-modulated radar signal based on a control voltage provided by the PLL 630. The crystal oscillator 632 generates a reference signal for signal generation, frequency conversion (e.g., upconversion or downconversion), or timing operations within the radar system 104. By turning these components on and off, the power management module 620 enables the radar system 104 to rapidly switch between active and inactive operating states and conserve power during various inactive periods. These inactive periods may be on the order of microseconds (μs), milliseconds (ms), or seconds (s).

[0056] The processor 608 is shown to include multiple processors, such as a low-power processor 608-1 and a high-power processor 608-2, that consume different amounts of power. As an example, the low-power processor 608-1 may include a processor embedded within the radar system 104, while the high-power processor may include the computer processor 402 or some other processor external to the radar system 104. The difference in power consumption may be due to different amounts of available memory or computing power. For example, the low-power processor 608-1 may utilize less memory, perform fewer calculations, or utilize simpler algorithms than the high-power processor 608-2. Despite these limitations, the low-power processor 608-1 can process data for less complex radar-based applications, such as proximity detection or motion detection (based on radar data rather than inertial data). The high-power processor 608-2, in contrast, may utilize a large amount of memory, perform a large amount of calculations, or run complex signal processing, tracking, or machine learning algorithms. The high-power processor 608-2 can process data for high-profile radar-based applications such as gesture recognition (for authentication system 114), facial recognition, and provide accurate, high-resolution data through resolution of angle ambiguity or identification of multiple users and their characteristics.

[0057] To conserve power, the power management module 620 can control whether the low-power processor 608-1 or the high-power processor 608-2 is used to process radar data. In some cases, the low-power processor 608-1 can perform part of the analysis and pass the data to the high-power processor 608-2. Example data may include clutter maps, raw or minimally processed radar data (e.g., in-phase and quadrature data, or range-Doppler data), or digital beamformed data. The low-power processor 608-1 may also perform some low-level analysis to determine whether there is anything in the environment that the high-power processor 608-2 should analyze. In this way, power can be conserved by limiting the operation of the high-power processor 608-2 while utilizing the high-power processor 608-2 for cases where high-fidelity or accurate radar data is required by a radar-based application. Other factors that may affect power consumption within the radar system 104 are further described with respect to FIG. 6-1.

[0058] These and other capabilities and configurations, and the manner in which the entities in Figures 1, 2, 4, and 6-9 act and interact, are described in more detail below. may be further divided or combined. The environment 100 of FIG. 1 and the detailed illustrations of FIGS. 2-9 illustrate some of the many possible environments and devices in which the described techniques may be employed. FIGS. 6-9 illustrate additional details and features of the radar system 104. In FIGS. 6-9, the radar system 104 is described in the context of a UE 102, but as noted above, the applicability of the features and advantages of the described system and techniques is not necessarily so limited, and other embodiments involving other types of electronic devices may also be within the scope of the present teachings.

[0059] Figure 7 shows an example arrangement 700 of receive antenna elements 702. If the antenna array 604 includes, for example, at least four receive antenna elements 702, the receive antenna elements 702 may be arranged in a rectangular arrangement 704-1 as shown in the center of Figure 7. Alternatively, if the antenna array 604 includes at least three receive antenna elements 702, a triangular arrangement 704-2 or an L-shaped arrangement 704-3 may be used.

[0060] Due to size or layout constraints of the UE 102, the element spacing between the receive antenna elements 702 or the number of receive antenna elements 702 may not be ideal for the angle monitored by the radar system 104. In particular, the element spacing may result in angular ambiguity that makes it difficult for conventional radar to estimate the angular position of a target. Conventional radar may therefore limit its field of view (e.g., the angle to be monitored) to avoid ambiguous zones with angular ambiguity and thereby reduce false detections. For example, conventional radar may limit its field of view to an angle of approximately -45 degrees to 45 degrees to avoid the angular ambiguity that occurs using an 8 millimeter (mm) wavelength and 6.5 mm element spacing (e.g., the element spacing is 90% of the wavelength). Thus, conventional radar may not be able to detect targets beyond the 45-degree limit of its field of view. In contrast, radar system 104 includes a digital beamformer 616 and an angle estimator 618, which resolve angle ambiguities and enable radar system 104 to monitor angles beyond the 45-degree limit, such as angles from approximately -90 degrees to 90 degrees, or angles from approximately -180 degrees to 180 degrees. These angle ranges may apply across one or more directions (e.g., azimuth and / or elevation). Thus, radar system 104 can achieve low false alarm probabilities for a variety of different antenna array designs, including element spacings less than, greater than, or equal to half the central wavelength of the radar signal.

[0061] Using the antenna array 604, the radar system 104 can form beams that are steered or unsteered, wide or narrow, or shaped (e.g., as a hemisphere, cube, fan, cone, or cylinder). By way of example, one or more transmit antenna elements (not shown) may have an unsteered omnidirectional radiation pattern or may be capable of generating a wide beam, such as wide transmit beam 706. Either of these approaches allows the radar system 104 to illuminate a large volume of space. However, to achieve a desired angular accuracy and resolution, the receive antenna elements 702 and digital beamformer 616 may be used to generate thousands of narrow steered beams, such as narrow receive beam 708 (e.g., 3,000 beams, 7,000 beams, or 9,000 beams). In this manner, the radar system 104 can efficiently monitor the external environment and accurately determine the angle of arrival of reflections within the external environment.

[0062] Returning to FIG. 6-1, the transceiver 606 includes circuitry and logic for transmitting and receiving radar signals via the antenna array 604. Components of the transceiver 606 may include amplifiers, mixers, switches, analog-to-digital converters, filters, etc. for conditioning the radar signal. The transceiver 606 also performs in-phase modulation, such as modulation or demodulation. The transceiver 606 may include logic for performing I / Q (In-phase / Quadrature / Quadrature) operation. The transceiver 606 may be configured for continuous wave or pulsed radar operation. Various modulations may be used to generate the radar signal, including linear frequency modulation, triangular frequency modulation, stepped frequency modulation, or phase modulation.

[0063] The transceiver 606 can generate radar signals within a range of frequencies (e.g., a frequency spectrum), such as 1 gigahertz (GHz) to 400 GHz, 4 GHz to 100 GHz, or 57 GHz to 63 GHz. The frequency spectrum can be divided into multiple subspectrums with similar or different bandwidths. The bandwidth can be on the order of 500 megahertz (MHz), 1 GHz, 2 GHz, etc. As an example, the different frequency subspectrums may include frequencies from approximately 57 GHz to 59 GHz, 59 GHz to 61 GHz, or 61 GHz to 63 GHz. Multiple frequency subspectrums with the same bandwidth, which may be adjacent or non-adjacent, may also be selected for coherence. Multiple frequency subspectrums may be transmitted simultaneously or spaced apart in time, using a single radar signal or multiple radar signals. Adjacent frequency subspectrums allow the radar signal to have a wider bandwidth, while non-adjacent frequency subspectrums can further emphasize amplitude and phase differences that allow the angle estimator 618 to resolve angle ambiguities. The attenuation mitigator 614 or angle estimator 618 may cause the transceiver 606 to utilize one or more frequency subspectrums to improve the performance of the radar system 104, as further described with respect to Figures 8 and 9. Some embodiments of the approach are particularly advantageous, for example, when the UE 102 is a handheld smartphone and the radar signal is in the 57 GHz to 64 GHz band, with a peak equivalent isotropic radiated power (EIRP) in the range of 10 dBm to 20 dBm (10 mW to 100 mW) and an average power spectral density of approximately 13 dBm / MHz, which has been found to provide a reasonably sized "bubble" of radar detection (e.g., at least 1 meter in range, often up to 2 meters or more) in the vicinity of the smartphone and user while favorably addressing radiation health and coexistence issues. Within the bubble, the described method for authentication management via IMU and radar offers particularly significant time-saving advantages while conserving power.

[0064] The power management module 620 manages power usage to balance performance and power consumption. For example, the power management module 620 communicates with the radar manager 106 to cause the radar system 104 to collect data using predefined radar power states. Each predefined radar power state may be associated with a particular framing structure, a particular transmit power level, or particular hardware (e.g., the low-power processor 608-1 or the high-power processor 608-2 in FIG. 6-2). Adjusting one or more of these affects the power consumption of the radar system 104. However, reducing power consumption impacts performance such as gesture frame update rate and response delay, which are described below.

[0065] FIG. 6-3 illustrates an example relationship between power consumption, gesture frame update rate 634, and response delay. In graph 636, radar power states 638-1, 638-2, and 638-3 are associated with different levels of power consumption and different gesture frame update rates 634. The gesture frame update rate 634 represents how frequently the radar system 104 actively monitors the external environment by transmitting and receiving one or more radar signals. Generally speaking, power consumption is proportional to the gesture frame update rate 634. Thus, a higher gesture frame update rate 634 results in a greater amount of power being consumed by the radar system 104.

[0066] In graph 636, radar power state 638-1 utilizes the least amount of power, while radar power state 638-3 consumes the greatest amount of power. By way of example, radar power state 638-1 consumes power on the order of a few milliwatts (mW) (e.g., approximately 2 mW to 4 mW), while radar power state 638-3 consumes power on the order of tens of milliwatts (e.g., approximately 6 mW to 20 mW). With respect to gesture frame update rate 634, radar power state 638-1 uses an update rate on the order of a few hertz (e.g., approximately 1 Hz, or less than 5 Hz), while radar power state 638-3 uses a gesture frame update rate 634 on the order of tens of hertz (e.g., approximately 20 Hz, or more than 10 Hz).

[0067] Graph 640 represents the relationship between response delay and gesture frame update rate 634 for different radar power states 638-1 through 638-3. Generally speaking, response delay is inversely proportional to both the gesture frame update rate 634 and power consumption. In particular, response delay decreases exponentially while the gesture frame update rate 634 increases. The response delay associated with radar power state 638-1 may be on the order of hundreds of milliseconds (ms) (e.g., greater than 1000 ms, or 200 ms), while the response delay associated with radar power state 638-3 may be on the order of a few milliseconds (e.g., 50 ms, or less than 100 ms). For radar power state 638-2, the power consumption, gesture frame update rate 634, and response delay are between the values ​​for radar power state 638-1 and radar power state 638-3. For example, the power consumption of radar power state 638-2 is about 5 mW, the gesture frame update rate is about 8 Hz, and the response delay is about 100 ms to 200 ms.

[0068] Instead of operating in either radar power state 638-1 or radar power state 638-3, power management module 620 dynamically switches between radar power states 638-1, 638-2, and 638-3 (as well as sub-states between each of these radar power states 638) so that response delay and power consumption are both managed based on activity in the environment. As an example, power management module 620 activates radar power state 638-1 to monitor the external environment or detect an approaching user. Thereafter, if radar system 104 determines that the user has indicated an intent to engage, or may be beginning to do so, or may be beginning to make a gesture, power management module 620 activates radar power state 638-3. Different triggers may cause power management module 620 to switch between the different radar power states 638-1 through 638-3. Exemplary triggers include motion or lack of motion, a user appearing or disappearing, a user entering or leaving a designated area (e.g., an area defined by range, azimuth, or elevation), a change in the rate of motion associated with the user, an intent to engage (e.g., "reach," although some intents to engage require additional power, such as facial feature tracking), or a change in reflected signal strength (e.g., due to a change in radar cross section) as determined by the radar manager 106. In general, triggers indicating that the user is less likely to interact with the UE 102 or prefers to collect data using a longer response delay may activate radar power state 638-1 to conserve power.

[0069] In general, the power management module 620 determines when and how power can be conserved and gradually adjusts power consumption to allow the radar system 104 to operate within the power limits of the UE 102. In some cases, the power management module 620 may monitor the amount of remaining available power and adjust the operation of the radar system 104 accordingly (e.g., due to a low battery). For example, if the amount of remaining power is low, the power management module 620 may adjust the power consumption accordingly. The control module 620 may continue to operate in radar power state 638-1 instead of switching to either radar power state 638-2 or 638-3.

[0070] Each power state 638-1 through 638-3 may be associated with a particular framing structure. The framing structure specifies the configuration, scheduling, and signal characteristics associated with transmitting and receiving radar signals. Generally, the framing structure is set up so that appropriate radar data can be collected based on the external environment. The framing structure may be customized to facilitate the collection of different types of radar data for different applications (e.g., proximity detection, feature recognition, or gesture recognition). During inactive times across each level of the framing structure, the power management module 620 may turn off components within the transceiver 606 of FIG. 6-2 to conserve power. An example framing structure is further described with respect to FIG. 6-4.

[0071] FIG. 6-4 illustrates an exemplary framing structure 642. In the illustrated configuration, the framing structure 642 includes three different types of frames. At a top level, the framing structure 642 includes a sequence of gesture frames 644, which may be in an active or inactive state. Generally speaking, the active state consumes a greater amount of power compared to the inactive state. At an intermediate level, the framing structure 642 includes a sequence of feature frames (F), which may also be in an active or inactive state. The framing structure 642 includes a sequence of radar frames (RF) 646. Different types of feature frames include pulse mode feature frames 648 (shown in the bottom left of FIG. 6-4) and burst mode feature frames 650 (shown in the bottom right of FIG. 6-4). At a lower level, the framing structure 642 also includes radar frames (RF), which may also be in an active or inactive state. F) Contains 652 sequences.

[0072] The radar system 104 transmits and receives radar signals during active radar frames (RF) 652. In some cases, the radar frames 652 are individually analyzed for basic radar operations such as search and track, clutter map generation, determining user location, etc. The radar data collected during each active radar frame 652 may be stored in a buffer after the radar frame 652 is completed or may be provided directly to the processor 608 of FIG. 6-1.

[0073] The radar system 104 analyzes radar data across multiple radar frames 652 (e.g., across a group of radar frames 652 associated with an active feature frame 646) to identify specific features associated with one or more gestures. Exemplary types of features include specific types of movements, movements associated with specific appendages (e.g., a hand or individual fingers), and features associated with different parts of a gesture. To recognize a gesture performed by the user 120 during an active gesture frame 644, the radar system 104 analyzes radar data associated with one or more active feature frames 646.

[0074] Depending on the type of gesture, the duration of the gesture frame 644 may be on the order of milliseconds or seconds (e.g., approximately 10 ms to 10 s). After the active gesture frame 644 occurs, the radar system 104 becomes inactive as indicated by the inactive gesture frames 644-3 and 644-4. The duration of the inactive gesture frame 644 is characterized by a deep sleep time 654, which may be on the order of tens of milliseconds or more (e.g., greater than 50 ms). In an example implementation, the radar system 104 may turn off all components within the transceiver 606 to conserve power during the deep sleep time 654.

[0075] In the illustrated framing structure 642, each gesture frame 644 includes K feature frames 646, where K is a positive integer. When a gesture frame 644 is in an inactive state, all feature frames 646 associated with that gesture frame 644 are also in an inactive state. In contrast, an active gesture frame 644 includes J active feature frames 646 and KJ inactive feature frames 646, where J is a positive integer less than or equal to K. The number of feature frames 646 may be based on the complexity of the gesture and may include from a few to 100 feature frames 646 (e.g., K may be equal to 2, 10, 30, 60, or 100). The duration of each feature frame 646 may be on the order of milliseconds (e.g., approximately 1 ms to 50 ms).

[0076] To conserve power, the active feature frames 646-1 through 646-J occur before the inactive feature frames 646-(J+1) through 646-K. The duration of the inactive feature frames 646-(J+1) through 646-K is characterized by a sleep time 656. In this manner, the inactive feature frames 646-(J+1) through 646-K are executed consecutively such that the radar system 104 can be in a powered-down state for a longer duration compared to other approaches that interleave the inactive feature frames 646-(J+1) through 646-K with the active feature frames 646-1 through 646-J. Generally speaking, increasing the duration of the sleep time 656 allows the radar system 104 to turn off components within the transceiver 606 that require a longer startup time.

[0077] Each feature frame 646 includes L radar frames 652, where L is a positive integer that may or may not be equal to J or K. In some implementations, the number of radar frames 652 may vary among different feature frames 646 and may include a few frames or hundreds of frames (e.g., L may be equal to 5, 15, 30, 100, or 500). The duration of a radar frame 652 may be on the order of tens or thousands of microseconds (e.g., approximately 30 μs to 5 ms). The radar frames 652 within a particular feature frame 646 may be customizable for predetermined detection ranges, range resolutions, or Doppler sensitivities, which facilitate detection of particular features and gestures. For example, the radar frames 652 may utilize a particular type of modulation, bandwidth, frequency, transmit power, or timing. When a feature frame 646 is in an inactive state, all radar frames 652 associated with that feature frame 646 are also in an inactive state.

[0078] The pulse mode feature frame 648 and the burst mode feature frame 650 include different sequences of radar frames 652. Generally speaking, the radar frames 652 in the active pulse mode feature frame 648 transmit pulses that are spaced apart in time by a predetermined amount. In contrast, the radar frames 652 in the active burst mode feature frame 650 transmit pulses consecutively over a portion of the burst mode feature frame 650 (e.g., the pulses are not spaced apart by a predetermined amount of time).

[0079] Within each active pulse mode characteristic frame 648, a sequence of radar frames 652 alternates between active and inactive states. Each active radar frame 652 transmits a radar signal (e.g., a chirp), illustrated by a triangle. The duration of the radar signal is characterized by an active time 658. During the active time 658, components within the transceiver 606 are powered on. During a short idle time 660, which includes the remainder of the active radar frame 652 and the duration of the next inactive radar frame 652, the radar system 104 performs a short idle period 660. Power is conserved by turning off components within the transceiver 606 that have a wake-up time within the duration of dollar time 660.

[0080] The active burst mode feature frame 650 includes M active radar frames 652 and L M inactive radar frames 652, where M is a positive integer less than or equal to L. To conserve power, the active radar frames 652-1 through 652-M occur before the inactive radar frames 652-(M+1) through 652-L. The duration of the inactive radar frames 652-(M+1) through 652-L is characterized by a long idle time 662. By grouping the inactive radar frames 652-(M+1) through 652-L together, the radar system 104 can be in a powered-down state for a longer duration than the short idle time 660 that occurs during the pulse mode feature frame 648. Additionally, the power management module 620 can turn off additional components within the transceiver 606 that have wake-up times that are longer than the short idle times 660 but shorter than the long idle times 662.

[0081] Each active radar frame 652 in the active burst mode feature frame 650 transmits a portion of the radar signal. In this example, active radar frames 652-1 through 652-M alternate between transmitting portions of the radar signal with increasing frequencies and portions of the radar signal with decreasing frequencies.

[0082] The framing structure 642 allows power to be conserved through adjustable duty cycles within each frame type. A first duty cycle 664 is based on the number (J) of active feature frames 646 relative to the total number (K) of feature frames 646. A second duty cycle 665 is based on the number (e.g., L / 2 or M) of active radar frames 652 relative to the total number (L) of radar frames 652. A third duty cycle 668 is based on the duration of the radar signal relative to the duration of a radar frame 652.

[0083] Consider an exemplary framing structure 642 for power state 638-1, which consumes approximately 2 mW of power and has a gesture frame update rate 634 of approximately 1 Hz to 4 Hz. In this example, the framing structure 642 includes gesture frames 644 having durations of approximately 250 ms to 1 second. The gesture frames 644 include 31 pulse mode feature frames 648 (e.g., L equals 31). One of the 31 pulse mode feature frames 648 is in an active state. This results in a duty cycle 664 equal to approximately 3.2%. The duration of each pulse mode feature frame 648 is approximately 8 ms to 32 ms. Each pulse mode feature frame 648 is composed of eight radar frames 652. Within the active pulse mode feature frame 648, all eight radar frames 652 are in an active state. This results in a duty cycle 665 equal to 100%. Each radar frame 652 has a duration of approximately 1 ms to 4 ms. The active time 658 within each active radar frame 652 is approximately 32 μs to 128 μs. The resulting duty cycle 668 is therefore approximately 3.2%. This exemplary framing structure 642 has been found to produce good performance results. These good performance results relate to good gesture recognition and presence detection, and also produce good power efficiency results in handheld smartphone applications in low power states (e.g., low power state 504-3).

[0084] Based on the framing structure 642, the power management module 620 can determine times when the radar system 104 is not actively collecting radar data. Based on these periods of inactivity, the power management module 620 can adjust the operating state of the radar system 104 and one or more of the transceivers 606, as described further below. By turning off components, power can be conserved.

[0085] As mentioned, the power management module 620 can conserve power by turning off one or more components (e.g., a voltage-controlled oscillator, a multiplexer, an analog-to-digital converter, a phase-locked loop, or a crystal oscillator) within the transceiver 606 during inactive periods. These inactive periods occur when the radar system 104 is not actively transmitting or receiving radar signals, which may be on the order of microseconds (μs), milliseconds (ms), or seconds (s). The power management module 620 can also modify the transmit power of the radar signals by adjusting the amount of amplification provided by the signal amplifier. Additionally, the power management module 620 can control the use of different hardware components within the radar system 104 to conserve power. For example, if the processor 608 includes a lower-power processor and a higher-power processor (e.g., processors with different amounts of memory and computing power), the power management module 620 can switch between utilizing the lower-power processor for low-level analysis (e.g., detecting motion, determining a user's location, or monitoring the environment) and utilizing the higher-power processor for when high-fidelity or accuracy radar data is required by the radar manager 106 (e.g., to achieve a high-power state 504-1 of the authentication system 114 for authenticating a user using radar data).

[0086] In addition to the internal power conservation techniques described above, the power management module 620 can also conserve power within the UE 102 by activating or deactivating other external components or sensors within the UE 102, either alone or at the direction of the authentication system 114. These external components may include a speaker, camera sensor, global positioning system, wireless communication transceiver, display, gyroscope, or accelerometer. Because the radar system 104 can monitor the environment using a small amount of power, the power management module 620 can appropriately turn these external components on or off based on where the user is located or what the user is doing. In this way, the UE 102 can conserve power by using automatic shut-off timers or in seamless response to the user without the user having to physically touch or verbally control the UE 102.

[0087] FIG. 8 shows additional details of an example implementation 800 of the radar system 104 within the UE 102. In the example 800, the antenna array 604 is positioned under an external housing of the UE 102, such as a glass cover or external case. Depending on its material properties, the external housing may act as an attenuator 802, which attenuates or distorts radar signals transmitted and received by the radar system 104. The attenuator 802 may include different types of glass or plastic, some of which may be found within the display screen, external housing, or other components of the UE 102, and may have a dielectric constant (e.g., a relative permittivity) of approximately 4 to 10. Thus, the attenuator 802 is opaque or semi-transparent to the radar signal 806, which may cause a portion of the transmitted or received radar signal 806 to be reflected (as shown by the reflected portion 804). For conventional radar, the attenuator 802 may reduce the monitorable range, prevent small targets from being detected, or reduce overall accuracy.

[0088] Assuming that the transmit power of the radar system 104 is limited and that redesigning the external housing is undesirable, one or more attenuation-dependent characteristics of the radar signal 806 (e.g., frequency subspectrum 808 or steering angle 810) or attenuation-dependent characteristics of the attenuator 802 (e.g., distance 812 between the attenuator 802 and the radar system 104, or distance 813 between the attenuator 802 and the radar system 104) may be varied. The thickness 814 of the attenuator 802 is adjusted to mitigate the effect of the attenuator 802. Some of these characteristics may be set during manufacturing or adjusted by the attenuation mitigator 614 during operation of the radar system 104. The attenuation mitigator 614 may, for example, cause the transceiver 606 to transmit the radar signal 806 using a selected frequency subspectrum 808 or steering angle 810, cause the platform to move the radar system 104 closer to or farther from the attenuator 802 to change the distance 812, or prompt the user to apply another attenuator to increase the thickness 814 of the attenuator 802.

[0089] Appropriate adjustments can be made by the attenuation mitigator 614 based on predetermined characteristics of the attenuator 802 (e.g., characteristics stored in the computer-readable medium 404 of the UE 102 or in the system medium 610) or by processing the return of the radar signal 806 to measure one or more characteristics of the attenuator 802. Even if some of the attenuation-dependent characteristics are fixed or constrained, the attenuation mitigator 614 can take these limitations into account and balance each parameter to achieve the target radar performance. As a result, the attenuation mitigator 614 enables the radar system 104 to achieve increased accuracy and greater range for detecting and tracking users located on the other side of the attenuator 802. These approaches provide alternatives to increasing transmit power, which increases the power consumption of the radar system 104, or to changing the material properties of the attenuator 802, which can be difficult and expensive once the device begins production.

[0090] 9 shows an example scheme 900 implemented by radar system 104. Portions of scheme 900 may be performed by processor 608, computer processor 402, or other hardware circuitry. Scheme 900 is customizable to support different types of electronic devices and radar-based applications (e.g., radar manager 106) and also enables radar system 104 to achieve target angle accuracy despite design constraints.

[0091] The transceiver 606 generates raw data 902 based on the individual responses of the receive antenna elements 702 to the received radar signal. The received radar signal may be associated with one or more frequency subspectrums 904 selected by the angle estimator 618 to facilitate resolution of angle ambiguities. The frequency subspectrums 904 may be selected, for example, to reduce the amount of sidelobes or to reduce the amplitude of the sidelobes (e.g., reduce the amplitude by 0.5 dB, 1 dB, or more). The amount of frequency subspectrum may be determined based on the target angle accuracy or computational limitations of the radar system 104.

[0092] The raw data 902 includes digital information (e.g., in-phase and quadrature data) for a time period, different wavenumbers, and multiple channels, each associated with a receive antenna element 702. A Fast-Fourier Transform (FFT) 906 is performed on the raw data 902 to generate preprocessed data 908. The preprocessed data 908 includes digital information for different ranges (e.g., range bins) over the time period and for the multiple channels. A Doppler filtering process 910 is performed on the preprocessed data 908 to generate range-Doppler data 912. The Doppler filtering process 910 may include another FFT to generate amplitude and phase information for multiple range bins, multiple Doppler frequencies, and for the multiple channels. Based on the range-Doppler data 912, the digital beamformer 616 generates beamformed data 914. The beamforming data 914 includes digital information about a set of azimuth and / or elevation angles that represent a field of view over which different steering angles or beams are formed by the digital beamformer 616. Alternatively, the digital beamformer 616 may generate beamformed data 914 based on the preprocessed data 908, and the Doppler filtering process 910 may generate range-Doppler data 912 based on the beamformed data 914. To reduce computational complexity, the digital beamformer 616 may process portions of the range-Doppler data 912 or preprocessed data 908 based on range of interest, time, or Doppler frequency interval.

[0093] The digital beamformer 616 may be implemented using a single-look beamformer 916, a multi-look interferometer 918, or a multi-look beamformer 920. Generally, a single-look beamformer 916 may be used for deterministic objects (e.g., point-source targets with a single phase center). For non-deterministic targets (e.g., targets with multiple phase centers), a multi-look interferometer 918 or a multi-look beamformer 920 is used to improve accuracy compared to a single-look beamformer 916. A human being is an example of a non-deterministic target and has multiple phase centers 922 that can vary based on different aspect angles, as shown by 924-1 and 924-2. Variations in constructive or destructive interference generated by multiple phase centers 922 can make it difficult for conventional radar systems to accurately determine angular position. However, the multi-look interferometer 918 or the multi-look beamformer 920 performs coherent averaging to improve the accuracy of the beamformed data 914. The multi-look interferometer 918 performs coherent averaging of the two channels to produce phase information that can be used to accurately determine angular information, while the multi-look beamformer 920 performs Fourier, Capon, multiple signal classification (MUSIC), or other beamformers. This is called minimum variance distortion less response (MVDR). A linear or nonlinear beamformer can be used to perform coherent averaging of two or more channels. The increased accuracy provided via the multi-look beamformer 920 or multi-look interferometer 918 allows the radar system 104 to recognize small gestures or distinguish between multiple parts of a user (e.g., facial features).

[0094] The angle estimator 618 analyzes the beamforming data 914 to estimate one or more angular positions. The angle estimator 618 may utilize signal processing techniques, pattern matching techniques, or machine learning. The angle estimator 618 also resolves angle ambiguities that may arise from the design of the radar system 104 or the field of view monitored by the radar system 104. An exemplary angle ambiguity is shown in an amplitude plot 926 (e.g., amplitude response).

[0095] The amplitude plot 926 shows the amplitude difference that can occur for different angular positions of the target and for different steering angles 810. A first amplitude response 928-1 (shown as a solid line) is shown for a target positioned at a first angular position 930-1. Similarly, a second amplitude response 928-2 (shown as a dotted line) is shown for a target positioned at a second angular position 930-2. In this example, the difference is considered to span angles from -180 degrees to 180 degrees.

[0096] As shown in the amplitude plot 926, there are ambiguity zones for two angular positions 930-1 and 930-2. The first amplitude response 928-1 has a highest peak at the first angular position 930-1 and a lower peak at the second angular position 930-2. The highest peak corresponds to the actual position of the target, while the lower peak ambiguizes the first angular position 930-1 because it is within some threshold where a conventional radar may not be able to determine with certainty whether the target is at the first angular position 930-1 or the second angular position 930-2. In contrast, the second amplitude response 928-2 has a lower peak at the second angular position 930-2 and a higher peak at the first angular position 930-1. In this case, the lower peak corresponds to the actual position of the target. corresponds to the location of

[0097] While conventional radars may be limited to using the highest peak amplitude to determine angular position, the angle estimator 618 instead analyzes subtle differences in the shape of the magnitude responses 928-1 and 928-2. Shape characteristics may include, for example, roll-off, width of the peak or null, angular position of the peak or null, height or depth of the peak and null, shape of the sidelobes, symmetry within the magnitude responses 928-1 or 928-2, or lack of symmetry within the magnitude responses 928-1 or 928-2. Similar shape characteristics may also be analyzed in the phase responses, which can provide additional information for resolving angle ambiguities. The angle estimator 618 thus maps unique angular signatures or patterns to angular positions.

[0098] The angle estimator 618 may include a suite of algorithms or tools that may be selected according to the type of UE 102 (e.g., computational capabilities or power constraints) or the target angle resolution for the radar manager 106. In some implementations, the angle estimator 618 may include a neural network 932, a convolutional neural network (CNN) 934, or a long short-term memory (LSTM) network 936. The neural network 932 may have various depths or numbers of hidden layers (e.g., three hidden layers, five hidden layers, or ten hidden layers) and may include different numbers of connections (e.g., the neural network 932 may include a fully connected neural network or a partially connected neural network). In some cases, the CNN 934 may be used to increase the computation speed of the angle estimator 618. The LSTM network 936 may be used to enable the angle estimator 618 to track targets. Using machine learning techniques, the angle estimator 618 employs a nonlinear function to analyze the shape of the magnitude response 928-1 or 928-2 to generate angle probability data 938, which indicates the likelihood that a user or portion of a user is located within an angle bin. The angle estimator 618 may provide angle probability data 938 for a small number of angle bins, such as two angle bins, to provide the probability that a target is to the left or right of the UE 102, or may provide angle probability data 938 for thousands of angle bins (e.g., to provide angle probability data 938 for continuous angle measurements).

[0099] Based on the angular probability data 938, the tracker module 940 generates angular position data 942, which identifies the angular position of the target. The tracker module 940 may determine the angular position of the target based on the angular bin with the highest probability in the angular probability data 938 or based on predictive information (e.g., previously measured angular position information). The tracker module 940 may also track one or more moving targets to enable the radar system 104 to confidently distinguish or identify the targets. Other data, including range, Doppler, velocity, or acceleration, may also be used to determine the angular position. In some cases, the tracker module 940 may include an alpha-beta tracker, a Kalman filter, a multiple hypothesis tracker (MHT), etc.

[0100] A quantizer module 944 takes the angular position data 942 and quantizes the data to generate quantized angular position data 946. The quantization may be based on a target angular resolution for the radar manager 106. In some cases, fewer quantization levels may be used so that the quantized angular position data 946 indicates whether a target is to the right or left of the UE 102, or identifies the 90-degree quadrant in which the target is located. This may be sufficient for some radar-based applications, such as user proximity detection. In other cases, the quantized angular position data 946 may be More quantization levels can be used to indicate the angular position of the target within an accuracy of, for example, a fraction of a degree, one degree, five degrees, etc. This resolution can be used for higher resolution radar-based applications such as gesture recognition, or in attentional or interaction state implementations as described herein. In some implementations, the digital beamformer 616, angle estimator 618, tracker module 940, and quantizer module 944 are implemented together in a single machine learning module.

[0101] Implementations include those in which radar is used to determine a user's intent to engage, disengage, or remain engaged, and those in which radar is used to detect user actions classified as indicative of the user's intent to engage or interact with an electronic device (any of these implementations may alternatively be achieved using the on-device camera found in most modern smartphones). Among the advantages of the described implementations is that, while the power usage of a radar system is substantially less than that of a camera system, the correctness of the results can often be better with a radar system than with a camera system. For example, using the above-described radar system 104, desired user intent detection can be achieved with average power in the range of a few milliwatts to tens of milliwatts (e.g., 10 mW, 20 mW, 30 mW, or 40 mW), including processing power for processing radar vector data to make a decision. At these low power levels, it would be readily acceptable to have the radar system 104 always enabled. So, for example, if the smartphone radar system 104 is always enabled, the desired enjoyable and seamless experience described herein can still be provided to a user who is sitting in a room away from their smartphone for long periods of time.

[0102] In contrast, optical cameras in most today's smartphones typically operate at hundreds of milliwatts of power (e.g., an order of magnitude higher than 40 mW, i.e., 400 mW). At such power rates, optical cameras would be at a disadvantage because they would significantly reduce the battery life of most today's smartphones, making it highly impractical, if not prohibitive, to keep the optical camera on all the time. An additional advantage of radar system 104 is that its field of view can be quite large, even when resting face-up flat on a table (for many typical implementations in which the radar chip generally faces outward in the same direction as the selfie camera), so that it can easily detect a user walking toward it from any direction, and furthermore, its Doppler processing capabilities can make it very effective (especially at operating frequencies near 60 GHz) at detecting even relatively subtle movements of a moving body from various directions.

[0103] Additionally, the radar system 104 can operate in environments where the performance of a camera system is reduced or limited. For example, in lower light environments, a camera system may have a reduced ability to detect shape or motion. In contrast, the radar system 104 functions in low light as well as in full light. The radar system 104 can also detect presence and gestures through some obstructions. For example, if a smartphone is in a jacket or pants pocket, the camera system cannot detect the user or gestures. However, the radar system 104 can still detect objects in the radar field even through fabric that would block the camera system. An additional advantage of using the radar system 104 over a smartphone's built-in video camera system is privacy, as users can enjoy the enjoyable and seamless experience described herein without having to worry about a video camera capturing their footage for such purposes.

[0104] The entities of Figures 1, 2, 4, and 6-9 may be further divided, combined, or used with other sensors or components. Thus, different implementations of the UE 102 having different configurations of the radar system 104 and IMU 108 may be used to implement authentication management via the IMU and radar. The example operating environment 100 of Figure 1 and the detailed illustrations of Figures 2-9 illustrate only a few of the many possible environments and devices in which the described techniques may be employed.

[0105] Exemplary Methods This section presents exemplary methods that may operate separately or together in whole or in part. Various exemplary methods are described, each of which is described in subsections for ease of reading, but the titles of these subsections are not intended to limit the interoperability of each of these methods with other methods.

[0106] Authentication Management FIG. 10 illustrates an exemplary method 1000 for managing authentication via an IMU and radar, and is an example of managing power states for user equipment. Method 1000 is illustrated as a set of blocks that identify operations to be performed, but are not necessarily limited to the order or combination shown for performing the operations by each block. Also, any one or more of the operations may be repeated, combined, rearranged, or linked to provide a wide range of additional and / or alternative methods (e.g., methods 1200 and 1400). In portions of the description that follows, reference may be made to entities or processes as detailed in the exemplary operating environment 100 of FIG. 1 or other figures, and such references are made for illustrative purposes only. The techniques are not limited to execution by one entity or multiple entities operating on one device.

[0107] At 1002, a user intent to engage is determined based on the radar data and by the user equipment. The intent to engage indicates that the user intends to engage with the user equipment. As described above, the intent to engage may be indicated by determining that the user 120 is reaching out toward the UE 102, looking at the UE 102, or leaning or orienting their body toward the UE 102, to name just a few examples.

[0108] At 1004, instead of or in addition to determining intent to engage through radar data, user equipment movement is determined based on inertial data, which may indicate the user 120 picking up the UE 102, touching the UE 102, and other movements as described above.

[0109] At 1006, in response to the determination of intent to engage and, optionally, a determination of movement of the user device, a power state of a power-consuming component of the authentication system is changed. The power state of the power-consuming component is changed from a first power state to a second power state, the second power state consuming more power than the first power state. This change may be based solely on intent to engage determined using radar data, or may also be via movement determined through inertial data. Further, based on the movement determination, the power state of the power-consuming component may be further increased, or other components may be further powered. As described above, this movement determination may confirm the user 120's intent to engage, may provide intent to engage, or may otherwise add speed and / or robustness to the determination, adding power, resources, etc. to the authentication system. Note that, in some cases, components of the authentication system may be powered on even if it is not determined that the user intends to engage. In such a case, the technique acts to perform the authentication process in response to a determined intent to engage. In such a case, latency is reduced even if power is not saved for the process. However, the technique may refrain from using resources not associated with the authentication system, thereby saving power in other ways.

[0110] The power state to which the power-consuming components of the authentication system are changed may or may not be sufficient to enable the authentication system to perform the authentication process for the user. In some cases, the second power state of the power-consuming components is not the high power state 504-1. In such cases, the second power state is the intermediate power state 504-2 as described above. This intermediate power state 504-2 is sometimes sufficient for the performance of a power-consuming component, such as a camera, that includes an intermediate power state that can still provide sensor data for authentication without fully powering up (e.g., capturing an image of a user in sufficient light rather than darkness). Another example is a display 116 that can be powered to accept touch input for a password without powering the display's luminosity to full power. Another case includes a radar system 104 where full power is not required to provide sufficiently accurate facial characteristics to the authentication system 114 when the user's face is within a fairly close range of the radar system 104.

[0111] In some cases, powering up a component is an intermediate step, such as a warm-up sequence, that may prepare the component by giving it additional time or may simply reduce latency. In such cases, the state manager 112 may determine not to proceed to high power if, for example, it determines an intent to disengage by the user 120 moving the UE 102 (e.g., into a pocket) before the component is ready to authenticate, thereby preventing authentication. In some cases, powering up is an intermediate step that is subsequently fully powered up to sufficient power to conduct the authentication process in response to a determination that the user 120 has moved the UE 102, as indicated at 1004. This warm-up sequence powers the component to an intermediate power state 504-2, and then, after some short period of time, the component is fully powered up (e.g., to the high power state 504-1) to be used in the authentication process. In such cases, the component is at high power (or nearly high power) while in the post-warm-up sequence that follows the warm-up sequence. For components that consume significant power if left on when not needed, but require a significant amount of time to power up, such as some infrared (IR) or near-infrared (NIR) sensors, an intermediate power state in which a warm-up sequence occurs can save significant power or reduce significant latency that could detract from the user experience.

[0112] Exemplary power-consuming components of an authentication system are described above and include, for example, the facial recognition unlock sensor 212, the touchscreen of the display 116, the radar system 104, and the processor 608 (e.g., high-power processor 608-2) of the authentication system 114 of Figure 1. For specific details of the many potential power-consuming components of a facial recognition system for authentication, see Figure 2 and its discussion.

[0113] At 1008, an authentication process is performed by the authentication system. In doing so, the authentication system 114 uses the power consuming components in an altered power state, such as the second power state or a higher third power state. The authentication process is effective to authenticate the user or to determine that the user is not authenticated and should not be allowed access to the UE 102. As mentioned, the authentication process may be performed using techniques such as facial recognition, fingerprint reading, etc. The authentication process may be via a touch or voice interface (e.g., a touchscreen data entry component of the display 112), such as through entry of a password or other authentication information. The authentication process compares identifying features of the user or authentication information with some secure storage of equivalent features or authentication information to determine the user's identity as authentic and therefore authorized to access the UE 102. This may be as simple as comparing a six-digit password entered through the display's touchscreen, or may require greater computational and system complexity, such as determining facial features based on sensor data received from a power-consuming component and comparing the determined facial features to a facial feature library. Although not required, this facial feature library may be stored locally from the UE 102 and created during facial feature initialization by the UE 102 with the authentication system 114. Additionally, this library may be securely stored at the UE 102, such as embedded on a secure chip integral with the UE 102. This is one way in which the privacy of the user 120 can be maintained.

[0114] Throughout this disclosure, examples are described in which a computing system (e.g., UE 102, client device, server device, computer, or other type of computing system) analyzes information associated with a user (e.g., radar data, inertial data, and facial recognition sensor data), such as facial features in the just-mentioned operation 1008. However, the computing system may be configured to use the information only after the computing system receives explicit permission to use the data from a user of the computing system. For example, when UE 102 analyzes sensor data for facial features to authenticate user 102, individual users may be given the opportunity to provide input to control whether programs or features of UE 102 can collect and use the data. Individual users may have some control over which programs can or cannot use the sensor data. Additionally, collected information may be preprocessed in one or more ways to remove personally identifiable information before it is transferred, stored, or otherwise used. For example, before the UE 102 shares sensor data with another device (e.g., to train a model running on the other device), the UE 102 may preprocess the sensor data to ensure that any user-identifying or device-identifying information embedded in the data is removed. Thus, a user may have control over whether information is collected about the user and their device, and if collected, how such information may be used by computing devices and / or remote computing systems.

[0115] Returning to method 1000, at 1010, alternatively or additionally, the power state of the display is changed in response to a determination that the user equipment has moved or is moving. This change may be increasing power to the display's touch input reception capability or simply changing the visual presentation of the display. One example includes adding a luminosity to the display 116 when the user touches the UE 102 so that the user sees that the UE 102 recognizes the user's intent and is thus possibly preparing to engage the user 120. Similarly, the UE 102 may do so in response to an intent to engage determined at 1002.

[0116] In some cases, the authentication process is performed for some period of time or repetitions (e.g., some preset number of times or period) without success. In such cases, method 1000 can continue by re-performing the authentication process, as shown at 1012, in response to the movement determination at 1004, or the process can continue. This alternative is indicated in FIG. 10 using some of the dashed arrows.

[0117] At 1014, in response to the user's authentication process at 1008 (or retry at 1012) being successful, the user is authenticated and the access state of the UE 102 is changed. This change can raise the UE 102's access from a non-, low-, or medium-access state to a high-access state, in which case the UE 102 is "unlocked." However, this high-access state (e.g., high-access state 502-1 in FIG. 5) is not required. Several levels of authentication can reserve access, power, or information for subsequent authentication. Examples include authenticating the user for use of some, but not all, of the UE 102's applications and / or accounts (e.g., music purchasing accounts, bank accounts, etc.) and requiring additional authentication for those reserved-access accounts and applications. For example, in addition to the high-access state 502-1, the state manager 112 can cause the UE 102 to be placed in a high-information state 506-1. Examples of this change to the information state include presenting the last engaged application or web page that contains the last engaged portion, such as page 4 of a 10-page article on a web page, or the middle of a song or video, playing the location where the user 120 last engaged or authenticated to the UE 102. The state manager 112 may quickly and seamlessly change these states in response to authentication of the user 120.

[0118] As an example, consider one implementation of method 1000 applied to scenario 1100 shown in FIG. 11 . Scenario 1100 includes five parts, each chronologically following the previous part. In the first part of scenario 1100, designated scenario portion 1100-1, user 1102 is not looking at, touching, or otherwise interacting with smartphone 1104. Assume here that smartphone 1104 is in low access state 502-3, low power state 504-3, and low information state 506-3 (e.g., smartphone 1104 appears to be “off” but has sufficient power to determine intent to engage). This scenario portion 1100-1 is assumed to be a state prior to the operation of the method at 1002 in FIG. 10 . The second part, designated 1100-2, is during which user 1102 faces smartphone 1104 and looks at it, but does not interact with it. In this regard, the technique determines, at operation 1002, based on radar data, that the user 1102 intends to engage with the smartphone 1104. This intent to engage is determined without using a reaching motion, but instead based on the user 1102 looking toward the smartphone 1104 and orienting their body toward the smartphone 1104. The technique makes this determination at operation 1002 through the radar manager 106, which passes the determination to the state manager 112. Following this, the state manager 112 changes the power state of a power-consuming component (the facial recognition unlock sensor 212) of the authentication system 114 at operation 1006. Note that this occurs well before the user reaches for or picks up the smartphone 1104, reducing latency in getting the authentication system 114 ready to authenticate the user.

[0119] Also, assume that over the next 0.5 seconds, while the power-consuming components are powering up, the user 1102 moves closer to the smartphone 1104 and extends his hand toward the smartphone 1104 (the extending hand is indicated by hand 1106). This is shown in third portion 1100-3. At this point, the authentication system 114 performs the authentication process (operation 1008), but assume that the authentication process is unsuccessful after several iterations and / or over a period of time. The approach may abort the attempt to authenticate the user 1102, thereby conserving power. However, now, as shown in portion 1100-4, the user 1102 touches the smartphone 1104. This is operation 1004, and through inertial data sensed by the IMU 108 of FIG. 1, the movement of the smartphone 1104 is It is determined that the user 1102 has touched the smartphone 1104. This motion determination is passed to the state manager 112. Based on this motion, the state manager 112 causes the authentication system 114 to continue attempting to authenticate the user 1102, as indicated by operation 1012 of method 1000. Further, at operation 1010, also based on the motion, the state manager 112 illuminates the display 1108 of the smartphone 1104. This illumination or powering up of the display 1108 may occur in scenario portions 1100-2, 1100-3, or 1100-4, but here is indicated in response to determining that the user 1102 has touched the smartphone 1104 (along with time and notification information at 1110). Doing so provides feedback to the user 1102 that the smartphone 1104 recognizes that the user intends to engage.

[0120] As will be described, state manager 112 causes authentication system 114 to continue the authentication process and authenticate user 1102 through these continued attempts. This is shown in portion 1100-5 and results in smartphone 1104 being in different states: high access state 501-1, high power state 504-1, and high information state 506-1, with high access state 502-1 indicated by display 1108 presenting unlock icon 1112. These state levels can be automatically elevated by state manager 112 to provide a seamless user experience for user 1102.

[0121] In this example scenario 1100, the inertial data provided by the IMU 108 causes the state manager 112 to confirm with a higher level of confidence that the user 1102 intends to engage with the smartphone 1104 and therefore wishes to be authenticated, thus justifying additional power. This is just one example scenario that shows how inertial data from an IMU and radar data from a radar system can be used to authenticate a user quickly, easily, and with reduced power consumption.

[0122] Reducing high-level conditions FIG. 12 illustrates an example method 1200 for mitigating a high-level condition via an IMU and radar. Method 1200 is illustrated as a set of blocks that identify operations to be performed, but are not necessarily limited to the order or combination shown for performing the operations by each block. Also, any one or more of the operations may be repeated, combined, rearranged, or linked to provide a wide range of additional and / or alternative methods, including other methods described in this document (e.g., methods 1000 and 1400). In portions of the description that follow, reference may be made to entities or processes as detailed in the example operating environment 100 of FIG. 1 or other figures, and such references are made for illustrative purposes only. The techniques are not limited to execution by one entity or multiple entities operating on one device.

[0123] Optionally, at 1202 and prior to operations 1204 or 1206, an inactivity period is determined to have expired. In contrast to some other conventional approaches that rely solely on the expiration of a period, method 1200 may use or refrain from using an inactivity period to reduce a high-level state for the user equipment. While this inactivity timer is not required, the use of a timer, even a short timer, can potentially save power. More specifically, the inactivity timer starts when the last user action on the user equipment is received, e.g., the last touch on a touchscreen or button, the last voice command, or the last gesture input received by the user equipment. Note that while some conventional approaches use only a timer, and thus conventional timers often last several minutes (e.g., 1 minute, 3 minutes, 5 minutes, or 10 minutes), method 1200 can use a relatively short period, such as 0.5 seconds, 1 second, 3 seconds, 5 seconds, 10 seconds, or 20 seconds. Doing so reduces the possibility that the user equipment may disclose information, making inappropriate access available. While the reliability is very low, the use of short periods of inactivity can operate to save some amount of power by refraining from performing the operations of 1204 and / or 1206 during periods of inactivity.

[0124] At 1204, motion is determined during a high-level state of the user equipment where the user is interacting or has recently interacted with the user equipment. The motion manager 110 determines this motion based on inertial data received from the IMU 108 associated with the UE 102. As indicated using the dashed arrow, this action may optionally be in response to actions 1206 and / or 1202 (not shown). This determined motion may be one or more of the various motions described above, such as a motion indicating the user 120 picking up the UE 102, walking with the UE 102, putting the UE 102 down, placing it in a pocket or casing, or simply being near or touching the UE 102. In some cases, the motion manager 110 determines that the motion is sufficient or not sufficient to change the state of the UE 102 and passes the determination accordingly to the state manager 112. Examples include those described above that do not exceed a threshold motion, those caused by ambient vibrations, and those that are not sufficient changes to ongoing motion while moving. Thus, the motion manager 110 may determine that the UE 102 is moving because the user 120 is walking with the UE 102, but the motion may not be a sufficient change to indicate that the user 120 may disengage from the UE 102. Another way to look at this is that motion may be based on a change, not just the current motion of the UE 102. An example change includes moving and then not moving, e.g., the user walking with the UE 102 and placing it on a table. While the inertial data from the IMU 108 may not capture the user 120 placing the UE 102 on the table, a determination that the inertial data shows little or no motion when there was immediately prior motion (the user 120 walking with the UE 102) may still be determined to be motion in operation 1204 based on this immediately prior motion.

[0125] More particularly, the technique can align the state of the user equipment with the user's engagement. Thus, in some cases, the user equipment is in a high-level state due to the user being highly engaged with the user equipment. For example, method 1200 may determine that the user is interacting with the user equipment prior to operations 1204 or 1206. This determination of user engagement may be based on prior radar data indicating the user's intent to engage, on voice or touch input from the user, on commands or input received from the user through a voice or touch sensor, on a successful authentication process, etc.

[0126] At 1206, an intent to disengage is determined based on the radar data and by the user equipment. The radar manager 106 receives radar data from the radar system 104 and uses the radar data to determine whether the user intends to disengage from the UE 102. This intent to disengage may include various types discussed above, such as the user 120 withdrawing their hand from the UE 102, a facial orientation change relative to the UE 102, the user 120 turning their face away from the UE 102 or turning their back to the UE 102, etc.

[0127] As indicated with the dashed arrow, this operation 1206 may optionally be responsive to operation 1204 (and / or 1202, not shown). In these cases, the state manager 112 or radar manager 106 acts to conserve power by refraining from determining the user's 120 intent to disengage until motion is determined, and doing the opposite for the motion determination at 1204. Doing so can conserve power. Thus, the power management module 620 may employ a technique to keep the radar system 104 at reduced power until motion is determined at 1204. Once movement is determined, the state manager 112 causes the power management module 620 to power up the radar system 104 in preparation to determine if the user 120 is acting in a manner that indicates an intent to disengage.

[0128] At 1208, in response to the determination of movement and / or intent to disengage, the user equipment's high-level state is reduced to an intermediate or low-level state. See exemplary high-level state 1208-1, which may be one or more states involving access, power, or information, such as those shown in FIG. 5 (high access state 502-1, high power state 504-1, or high information state 506-1). In response to the determination of movement, intent to disengage, or both, the state manager 112 determines to reduce one or more of the UE 102's states. This is illustrated in FIG. 12 using arrows indicating a reduction from the high level 1208-1 to the intermediate level 1208-2 or low level 1208-3. These are just two of the various granularities of power, access, and information. 5, the intermediate level 1208-2 and the low level 1208-3 include the intermediate access state 502-2, the intermediate power state 504-2, and the intermediate information state 506-2, each of which is described above. The low level 1208-3 is exemplified by three lower states: the low access state 502-3, the low power state 504-3, and the low information state 506-3. These states are described in detail above. Note that any one, two, or all three of these states can be lowered to the same or different levels by the state manager 112 in operation 1208. Thus, the state manager 112 may lower the high access state 502-1 to an intermediate state or a low state and maintain the power and information states at high or mixed levels. Similarly, the state manager 112 may reduce the power state 504 to a lower power state 504-3 while keeping the UE 102 in the high access state 502-1 (eg, an "unlocked" state).

[0129] As an example, consider the application of method 1200 to scenario 1300 shown in FIG. 13. Scenario 1300 includes three parts, each chronologically following the previous part. Assume that prior to the first part of scenario 1300, user 1302 is actively engaged with smartphone 1304, and smartphone 1304 is in a high-level state, i.e., a high power state, a high access state, and a high information state. In the first part, shown as scenario portion 1300-1, user 1302 walks up to a table and places smartphone 1304 on the table. At operation 1204, IMU 108 receives inertial data about smartphone 1304 contacting the tabletop, or a lack of inertial data if the inertial data indicated movement (based on user 1302 walking with smartphone 1304) prior to being placed on the tabletop. Based on either or both of these inertial data, the motion manager 110 determines the motion for the smartphone 1304 and passes this determination to the radar manager 106 and / or the state manager 112 .

[0130] Assume that radar manager 106 provides, or has provided, radar field 118 (not shown for visual simplicity; see, e.g., FIG. 1 ) immediately in response to the motion data, thus receiving radar data indicating, for example, the body position of user 1302. Based on this radar data, radar manager 106 determines that user 1302 does not intend to disengage with body, arm, and hand placement motion 1206 in scenario portion 1300-1 over the first iteration (and possibly multiple other iterations). This is because user 1302 has a body orientation toward smartphone 1304, and the user's hands and arms are oriented toward smartphone 1304. Therefore, high-information state 1306-1 is not changed.

[0131] However, in scenario portion 1300-2, after approximately 2 seconds, user 1302 picks up his coffee cup and walks away with his back to smartphone 1304. Assume that the user 1302 begins to leave. At this point, the radar manager 106 determines that the user 1302 intends to disengage from the smartphone 1304 based on the fact that the user's 1302 body orientation is partially facing away from the smartphone 1304 and that the user's 1302 arms and hands are directed toward the coffee cup rather than the smartphone 1304. The radar manager 106 passes this determination to the state manager 112.

[0132] In operation 1208, in response to receiving the movement determination and the intent to disengage, state manager 112 reduces the information state of smartphone 1304 from high information state 1306-1, shown in scenario portion 1300-1, to intermediate information state 1306-2. These example information states are illustrated by the information displayed in scenario portion 1300-1 showing content consisting of two text messages and the time of day. As soon as user 1302 turns around and picks up the coffee cup, the information state is reduced to intermediate information state 1306-2, shown with the time of day and reduced information about the text messages (the sender's name is shown, but not the context). This intermediate amount of information may be useful to user 1302 because the user may change their mind about engagement or may want to look at smartphone 1304 again to see if a new notification, such as a text from another person, has arrived.

[0133] In addition to or instead of indicating the intermediate information state 1306-2, and as part of operation 1208, the state manager 112 may progress to a lower level, either immediately or after first reaching the intermediate state. Now assume that the state manager 112, in response to an additional determination by the radar manager 106 indicating that the user 1302 intends to disengage or has a higher confidence level thereof (e.g., shown here with high confidence because the user 1302 is now several meters away and has his back turned completely toward the smartphone 1304), further reduces the information state to a lower information state 1306-3, shown as scenario portion 1300-3 presenting only the current time.

[0134] While this example shows a change to information state, access and power may be changed as well or instead. This is indicated in part using an unlock icon 1310 shown in scenario portion 1300-1, which indicates a high level of access (e.g., high level access 502-1 in FIG. 5). In scenario portion 1300-2, after state manager 112 receives the movement data and the intent to disengage, state manager 112 reduces access to a low level, which is indicated to the user using a lock icon 1312. Additionally, power state may be changed in scenario portions 1300-2 and / or 1300-3, such as by reducing the brightness of the smartphone 1304's display (not shown).

[0135] Staying authenticated FIG. 14 illustrates an example method 1400 for maintaining an authenticated state. Method 1400 is illustrated as a set of blocks that identify operations to be performed, but are not necessarily limited to the order or combination shown for performing the operations by each block. Also, any one or more of the operations may be repeated, combined, rearranged, or linked to provide a wide range of additional and / or alternative methods, including other methods described in this document (e.g., methods 1000 and 1200). In portions of the description that follows, reference may be made to entities or processes as detailed in the example operating environment 100 of FIG. 1 or other figures, and such references are made for illustrative purposes only. The techniques are not limited to execution by one entity or multiple entities operating on one device.

[0136] Before describing method 1400, it should be noted that any of the methods described above may be combined, in whole or in part, with method 1400. For example, consider the performance of method 1000 of FIG. 10 . Method 1000 describes one example of authentication management resulting in authentication of a user of user equipment. In response to this authentication, the user equipment enters an authenticated state. This state is described in more detail above. Thus, method 1000 (or some other aspect of authentication of a user) may precede method 1400.

[0137] At 1402, while the user equipment is in an authenticated state, potential disengagement by the user of the user equipment is determined. This determination of potential user disengagement may include determining the user's intent to disengage, as described above, as well as other determinations described below. Also, as described above, the authenticated state permits the user to access one or more of the user equipment's data, applications, features, accounts, or components. Examples of authenticated states include the high access state 502-1 and the medium access state 502-2 described above in FIG. 5. Both of these access states may be permitted by the UE 102 when in the authenticated state (often based on user preferences or operating system default settings), but the authenticated state presumes the user's previous authentication. However, user-selected preferences or settings may permit high access or medium access of the UE 102 without authentication. Thus, although the authenticated state may include access permitted by the high access and medium access states described above, high access and medium access are not necessarily authenticated states.

[0138] 14 , the determination of potential disengagement may optionally be made in response to (or through performing) operation 1404 or operation 1406, as well as other aspects described herein, such as through determining an intent to disengage at operation 1206 of method 1200. At 1404, the expiration of an inactivity period is determined. As described above, this inactivity period may begin when the last user action is received, active engagement with the user equipment ends (or is last received), or when a last intent to engage is determined. For example, an inactivity timer (e.g., a period) begins when the user last touches a touch-sensitive display or button, last received voice command is spoken, or last determined touch-independent gesture (e.g., a gesture determined using radar system 104 described above) is performed.

[0139] At 1406, motion of the user equipment is determined based on inertial data from an inertial measurement unit (IMU) associated with the user equipment. Exemplary motion and inertial data are described above, such as the inertial data received from the IMU 108 of FIG. 1. Thus, determining motion is one way in which the method may determine that the user has potentially disengaged, such as by placing the UE 102 in a locker, bag, or pocket (although placing in a bag or pocket may later be determined to be passive engagement, as described below).

[0140] At 1408, passive engagement by the user with the user equipment is determined based on the radar data. This determination of passive engagement may be responsive to the determination of potential disengagement at 1402 (indicated using a dashed arrow), or it may be independent of or coincident with that determination. Performing operation 1408 in response to the determination of potential disengagement may, in some cases, save power or reduce latency. For example, method 1400 may power up components of radar system 104 (see also FIGS. 6-1 and 6-2) in response to the determination of potential disengagement. This may save power, as discussed above, or may prepare radar system 104 to determine whether the user is passively engaged with radar system 104. Additional time may be granted.

[0141] 1 , the radar manager 106 determines that the user 120 is passively engaged with the UE 102. This passive engagement may be determined by the radar manager 106 in multiple ways, which may be exclusive or overlapping with each other. For example, the radar manager 106 may determine that the user is passively engaged based on radar data that indicates that the user's 120 hands are holding the user equipment 102 in an orientation that maintains the display 116 of the user equipment 102. Thus, if the user 120 holds the UE 102 steady (or steady enough to view content or show content to others), the user 120 is passively engaged. Other examples of determining passive engagement are described above and include the user 120 looking at the UE 102 or orienting their body toward the UE 102.

[0142] Additionally, the radar manager 106 can determine passive engagement based on radar data indicating the presence of the user 120, such as by the user being within two meters of the UE 102. Other distances, such as 1.5 meters, 1 meter, or even 0.5 meters, may be used as well or instead. In practice, the radar manager 106 can determine that the user 120 is passively engaged by the user generally being within arm's reach of the UE 102. The radar manager 106 may explicitly determine that the user 120 is passively engaged by indicating that the user 120 is passively engaged, or may simply pass information indicating the distance from the UE 102 to the state manager 112. The state manager 112 then determines passive engagement based on the proximity of the user 120 and, possibly, other people (or the absence of other people), whether the user 120 is in a vehicle (car, bus, train), whether the user 120 is at a desk, etc. For example, a user sitting in their home may have a greater permitted distance than a user sitting in a crowded coffee shop or on a train.

[0143] At 1410, in response to a determination of passive engagement by the user with the user equipment, the authenticated state is maintained. This maintenance of the authenticated state can continue until another potential disengagement is determined or for a period of time, after which method 1400 can be performed again. An example of an authenticated state is the high access state 502-1 of FIG. 5. In many cases, this authenticated state is an unlocked state for the UE 102, but in some other cases, the authenticated state allows some, but not all, access to the UE 102, such as the medium access state 502-2 described above.

[0144] Maintaining an authenticated state for the UE 102 does not require that other states be maintained. For example, if the user 120 is within two meters of the UE 102 but is not known to be looking at or oriented toward the UE 102, the state manager 112 may reduce the power or information state of the UE 102 from, for example, the high power state 504-1 and high information state 506-1 set forth in FIG. 5 to an intermediate or low power or information state. However, if passive engagement includes the user looking at the UE 102, the power or information state may also be maintained, such as to continue presenting content to the user 120 through the display 116.

[0145] Optionally, method 1400 may proceed to operation 1412, where the presence or intent to engage of non-users is determined based on radar data. This radar data may be the same radar data on which the passive engagement was based, or may be subsequently received radar data, such as radar data from radar system 104 received several seconds or minutes later than the radar data on which the passive engagement was based. Thus, at 1412, radar manager 106 determines whether non-users are present or intend to engage. Determine presence or intent to engage with the UE 102. Thus, if a non-user reaches out to the UE 102 or looks at the display 116 of the UE 102, the radar manager 106 may determine this presence or intent and pass it on to the state manager 112.

[0146] At 1414, maintaining the authenticated state is discontinued in response to determining that a non-user is present or intends to engage with the user equipment. Thus, if a non-user walks up to, reaches out to, or looks at the display 116 of the UE 102, the state manager 112 discontinues maintaining the authenticated state (or actively de-authenticates the UE 102). Along with this discontinuation, the state manager 112 may also degrade other states, such as information states, that are effective to reduce or eliminate information presented to non-users. For example, assume an authenticated user is reading a private email on a subway train. If a person sitting behind the user looks at the display, perhaps to read the private email, the state manager 112 can lock the UE 102 and discontinue displaying the private email. This can be done quickly and seamlessly, further enhancing the user's privacy.

[0147] At 1416, optionally, after ceasing to maintain the authenticated state, the method may return to the authenticated state in response to determining that the non-user is no longer present or no longer intends to engage. Continuing with the example above, when a non-user on a subway train looks away from the display 116 of the UE 102, the state manager 112 may re-authenticate the user 120 through an authentication process or simply by switching to an authenticated state without re-authentication. This allows the user 120 to easily return to the previous state once the condition that caused the de-authentication ceases. While some authentication processes, such as the systems and processes described herein, are quick and power-efficient, not performing an authentication process may be quicker and more power-efficient. Upon returning to the authenticated state, the state manager 112 may return the information state to the previous level with content consistent with the content last presented to the user 120. In this example, when the non-user looks away, the display 116 presents private emails in the same location as they were last presented to the user 120 by the UE 102. Doing so provides the user with seamless management of authentication and improved information privacy. Note that selections by the user 120, such as the user's selection to de-authenticate, can override the operation of the technique. In some cases, the user 120 simply turns off the UE 102, which is permitted by the methods described herein.

[0148] Consider another example, illustrated through scenario 1500 in Figure 15. Scenario 1500 includes four parts. In a first part 1500-1, assume that a user 1502 has been authenticated to a smartphone 1504, such as through authentication information or facial feature analysis, and thus the smartphone 1504 is in an authenticated state 1506. This authenticated state 1506 allows the user 1502 to access the smartphone 1504, as illustrated through the user 1502 accessing content on the smartphone 1504 by watching a television program about a volcanic eruption.

[0149] Scenario 1500 is shown branching along two different paths. In one path, an inactivity timer starts when user 120 stops touching or providing input to smartphone 1504, in this case, when the user relaxes and watches a television program. In another case, the inactivity timer may or may not start, and its expiration would not be necessary to determine potential disengagement. Thus, in scenario portion 1500-2, after three minutes of inactivity, the inactivity timer expires. Returning to FIG. 14, operation 1402 determines that potential disengagement by the user has occurred due to the expiration of the inactivity period in operation 1404. For the second path shown in scenario portion 1500-3, operation 1402 determines that a potential disengagement by the user has occurred by determining that a movement of smartphone 1504 has occurred based on inertial data through performing operation 1406. The movement is caused by user 1502 placing his or her foot on the edge of the table on which smartphone 1504 rests.

[0150] In response to any of these determinations of potential disengagement, the radar manager 106 determines, based on the radar data, that the user 1502 is passively engaged with the smartphone 1504. This operation occurs at 1408. Now, assume that it is determined that the user 1502 is present or that the user 1502 is looking at the smartphone 1504, either of which indicates that the user 1502 is passively engaged.

[0151] In response, the state manager 112 maintains the authenticated state at operation 1410. This can all be done seamlessly and without the user 1502 realizing that it has happened. As shown in scenario portion 1500-4, the smartphone 1504 simply continues to present the television program through either path.

[0152] Consider another scenario 1600 in Figure 16, which may follow scenario 1500 or may be an alternative, independent scenario. Scenario 1600 includes three scenario portions. In a first scenario portion 1600-1, a user 1502 is watching a television program about volcanoes, similar to that shown in Figure 15, but now marked in content 1602 on smartphone 1504. During this presentation of the program, smartphone 1504 is in an authenticated state, such as authenticated state 1506 described in Figure 15.

[0153] However, in scenario portion 1600-2, non-user 1604 sits on the couch with user 1502. Because non-user 1604 is a colleague of user 1502, user 1502 turns to face non-user 1604 and begins to speak to him. As discussed above, these actions of user 1502, such as turning, speaking, or both, may be considered potential disengagement. If considered a potential disengagement by user 1502, state manager 112 reduces the state of smartphone 1504, such as by reducing the access state or information state (e.g., acts 1206 and 1208 of method 1200) as described in FIGS. 5 and 12 .

[0154] Assume, however, that radar manager 106 determines, through operation 1412 of method 1400 and based on the radar data, the presence of non-user 1604. Based on this presence of non-user 1604, state manager 112 ceases maintaining the authenticated state 1506 after state manager 112 previously acted to maintain the authenticated state of smartphone 1504 (e.g., through operation 1410 shown in FIG. 15 ). Thus, state manager 112 may demote smartphone 1504 to the unauthenticated state 1604, shown in the expanded view of scenario portion 1600-2. This change is indicated to user 1502 through lock icon 1606 and by ceasing presentation of content 1602.

[0155] In scenario portion 1600-3, non-user 1604 walks away and user 1502 again sees smartphone 1504. Radar manager 106 determines that non-user 1604 is no longer present and indicates this determination to state manager 112, which then returns smartphone 1504 to authenticated state 1506. Note that state manager 112 may also require a determination that user 1502 intends to engage with smartphone 1504, or may simply return to authenticated state based on non-user 1604 having left smartphone 1504. Also, as described in this document, The approach described above allows the user to seamlessly return to where they left off, thereby providing a superior user experience. This is illustrated in FIG. 16 by the state manager 112 returning the smartphone 1504 to the same or nearly the same point in the same television program that was last presented to the user 1502. For some embodiments, the approach allows the user to direct, in a setup screen or similar device configuration screen, whether the smartphone 1504 will return to an authenticated state in response to determining in step 1416 that a non-user is no longer present or does not intend to engage, or whether the smartphone 1504 will remain in an unauthenticated state until a more rigorous authentication process (e.g., step 1006 described above) using a power-consuming component of the authentication system is performed. In other words, the approach can provide a user-selectable setting through a setup configuration or similar device configuration that, once a non-user trace is present, keeps the smartphone 1504 deauthenticated, even if the trace is no longer there.

[0156] example The following sections provide examples.

[0157] Example 1 1. A method comprising: determining, based on radar data and by user equipment, an intent to engage, wherein the intent to engage indicates that a user intends to engage with the user equipment; the method further comprising, in response to determining the intent to engage, changing a power state of a power consuming component of an authentication system from a first power state to a second power state, wherein the second power state consumes more power than the first power state; and performing, by the authentication system and using the power consuming component in the second power state or a third, higher power state, an authentication process, wherein the authentication process is effective to authenticate the user.

[0158] Example 2 10. The method of claim 1, wherein the intent to engage is determined based on radar data indicating the user is reaching out toward the user device, looking at the user device, and / or leaning toward the user device.

[0159] Example 3 3. The method of any one of Examples 1 and 2, further comprising: determining a motion of the user equipment based on motion data received through an inertial measurement unit integral with the user equipment; and, in response to the determination of the motion of the user equipment, effectively changing a power state of the display to change the visual presentation of the display and / or to change the touch input reception capability of the display.

[0160] Example 4 3. The method of any one of Examples 1 and 2, wherein the authentication process occurs without user authentication for a first period of time or a preset number of iterations, and the method further includes continuing the authentication process after the first period of time or the preset number of iterations in response to determining movement of the user equipment based on movement data received through an inertial measurement unit integral with the user equipment.

[0161] Example 5 The method of any one of Examples 1 to 4, wherein the authentication system includes a facial recognition sensor, and the power-consuming component of the authentication system is the facial recognition sensor.

[0162] Example 6 6. The method of example 5, wherein the facial recognition sensor includes a camera and at least one infrared or near-infrared emitter, projector, or sensor.

[0163] Example 7 7. The method of Example 6, wherein performing the authentication process includes determining facial features of the user using a reflected signal sensed by the camera under low or no ambient light conditions, the reflected signal comprising an infrared or near-infrared reflection of an infrared or near-infrared signal provided by an infrared or near-infrared light emitter.

[0164] Example 8 7. The method of Example 6, wherein performing the authentication process includes using a reflected signal sensed by the camera to determine a depth map of the user's facial features, the reflected signal comprising an infrared or near-infrared reflection of an infrared or near-infrared signal provided by an infrared or near-infrared projector.

[0165] Example 9 The method of Examples 5-8, wherein the face recognition sensor includes a radar system, and the radar data is received from the radar system, and the power consuming component of the recognition system is the radar system.

[0166] Example 10 The method of any one of Examples 1-9, wherein the authentication system includes a touchscreen data entry component, and the power-consuming component of the authentication system is the touchscreen data entry component.

[0167] Example 11 11. The method of any one of Examples 1-10, further comprising, in response to determining an intent to engage based on the radar data, changing power of a display of the user equipment from a first power state to a second power state, the second power state consuming more power than the first power state.

[0168] Example 12 The method of any one of Examples 1-11, wherein the second power state of the power consuming components of the authentication system is insufficient to enable the authentication system to perform an authentication process for the user, and the method further includes changing the power state of the power consuming components of the authentication system from the second power to a third power state in response to a determination based on the inertial data and by the user equipment that the user equipment is being moved or moved by the user, and the third power state is sufficient to enable the authentication system to perform the authentication process.

[0169] Example 13 The method of any one of Examples 1 to 11, wherein the second power state of the power consuming component of the authentication system includes a warm-up sequence for the power consuming component, the warm-up sequence including a period of time during which the power supply to the power consuming component is insufficient to perform an authentication process, and the third power state includes a post-warm-up sequence during which the power consuming component is sufficiently powered to enable the authentication system to perform an authentication process for the user, and the authentication process is performed using the power consuming component in the third higher power state.

[0170] Example 14 The method of any one of Examples 1 to 11, wherein the authentication process performed by the authentication system determines facial features based on sensor data received from the power-consuming component and compares the determined facial features with a locally stored facial feature library from the user equipment, the facial feature library being created during facial feature initialization by the user equipment having the authentication system.

[0171] Example 15 An apparatus comprising: a radar system configured to provide a radar field; an authentication system; and a processor and memory system coupled to the radar system and the authentication system and configured to perform a method according to any one of Examples 1 to 14.

[0172] Example 16 An apparatus comprising means for carrying out the method according to any one of Examples 1 to 14.

[0173] conclusion Although implementations of techniques for identity management via IMUs and radars and apparatus enabling such identity management have been described in feature and / or method specific language, it should be understood that claimed subject matter is not necessarily limited to the particular features or methods described. Rather, the particular features and methods are disclosed as example implementations enabling identity management via IMUs and radars.

Claims

1. 1. A method comprising: determining, based on the radar data and by the user equipment, an intent to engage, wherein the intent to engage indicates that a user intends to engage with the user equipment, the method further comprising: and in response to the determination of intent to engage, changing a power state of a power consuming component of the authentication system from a first power state to a second power state, the second power state consuming more power than the first power state, the method further comprising:

22. The method of claim 21, further comprising: performing an authentication process by the authentication system and using the power consuming component in the second power state or a third, higher power state, the authentication process being effective to authenticate the user.

2. 10. The method of claim 1, wherein the intent to engage is determined based on the radar data indicating the user is reaching out toward, looking at, and / or leaning toward the user device.

3. determining motion of the user device based on motion data received through an inertial measurement unit associated with the user device; 3. The method of claim 1, further comprising: in response to determining the movement of the user equipment, effectively changing a power state of a display to change a visual presentation of the display and / or to change an ability of the display to receive touch input.

4. 3. The method of claim 1, wherein the authentication process is performed for a first period or a preset number of iterations without authentication of the user, and the method further includes continuing the authentication process after the first period or the preset number of iterations in response to determining movement of the user equipment based on movement data received through an inertial measurement unit integral with the user equipment.

5. The method of any one of claims 1 to 4, wherein the authentication system includes a facial recognition sensor, and the power consuming component of the authentication system is the facial recognition sensor.

6. The method of claim 5 , wherein the facial recognition sensor includes a camera and at least one infrared or near-infrared emitter, projector, or sensor.

7. 7. The method of claim 6, wherein performing the authentication process includes determining facial features of the user using a reflected signal sensed by the camera under low or no ambient light conditions, the reflected signal comprising an infrared or near-infrared reflection of an infrared or near-infrared signal provided by the infrared or near-infrared light emitter.

8. 7. The method of claim 6, wherein performing the authentication process includes determining a depth map of the user's facial features using reflected signals sensed by the camera, the reflected signals comprising infrared or near-infrared reflections of infrared or near-infrared signals provided by the infrared or near-infrared projector.

9. The face recognition sensor includes a radar system, the radar data is received from the radar system, and the power consuming components of the authentication system are connected to the radar system. The method according to any one of claims 5 to 8, wherein the system is a

10. The method of any one of claims 1 to 9, wherein the authentication system includes a touchscreen data entry component, and the power consuming component of the authentication system is the touchscreen data entry component.

11. 11. The method of claim 1, further comprising: in response to determining the intent to engage based on the radar data, changing power of a display of the user equipment from a first power state to a second power state, the second power state consuming more power than the first power state.

12. 12. The method of claim 1, wherein the second power state of the power consuming components of the authentication system is insufficient to enable the authentication system to perform the authentication process for the user, and the method further comprises changing the power state of the power consuming components of the authentication system from the second power state to the third power state in response to a determination based on inertial data and by the user equipment that the user equipment is being moved or is being moved by the user, the third power state being sufficient to enable the authentication system to perform the authentication process.

13. 12. The method of claim 1, wherein the second power state of the power consuming component of the authentication system comprises a warm-up sequence for the power consuming component, the warm-up sequence comprising a period during which the power supply to the power consuming component is insufficient to perform the authentication process, and the third power state comprises a post-warm-up sequence during which the power consuming component is sufficiently powered to enable the authentication system to perform an authentication process for the user, the authentication process being performed using the power consuming component in the third, higher power state.

14. 12. The method of claim 1, wherein the authentication process performed by the authentication system comprises determining facial features based on sensor data received from the power consuming component and comparing the determined facial features with a facial feature library stored locally from the user equipment, the facial feature library being created during facial feature initialization by the user equipment having the authentication system.

15. a radar system configured to provide a radar field; an authentication system; An apparatus comprising: a processor and memory system coupled to said radar system and said authentication system, said processor and memory system configured to perform the method of any one of claims 1 to 14.

Images