Radar-based authentication status feedback

Abstract

This document describes technologies and systems for enabling radar-based authentication state feedback. A radar field (110) is used to enable an electronic device (102) to take into account a user's remote physical cues to determine and maintain awareness of the user's location and movement around the device. This awareness allows the device to predict some of the user's expected interactions and provide functionality in a timely and seamless manner, such as preparing an authentication system (118) to authenticate the user before the user touches the device or speaks to the device. These features also allow the device to provide visual feedback that can help the user understand that the device is aware of the user's location and movement. In some cases, feedback is provided using visual elements (1004-1010, 1104, 1106, 1204, 1206) presented on a display (114).

Description

Background Art

[0001] Electronic devices are used to communicate, store information and photos, and conduct business and financial transactions. Because of the type and amount of information stored on electronic devices, many devices turn off the display and lock access to the device after a certain period of time without active user interaction or based on some other criteria. Once an electronic device is locked, typically only authorized users can unlock and access the device. Therefore, electronic devices often also include technology that allows the device to authenticate the user (e.g., to determine that the user is an authorized user) before granting the user access to the device.

[0002] Depending on how long a user accesses their device per session or how frequently they interact with it, users may have to authenticate multiple times per day (according to some studies, a typical user accesses their electronic device more than 50 times per day). Typically, authentication requires the user to perform a specific interaction on a touchscreen, memorize and type a complex password, or hold their finger against a sensor long enough for the device to detect the fingerprint. Some electronic devices use cameras and facial recognition algorithms to attempt to reduce the complexity of the authentication process, but these methods may require additional time for operations such as camera warm-up. Furthermore, even with facial recognition, the user typically must first pick up the device and aim the camera towards their face for the device to recognize that the user wishes to be authenticated and access the device. Only after aiming the user's face can the electronic device prepare the camera or other authentication processes and grant the user's access request. Until the user physically interacts, the electronic device remains locked and unaware of the user's intent. As a result, any user experience provided by the electronic device before the user picks it up is often bland and lacks richness. Summary of the Invention

[0003] This document describes technologies and systems that enable radar-based authentication status feedback. These technologies and systems use radar fields to enable electronic devices, such as smartphones, to accurately determine the presence or absence of a user and, further, to determine the user's actions and movements, which have been categorized as actions indicating the user's intent to interact with the electronic device. Using these technologies, electronic devices can consider the user's physical cues at a distance to determine and maintain awareness of the user's position and movements within their environment. This awareness, in turn, allows the electronic device to anticipate certain intended user interactions and provide functionality in a timely and seamless manner, such as preparing the authentication system to authenticate the user before the user touches or speaks into the electronic device. This awareness and ability to determine user intent also allows the electronic device to provide feedback that helps the user understand that the electronic device is aware of the user and their position and movements relative to the electronic device. This feedback can educate the user about what the electronic device is aware of and allow the user to take advantage of the additional functionality and features provided by these technologies. In some cases, feedback is provided using visual elements presented on the electronic device's display.

[0004] Aspects described below include an electronic device comprising a display, a radar system, one or more computer processors, and one or more computer-readable media. The radar system is implemented at least partially in hardware and provides a radar field. The radar system also senses reflections from a user in the radar field, analyzes the reflections from the user in the radar field, and provides radar data based on the reflection analysis. The one or more computer-readable media include stored instructions executable by the one or more computer processors to implement a radar-based awareness manager. The radar-based awareness manager maintains the electronic device in a sleep mode. The awareness manager also determines the presence of a user within a recognition area of ​​the electronic device based on a first subset of the radar data, and in response to determining the presence of the user within the recognition area, causes the electronic device to enter an awareness mode. In the awareness mode, the display presents a first visual element indicating a first state of the electronic device. The awareness manager also detects a user action categorized as indicative of a user intent to interact with the electronic device based on a second subset of the radar data. In response to detecting the user action associated with the user intent to interact with the electronic device, the awareness manager causes the electronic device to prepare an authentication system to perform an authentication process. Based on a triggering event, the awareness manager also causes the prepared authentication system to perform an authentication process on the user. In response to the user being authenticated, the awareness manager causes the electronic device to enter an active mode.

[0005] Aspects described below also include a method implemented in an electronic device, the electronic device including a radar system and a radar-based awareness manager. The method includes providing a radar field by the radar system and sensing, by the radar system, reflections from objects in the radar field. The method also includes analyzing the reflections from objects in the radar field and providing radar data based on the reflection analysis. The method also includes maintaining the electronic device in a sleep mode by the radar-based awareness manager. The method also includes determining the presence of a user within a recognition area of ​​the electronic device based on a first subset of the radar data, and in response to determining the presence of the user within the recognition area, causing the electronic device to enter an awareness mode. In the awareness mode, a display presents a first visual element indicating a first state of the electronic device. The method also includes detecting a user action classified as indicating a user intent to interact with the electronic device based on a second subset of the radar data. The method further includes, in response to detecting the user action classified as indicating a user intent to interact with the electronic device, causing the electronic device to prepare an authentication system to perform an authentication process. The method also includes causing, by the radar-based awareness manager and based on a triggering event, the prepared authentication system to perform an authentication process on the user, and in response to the user being authenticated, causing the electronic device to enter an active mode by the radar-based awareness manager.

[0006] Aspects described below include a system comprising a display and an electronic device, the electronic device comprising or being associated with a device for providing a radar field that provides radar data based on sensing and analyzing reflections from objects in the radar field. The system also includes a device for maintaining the electronic device in a first mode. The system also includes a device for determining the presence of a user within a recognition area of ​​the electronic device based on the radar data and, in response to determining the presence of the user within the recognition area, causing the electronic device to enter a second mode, wherein the display presents a first visual element indicating a first state of the electronic device. The system also includes a device for detecting a user action based on the radar data that is classified as an indication of a user intent to interact with the electronic device, and, in response to detecting the user action associated with the user intent to interact with the electronic device, causing the electronic device to prepare an authentication system to perform an authentication process. The system also includes a device for causing the prepared authentication system to perform an authentication process on the user based on a triggering event. The system also includes a device for causing the electronic device to enter an active mode in response to the user being authenticated.

[0007] This summary is provided to introduce simplified concepts regarding radar-based authentication status feedback, which is further described in the detailed description below and the accompanying figures. This summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] This document describes details of one or more aspects of radar-based authentication status feedback with reference to the following figures. Like numbers are used throughout the figures to reference similar features and components:

[0009] Figure 1 Illustrated is an example environment in which techniques for enabling radar-based authentication status feedback may be implemented.

[0010] Figure 2 An example embodiment of an electronic device including a radar system that can implement radar-based authentication status feedback is illustrated.

[0011] Figure 3 Graphic Figure 1 and 2 Example implementation of a radar system.

[0012] Figure 4 Graphics for Figure 3 Example arrangement of receiving antenna elements of a radar system.

[0013] Figure 5 Graphic Figure 1 and Figure 2 Additional details of an example implementation of a radar system.

[0014] Figure 6 The diagram can be Figure 1 and Figure 2 Example scenario for radar system implementation.

[0015] Figures 7 to 9 An example method of enabling radar-based authentication status feedback is described.

[0016] Figures 10 to 12 Illustration of an example of a display operating in multiple modes with visual elements that can be presented in different modes on a display Figure 1 and Figure 2 electronic equipment.

[0017] Figures 13 to 17 When the gesture subscription application is running on the electronic device, it can be Figure 1 and Figure 2 Additional visual elements presented on the display of an electronic device.

[0018] Figure 18 The diagram can be implemented as shown in the reference Figures 1 to 17 Any type of client, server, and / or electronic device described, or example computing systems in which techniques for using radar-based authentication state feedback may be implemented. DETAILED DESCRIPTION

[0019] Overview

[0020] This document describes technologies and systems that enable radar-based authentication status feedback. As described above, authentication systems and processes used to maintain electronic device security can adversely impact the user experience by requiring multiple, time-consuming authentications during user interaction with the electronic device. Even when using less complex authentication processes such as facial recognition, operations such as camera warm-up and initiating the facial recognition program consume additional time. This problem is particularly frustrating for users when the device locks access after some period of time when the user is not actually inactive but merely passively interacting.

[0021] The described techniques and systems employ a radar system that can determine a user's intent to interact or not interact with an electronic device based on the user's location, the user's orientation relative to the device, user gestures, or other factors. These techniques allow electronic devices to provide a more timely and efficient authentication process by predicting when the user is ready to authenticate and more accurately determining when to lock the device when the user is away. Because the described techniques allow electronic devices to determine the user's intent, interactions can be more convenient and less frustrating, as the electronic device can anticipate the user's intent and enter the appropriate mode or state for interacting and receiving input.

[0022] In addition, the radar system can detect implicit user interactions with the electronic device, such as presence, location, and movement relative to the electronic device. Detection of implicit interactions enables the electronic device to provide feedback and prompts that inform the user of the electronic device's level of awareness regarding the implicit interaction and corresponding functionality.

[0023] Consider an example in which the electronic device is a smartphone that includes the described radar system and other components. In this example, the user can place the smartphone on the user's table or desk while the user performs other tasks. The awareness manager can maintain the electronic device in a sleep mode that reduces power consumption (e.g., by turning off the display or other sensors or operating the radar system at a lower sampling rate), but allows the radar system to detect the user's presence within a recognition zone around the electronic device. The recognition zone is the area around the electronic device. The size and shape of the recognition zone can be preset or user-selectable and can be dynamically adjusted. In some cases, the electronic device can determine the size and / or shape of the recognition zone based on a number of relevant factors, such as the type of electronic device, battery level, location of the electronic device, speed of the electronic device, or data received from one or more of the radar system, other sensors, or applications running on the electronic device. When the radar system detects that the user is within the recognition zone, the awareness manager can automatically transition the smartphone from sleep mode to awareness mode.

[0024] In awareness mode, the display presents visual elements that can help the user understand the awareness level and functionality of the smartphone and improve the user's experience with the smartphone. For example, when the user enters a recognition area and the smartphone transitions to awareness mode, the display may present elements such as the time, date, connection status indicator, battery level indicator, or other elements that may be predetermined or user-selectable. In some cases, if the smartphone is running an application that can receive input via radar-based remote, three-dimensional gestures, the display may present a visual element that indicates the application's availability to receive that type of gesture input.

[0025] The awareness manager can also detect user actions that indicate the user's intention to interact with the smartphone. For example, user actions such as turning, leaning over, or reaching out to the smartphone can be classified as indications of the user's intention to interact with the smartphone. In this case, when the radar system detects that the user reaches out to the smartphone (e.g., using radar data), the awareness manager can determine that the reach out is an indication of the user's intention to interact with the smartphone and automatically transition from awareness mode to engagement mode. Conversely, if the awareness manager determines that the user is in the recognition area but no user actions are detected that are classified as indications of the user's intention to interact with the smartphone, the awareness manager can maintain the smartphone in awareness mode until the awareness manager detects an action that is classified as an indication of the user's intention to interact. In other cases, without the need for an action to be classified as an indication of the user's intention to interact, the awareness manager can maintain the smartphone in awareness mode for an adjustable duration and then transition back to sleep mode, even if the user remains in the recognition area.

[0026] In engagement mode, the awareness manager can present additional or different visual elements on the display compared to awareness mode, which can help the user understand the changed awareness level and functionality of the smartphone and further improve the user's experience with the smartphone. For example, when the smartphone transitions to engagement mode, the display can present visual elements such as a background image (e.g., wallpaper) or a welcome message. The display can also present a visual element (e.g., a locked padlock icon) indicating that access to the smartphone is locked and that authentication is required for access. These visual elements help the user understand that the smartphone is aware that the user is taking actions that suggest the user's intention to interact with the smartphone.

[0027] In addition, when the smartphone detects an action that is classified as an indication of user interaction intent, the awareness manager can automatically prepare the authentication system to perform an authentication process, such as facial recognition, voice recognition, or other authentication process (e.g., a password, fingerprint, or other biometric input). This preparation enables the authentication system to be ready for authentication more quickly. For example, in a scenario where the awareness manager automatically prepares the authentication system when the user reaches toward the smartphone, the authentication process can be performed when the awareness manager detects a triggering event. The triggering event is another interaction that is separate from the action that causes the smartphone to transition to the engaged state and indicates that the user is ready for authentication, such as picking up the smartphone or leaning over the smartphone.

[0028] When the user successfully authenticates, the awareness manager can automatically transition the smartphone from engaged mode to active mode. In active mode, the smartphone is fully ready to interact with the user (e.g., running applications, receiving input, and providing output). In addition, when the smartphone is in active mode, the awareness manager can also present additional or different visual elements on the display that can help the user understand the changed awareness level and functionality of the smartphone. For example, when the smartphone transitions to active mode, the display can transition a locked padlock icon to an unlocked padlock icon (to indicate successful authentication). After a short period of time, the unlocked icon can fade out and the display can present other visual elements, such as a home screen, a background image (e.g., a wallpaper), a home screen, a task list, or instructions (e.g., "slide to open").

[0029] The described techniques and systems employ radar systems to provide a useful and rewarding user experience that changes based on the user's level of interaction with the electronic device, as determined by the user's natural motion and actions relative to the device. The electronic device can not only react to explicit user input but also provide feedback to the user indicating that the device is aware of the user's movements and alerting the user to available features and functionality. Furthermore, the device can automatically arm the authentication system so that the user can enjoy a fast, seamless, and (if desired) automated authentication process.

[0030] The described techniques and systems can enable electronic devices (or at least displays of electronic devices) to achieve lower power consumption relative to some conventional techniques, which can use an always-on camera (or other sensor or combination of sensors) to control some display features. For example, power consumption can be reduced because the display uses less power when the user is not near the electronic device and gradually increases power consumption based on the user's implicit interaction with the electronic device. These are just a few examples of how the described techniques and devices can be used to enable radar-based authentication state feedback. Other examples and implementations are described in this document. The document now turns to an example operating environment and then describes example devices, methods, and systems.

[0031] Sample Environment

[0032] Figure 1 An example environment 100 is illustrated in which techniques for enabling radar-based authentication state feedback can be implemented. Example environment 100 includes an electronic device 102 that includes or is associated with a radar system 104, a persistent radar-based awareness manager 106 (awareness manager 106), and optionally one or more non-radar sensors 108 (non-radar sensors 108). Non-radar sensor 108 can be any of a variety of devices, such as an audio sensor (e.g., a microphone), a touch input sensor (e.g., a touchscreen), an infrared (IR) sensor, or an image-capture device (e.g., a camera or video camera).

[0033] In the example environment 100, the radar system 104 provides a radar field 110 by transmitting one or more radar signals or waveforms, as described below with reference to Figure 3-6 as described. The radar field 110 is a volume of space from which the radar system 104 can detect reflections of radar signals and waveforms (e.g., radar signals and waveforms reflected from objects in the volume of space). The radar system 104 also enables the electronic device 102 or another electronic device to sense and analyze reflections from objects in the radar field 110. Some embodiments of the radar system 104 are particularly advantageous when applied in the context of a smartphone, such as the electronic device 102. For example, with a smartphone, there are convergences such as the need for low power, the need for processing efficiency, limitations on the spacing and placement of antenna elements, and other issues. The radar system 104 may be particularly advantageous in the specific context of a smartphone where radar detection of fine gestures is required. However, it should be noted that although these embodiments are particularly advantageous in the described context of a smartphone where fine radar detection of gestures is required, the applicability of the features and advantages of the present invention is not necessarily so limited and relates to other embodiments of other types of electronic devices (e.g., as described in reference to FIG. 2 ). Figure 2 described) are also within the scope of the present teachings.

[0034] The object may be any of a variety of objects such as wood, plastic, metal, fabric, a human body, or a human body part (e.g., a foot, hand, or finger of a user of electronic device 102) that radar system 104 may sense and analyze radar reflections. Figure 1 As shown in FIG, the object is a user 112 of the electronic device 102. Based on the analysis of the reflections, the radar system 104 may provide radar data including various types of information associated with the radar field 110 and the reflections from the user 112, as shown in FIG. Figure 3-6 As described (e.g., radar system 104 can communicate radar data to other entities, such as awareness manager 106).

[0035] It should be noted that radar data may be provided continuously or periodically over time based on reflections from user 112 sensed and analyzed in radar field 110. The location of user 112 may change over time (e.g., user 112 may move within radar field 110), and thus the radar data may vary over time corresponding to the changing location, reflections, and analysis. Because radar data may vary over time, radar system 104 may provide radar data that includes one or more subsets of radar data corresponding to different time periods. For example, radar system 104 may provide a first subset of radar data corresponding to a first time period, a second subset of radar data corresponding to a second time period, and so on. Note that in some cases, one or more subsets of radar data may overlap or be substantially the same as one or more other subsets.

[0036] The electronic device 102 may also include a display device, such as a display 114. The display 114 may include any suitable display device, such as a touch screen, a liquid crystal display (LCD), a thin film transistor (TFT) LCD, an in-plane switching (IPS) LCD, a capacitive touch screen display, an organic light emitting diode (OLED) display, an active matrix organic light emitting diode (AMOLED) display, a super AMOLED display, and the like. The display 114 is used to display visual elements associated with various modes of the electronic device 102, which will be referred to herein. Figures 10 to 17 Described in further detail herein.

[0037] Awareness manager 106 can be used to interact with or control various components of electronic device 102 (e.g., modules, managers, systems, interfaces, or one or more non-radar sensors 108). For example, when radar system 104 does not detect a nearby user, awareness manager 106 can maintain electronic device 102 in sleep mode. Sleep mode is a persistent, low-power mode. In sleep mode, display 114 may present fewer visual elements than in other modes, or no visual elements. In some embodiments of sleep mode, the display may be powered off. In other embodiments, the display may be powered on but black (e.g., presenting no elements and not illuminated). If the display includes or provides a touch interface that can receive contact input (e.g., a capacitive touch interface that can receive input via touch, tap, or swipe), the touch interface may be turned on or off. The term "persistent" with respect to awareness manager 106 and the sleep mode of the electronic device means that no user interaction is required to maintain radar system 104 in sleep mode or to activate awareness manager 106. In some embodiments, the "persistent" state can be suspended or turned off (e.g., by user 112). In other embodiments, the "persistent" state can be scheduled or otherwise managed based on one or more parameters of the smartphone 102 (or other electronic device). For example, the user 112 can schedule the "persistent" state so that it is only operational during the day, even if the smartphone 102 is turned on at night and during the day.

[0038] In sleep mode, awareness manager 106 may also determine the presence of user 112 without verbal, touch, or other input from the user. For example, while in sleep mode, awareness manager 106 may use one or more subsets of radar data provided by radar system 104 (as described herein) to determine the presence of user 112 and / or other objects that may be within recognition zone 116 of electronic device 102. Recognition zone 116 is an area around radar system 104 within which awareness manager 106 may accurately determine the presence of user 112. Recognition zone 116 may take any of a variety of shapes and forms. For example, recognition zone 116 may correspond to radar field 110 (e.g., as described, for example, with reference to FIG. 1 ). Figure 3 and 4In some cases, awareness manager 106 (in sleep mode or another mode) may determine the presence of multiple objects in radar field 110, such as user 112 and one or more other users (e.g., other people).

[0039] The identification zone may extend any of a variety of distances from radar system 104, such as approximately three, seven, ten, or fourteen feet (or approximately one, two, three, or four meters). Figure 1 , recognition area 116 may be smaller than the maximum range of radar field 110. Recognition area 116 may be a static size or shape that is predefined, user-selectable, or determined via another method (e.g., based on power requirements, remaining battery life, or another factor). In some cases, recognition area 116 may be dynamically and automatically adjustable by awareness manager 106 based on factors such as the speed or location of electronic device 102, the time of day, the state of an application running on electronic device 102, or another factor.

[0040] When the awareness manager 106 determines that the user 112 is present within the identification area 116, the awareness manager 106 may cause the electronic device 102 to exit sleep mode and enter awareness mode. Awareness mode is a lower power mode in which the display 114 presents one or more visual elements that may indicate the state or functionality level of the electronic device 102. In awareness mode, the electronic device 102 may provide different features and functions than those provided in sleep mode. In some embodiments of awareness mode, the visual elements presented include elements that describe environmental information or simple status information about the electronic device 102. For example, in awareness mode, the display 114 may present the time of day (clock), date, connection status (e.g., Wi-Fi, cellular or other network connectivity), or a battery level indicator. The rest of the screen may remain black or display a background image, such as wallpaper or other image. In other embodiments, the visual element indicating that the electronic device 102 is in awareness mode may be an image, such as wallpaper, a background image, or a series of images or wallpapers. Additionally or alternatively, radar system 104 may operate at a different frame rate, duty cycle, or power level than in sleep mode (e.g., radar system 104 and / or display 114 may use less power in sleep mode than in awareness mode). If user 112 exits recognition area 116, awareness manager 106 may return electronic device 102 to sleep mode (immediately, or after user 112 has been outside recognition area 116 for a selectable predetermined amount of time). For example, awareness manager 106 may determine that user 112 has turned away from electronic device 102 and / or is in the process of leaving electronic device 102 and immediately return to sleep mode.

[0041] In some embodiments (e.g., as described in reference Figure 13-17 As described above, an application running on the electronic device 102 may be able to receive input via radar-based, remote, three-dimensional (3D) gestures (remote 3D gestures). The term remote 3D gesture refers to the nature of a spatial gesture that is remote from the electronic device (e.g., the gesture does not require the user to touch the device, but it must be within a threshold distance of the device to be detected by the radar system 104). The gesture itself can be two-dimensional (2D), such as a swipe from the upper left to the lower right in a plane, but because the gesture also has a distance from the electronic device (e.g., a "third" dimension), the gesture is referred to herein as a remote 3D gesture, regardless of whether the gesture itself includes elements in three dimensions. In this case, the display in awareness mode can present a visual element that indicates whether the application can receive gesture input of that type. For example, the display 114 can present an icon, a luminous area, or another element that is presented only when a radar gesture-subscribing application (gesture-subscribing application) is running.

[0042] In awareness mode, awareness manager 106 may also detect user actions that are categorized as indicative of user intent to interact (or not interact) with electronic device 102. For example, the electronic device may have access to a library (e.g., in a storage device included with or associated with electronic device 102) of actions categorized as indicators of user intent to interact or not interact with the device (e.g., reaching for the electronic device, turning or walking toward or away from the electronic device, leaning over or looking at the electronic device). Actions categorized as indicative of user intent to interact with electronic device 102 may be referred to herein as user actions that indicate user intent, indications of user intent, or indications of user interaction intent and may include reaching, moving, or turning toward electronic device 102, looking at or leaning over electronic device 102, and the like. Thus, for example, while in awareness mode, awareness manager 106 may use one or more other subsets of radar data provided by radar system 104 (as described herein) to detect that user 112 is reaching toward electronic device 102 and determine that the reaching out is indicative of the user's intent to interact with electronic device 102.

[0043] Different actions that are classified as indicative of user interaction intent may have different thresholds for consideration. For example, if a part of the user's body moves within a threshold distance of the electronic device (e.g., five inches, eight inches, or twelve inches), then a reach toward the electronic device 102 may only be determined to indicate the user's intent. In some embodiments, the electronic device may also include machine learning technology that enables the electronic device 102 to learn different or additional indications of user intent and add, remove, or modify actions stored in the library (e.g., based on the user's device history and behavior).

[0044] Furthermore, if awareness manager 106 determines that a user is within the recognition area, but no user action is detected that is classified as an indication of an intent to interact with the smartphone, awareness manager 106 may maintain electronic device 102 in awareness mode until such user action is detected by awareness manager 106. In other cases, awareness manager 106 may maintain electronic device 102 in awareness mode for a selectable and adjustable duration (e.g., 60, 30, or 10 seconds) in the absence of such user action, and then transition back to sleep mode, even if the user remains within the recognition area.

[0045] In some embodiments, when awareness manager 106 detects an indication of a user's intent to interact with electronic device 102, awareness manager 106 may cause electronic device 102 to exit awareness mode and enter engaged mode. Engaged mode is a higher power mode for electronic device 102 and radar system 104. In embodiments in which electronic device 102 enters engaged mode, awareness manager 106 may cause display 114 to present additional or different visual elements that indicate the new or changed state of electronic device 102, including the availability of changed or new features, functionality, or services. For example, display 114 may present a background image (e.g., wallpaper) or a welcome message. Display 114 may also present a visual element (e.g., a locked padlock icon) indicating that access to electronic device 102 is locked and requires authentication. These visual elements help the user understand that the smartphone is aware that the user is taking an action that suggests the user intends to interact with the smartphone. In some cases, the visual element indicating that electronic device 102 is in engaged mode may be a wallpaper, background image, or a series of images, with the changing and / or additional images indicating the change in mode.

[0046] In some embodiments of the awareness mode or engagement mode, the awareness manager 106 can cause the electronic device 102 to automatically prepare the authentication system 118 to perform the authentication process. The authentication system 118 can use any suitable authentication technology, such as facial recognition, voice recognition, fingerprint recognition, etc. For example, the authentication system 118 can include or be associated with a camera (e.g., one of the non-radar sensors 108) and a facial recognition module 120, which can be used to authenticate the user based on a captured image (or multiple captured images) of the user's face. This preparation enables the authentication system 118 to be ready to perform authentication with lower latency (e.g., faster). For example, when the awareness manager 106 detects an action indicating an intent to interact, the awareness manager 106 can automatically prepare the camera 118 by putting it into a state where it can capture images (e.g., a camera "warm-up" process that prepares the camera hardware and any associated software or other procedures for capturing images). Furthermore, the awareness manager 106 can also put the facial recognition module 120 into a state where it can use the captured images to authenticate the user.

[0047] When the authentication system 118 is prepared, the awareness manager 106 can cause the authentication system 118 to perform an authentication process on the user 112 based on a triggering event. The triggering event indicating that the user is ready to be authenticated is another interaction separate from or in addition to the action of transitioning the electronic device 102 to the optional engagement mode. For example, the triggering event can be a change in the position or orientation of the user 112 (e.g., the user 112 moves toward or leans over the electronic device 102 so that the user's face is directly in front of the display 114). The triggering event can alternatively or also be a change in the orientation of the electronic device 102 (the user 112 picks up or rotates the electronic device 102) or an explicit user action such as a touch input.

[0048] The triggering event may be detected using radar data or non-radar data (e.g., from non-radar sensor 108 or another source). As described above, radar-based awareness manager 106 may determine a movement made by a user or a user's hand based on radar data (e.g., 3D gestures) provided by radar system 104. Awareness manager 106 may then process the movement in a manner that enables the user to interact with electronic device 102 via the movement. For example, as described with reference to Figures 3 to 6 As described, a radar system may use the radar field to sense and analyze reflections from objects in the radar field in a manner that enables high resolution and accuracy for movement identification of a user.

[0049] In some cases, there may be no triggering event, indicating that the user 112 does not intend to interact with the electronic device 102. For example, the user 112 may reach toward the electronic device 102 and then decide to do something else, or simply reach for the electronic device 102 to check the time and then withdraw. In these cases, the user is not authenticated, and if the electronic device 102 is in the engaged mode, the awareness manager 106 may maintain the electronic device 102 in the engaged mode for a selectable duration (e.g., 60, 30, or 10 seconds) and then transition back to the awareness mode.

[0050] When user 112 is successfully authenticated, awareness manager 106 causes electronic device 102 to exit awareness or engagement mode and enter active mode. Active mode is a higher-power mode in which user 112 has full rights and access to electronic device 102. When electronic device 102 transitions from awareness or engagement mode to active mode, awareness manager 106 may cause display 114 to present additional or different visual elements indicating the new or changed state of electronic device 102, including the changed or new features, functionality, or services available. For example, display 114 may present a background image (e.g., wallpaper) or a home screen. Display 114 may also present a visual element (e.g., an unlocked padlock icon) indicating that access to electronic device 102 is unlocked. Some or all of the additional background images or visual elements may be presented for a duration and then fade out or cease to be presented. These additional images and visual elements provide feedback to user 112 indicating that electronic device 102 is unlocked and fully accessible, even if the user has merely walked up to electronic device 102, reached toward it, and picked it up. As described above, awareness manager 106 can determine the presence of user 112 and other people. In this case, awareness manager 106, alone or in combination with other applications or components of electronic device 102, can use the presence of other people as a factor in determining which additional or different visual elements to present on display 114 in active mode. For example, when other humans are present, the active mode display can be a lock screen with an unlock icon or other image (such as a wallpaper). Conversely, when other humans are not present, the active mode display can be a home screen, a recent screen (e.g., opening the last application used before electronic device 102 was locked and presenting a screen from that application), or another screen or user interface. In this way, awareness manager 106 can provide an automatic authentication process while helping to maintain the privacy of user 112.

[0051] consider Figure 1. In detailed view 100-1, user 112 is outside radar field 110 and recognition area 116. Therefore, electronic device 102 (in this case, a smartphone) is in sleep mode, which in this case corresponds to display 114 not presenting any visual elements, as shown in example display 114-1. In this example, assume that user 112 walks towards electronic device 102 and reaches toward it, causing electronic device 102 to transition from sleep mode to awareness mode, and then (optionally) to engagement mode. As described above, this sequence prepares electronic device 102 to authenticate system 118. Then, as shown in detailed view 100-2, user 112 leans over and lifts electronic device 102 from the table. In response to this triggering event, electronic device 102 authenticates user 112 and transitions from awareness or engagement mode to active mode. In active mode, display 114 presents a visual element 122 (in this case, an unlocked padlock icon) indicating that electronic device 102 is unlocked, as shown in example display 114-2.

[0052] In more detail, consider Figure 2 , which illustrates an example implementation 200 of an electronic device 102 (including a radar system 104 , an awareness manager 106 , and non-radar sensors 108 ) that may implement radar-based authentication state feedback. Figure 2 The electronic device 102 is illustrated with various example devices, including a smartphone 102-1, a tablet 102-2, a laptop computer 102-3, a desktop computer 102-4, a computing watch 102-5, a gaming system 102-6, computing glasses 102-7, a home automation and control system 102-8, a smart refrigerator 102-9, and a car 102-10. The electronic device 102 may also include other devices such as televisions, entertainment systems, audio systems, drones, tracking pads, drawing tablets, netbooks, e-readers, home security systems, and other home appliances. Note that the electronic device 102 may be wearable, non-wearable but mobile, or relatively immobile (e.g., desktop computers and appliances).

[0053] In some embodiments, an exemplary overall lateral size of the electronic device 102 can be approximately 8 centimeters by approximately 15 centimeters. An exemplary coverage area of ​​the radar system 104 can be even further limited, such as approximately 4 millimeters by 6 millimeters when the antenna is included. This requirement for such a limited coverage area of ​​the radar system 104 is necessary to accommodate many other desirable features of the electronic device 102 (e.g., a fingerprint sensor, non-radar sensors 108, etc.) in such a space-constrained package. Combined with power and processing limitations, this size requirement can result in tradeoffs in the accuracy and effectiveness of radar gesture detection, at least some of which can be overcome in light of the teachings herein.

[0054] The electronic device 102 also includes one or more computer processors 202 and one or more computer-readable media 204, which include memory media and storage media. Applications and / or operating systems (not shown) implemented as computer-readable instructions on the computer-readable media 204 can be executed by the computer processor 202 to provide some or all of the functions described herein. The electronic device 102 can also include a network interface 206. The electronic device 102 can use the network interface 206 to communicate data through a wired, wireless, or optical network. By way of example and not limitation, the network interface 206 can communicate data through a local area network (LAN), a wireless local area network (WLAN), a personal area network (PAN), a wide area network (WAN), an intranet, the Internet, a peer-to-peer network, a point-to-point network, or a mesh network.

[0055] Various implementations of radar system 104 may include a system on a chip (SoC), one or more integrated circuits (ICs), a processor with embedded processor instructions or configured to access processor instructions stored in memory, hardware with embedded firmware, a printed circuit board with various hardware components, or any combination thereof. Radar system 104 may operate as a monostatic radar by transmitting and receiving its own radar signals.

[0056] In some embodiments, the radar system 104 may also cooperate with other radar systems 104 within the external environment to implement a bistatic radar, a multistatic radar, or a networked radar. However, the constraints or limitations of the electronic device 102 may affect the design of the radar system 104. The electronic device 102 may, for example, have limited power available to operate the radar, limited computing power, size constraints, layout restrictions, enclosures that attenuate or distort radar signals, etc. The radar system 104 includes several features that enable advanced radar functionality and high performance despite these constraints, as described below with respect to Figure 3 further described. Note that in Figure 2 , radar system 104 and awareness manager 106 are shown as part of electronic device 102. In other implementations, either or both of radar system 104 and awareness manager 106 may be separate or remote from electronic device 102.

[0057] These and other capabilities and configurations are described in more detail below and Figure 1 The way entities act and interact with each other. These entities can be further divided, combined, and so on. Figure 1 Environment 100 and Figures 2 to 18 The detailed diagrams illustrate some of the many possible environments and devices in which the described techniques can be employed. Figures 3 to 6 Additional details and features of radar system 104 are described. Figures 3 to 6In the present disclosure, radar system 104 is described in the context of electronic device 102, but as mentioned above, the applicability of the features and advantages of the described systems and techniques is not necessarily limited thereby, and other implementations involving other types of electronic devices may also be within the scope of the present teachings.

[0058] Figure 3 An example implementation 300 of a radar system 104 is shown that can be used to enable radar-based authentication status feedback. In example 300, the radar system 104 includes at least one of each of the following components: a communication interface 302, an antenna array 304, a transceiver 306, a processor 308, and system media 310 (e.g., one or more computer-readable storage media). The processor 308 can be implemented as a digital signal processor, a controller, an application processor, another processor (e.g., the computer processor 202 of the electronic device 102), or some combination thereof. The system media 310 can include being within or separate from the computer-readable medium 204 of the electronic device 102 and includes one or more of the following modules: a fade mitigator 314, a digital beamformer 316, an angle estimator 318, or a power manager 320. These modules can compensate for or mitigate the effects of integrating the radar system 104 within the electronic device 102, thereby enabling the radar system 104 to recognize small or complex gestures, distinguish between different orientations of a user, continuously monitor the external environment, or achieve a targeted false alarm rate. With these features, the radar system 104 can be used in applications such as Figure 2 The devices shown in FIG. 1 may be implemented in various different devices.

[0059] Using communication interface 302, radar system 104 can provide radar data to awareness manager 106. Communication interface 302 can be a wireless or wired interface based on radar system 104 being implemented separately from electronic device 102 or integrated within electronic device 102. Depending on the application, radar data can include raw or minimally processed data, in-phase and quadrature (I / Q) data, range-Doppler data, processed data including target location information (e.g., range, azimuth, elevation), cluttered map data, etc. Generally, radar data contains information that can be used by awareness manager 106 for radar-based authentication status feedback.

[0060] The antenna array 304 includes at least one transmitting antenna element (not shown) and at least two receiving antenna elements (e.g. Figure 4). In some cases, the antenna array 304 may include multiple transmit antenna elements to implement a multiple-input, multiple-output (MIMO) radar capable of transmitting multiple different waveforms at once (e.g., each transmit antenna element transmits a different waveform). The use of multiple waveforms may improve the measurement accuracy of the radar system 104. For embodiments including three or more receive antenna elements, the receive antenna elements may be positioned in a one-dimensional shape (e.g., a line) or a two-dimensional shape. The one-dimensional shape enables the radar system 104 to measure one angular dimension (e.g., azimuth or elevation), while the two-dimensional shape enables the measurement of two angular dimensions (e.g., azimuth and elevation). Reference Figure 4 , further describing an example two-dimensional arrangement of receive antenna elements.

[0061] Figure 4 An example arrangement 400 of receive antenna elements 402 is shown. If the antenna array 304 includes at least four receive antenna elements 402, for example, the receive antenna elements 402 may be as follows: Figure 4 Alternatively, if the antenna array 304 includes at least three receive antenna elements 402, a triangular arrangement 404-2 or an L-shaped arrangement 404-3 may be used.

[0062] Due to the size or layout constraints of electronic device 102, the element spacing between receive antenna elements 402 or the number of receive antenna elements 402 may not be ideal for the angles to be monitored by radar system 104. In particular, the element spacing may result in angular ambiguity, which makes it challenging for conventional radars to estimate the angular position of a target. Consequently, conventional radars may limit their field of view (e.g., the angle to be monitored) to avoid ambiguous regions with angular ambiguity and thereby reduce false detections. For example, a conventional radar may limit its field of view to angles between approximately -45 degrees and 45 degrees to avoid the angular ambiguity that occurs when using a 5 millimeter (mm) wavelength and an element spacing of 3.5 mm (e.g., an element spacing of 70% of the wavelength). Consequently, conventional radars may be unable to detect targets beyond the 45-degree limit of the field of view. In contrast, radar system 104 includes digital beamformer 316 and angle estimator 318, which resolve the angular ambiguity and enable radar system 104 to monitor angles beyond the 45-degree limit, such as angles between approximately -90 degrees and 90 degrees, or angles up to approximately -180 degrees and 180 degrees. These angular ranges can apply across one or more directions (e.g., azimuth and / or elevation). Thus, radar system 104 can achieve low false alarm rates for a variety of different antenna array designs including element spacings less than, greater than, or equal to half the center wavelength of the radar signal.

[0063] Using antenna array 304, radar system 104 can form a beam that is steered or unsteered, wide or narrow, or shaped (e.g., shaped as a hemisphere, cube, sector, cone, or cylinder). As an example, one or more transmit antenna elements (not shown) may have an unsteered omnidirectional radiation pattern, or may be capable of producing a wide beam, such as wide transmit beam 406. Any of these techniques enables radar system 104 to illuminate a large volume of space. However, to achieve target angular accuracy and angular resolution, receive antenna elements 402 and digital beamformer 316 may be used to generate thousands of narrow and steered beams (e.g., 2,000 beams, 4,000 beams, or 6,000 beams), such as narrow receive beam 408. In this way, radar system 104 can effectively monitor the external environment and accurately determine the angle of arrival of reflections within the external environment.

[0064] return Figure 3 , transceiver 306 includes circuitry and logic for transmitting and receiving radar signals via antenna array 304. Components of transceiver 306 may include amplifiers, mixers, switches, analog-to-digital converters, filters, etc. for conditioning radar signals. Transceiver 306 may also include logic for performing in-phase / quadrature (I / Q) operations, such as modulation or demodulation. Transceiver 306 may be configured for continuous wave radar operation or pulsed radar operation. Various modulations may be used to generate radar signals, including linear frequency modulation, triangular frequency modulation, stepped frequency modulation, or phase modulation.

[0065] The transceiver 306 can generate a radar signal within a frequency range (e.g., a spectrum) such as between 1 gigahertz (GHz) and 400 GHz, between 4 GHz and 100 GHz, or between 57 GHz and 63 GHz. The spectrum can be divided into multiple sub-spectra with similar or different bandwidths. The bandwidth can be on the order of 500 megahertz (MHz), 1 GHz, 2 GHz, etc. As an example, different frequency sub-spectra can include frequencies between approximately 57 GHz and 59 GHz, between 59 GHz and 61 GHz, or between 61 GHz and 63 GHz. Multiple frequency sub-spectra with the same bandwidth and that can be continuous or non-continuous can also be selected for coherence. Multiple frequency sub-spectra can be transmitted simultaneously or separated in time using a single radar signal or multiple radar signals. Continuous frequency sub-spectra enable the radar signal to have a wider bandwidth, while non-continuous frequency sub-spectra can further emphasize amplitude and phase differences, enabling the angle estimator 318 to resolve angle ambiguities. The fade mitigator 314 or the angle estimator 318 may enable the transceiver 306 to utilize one or more frequency sub-spectra to improve the performance of the radar system 104, such as with respect to Figure 5 and 6 Further described.

[0066] Power manager 320 enables radar system 104 to conserve power internally or externally to electronic device 102. In some embodiments, power manager 320 communicates with awareness manager 106 to conserve power within either or both radar system 104 and electronic device 102. Internally, for example, power manager 320 can cause radar system 104 to collect data using a predefined power mode or a specific duty cycle. In this case, power manager 320 dynamically switches between different power modes, managing response latency and power consumption based on the activity within the environment. Generally, power manager 320 determines when and how to conserve power and incrementally adjusts power consumption to enable radar system 104 to operate within the power constraints of electronic device 102. In some cases, power manager 320 can monitor the remaining available power and adjust radar system 104 operations accordingly. For example, if the remaining power is low, power manager 320 can continue operating in a lower power mode rather than switching to a higher power mode.

[0067] For example, a lower power mode may use a lower duty cycle on the order of a few hertz (e.g., approximately 1 Hz or less than 5 Hz), which reduces power consumption to a few milliwatts (mW) (e.g., between approximately 2 mW and 8 mW). On the other hand, a higher power mode may use a higher duty cycle on the order of tens of hertz (Hz) (e.g., approximately 20 Hz or greater than 10 Hz), which causes radar system 104 to consume several milliwatts (e.g., between approximately 6 mW and 20 mW). While the lower power mode may be used to monitor the external environment or detect an approaching user, if radar system 104 determines that a user is beginning to perform a gesture, power manager 320 may switch to the higher power mode. Different triggers may cause power manager 320 to switch between different power modes. Example triggers include motion or lack of motion, the appearance or disappearance of a user, the user moving into or out of a specified area (e.g., an area defined by distance, azimuth, or elevation), a change in the speed of motion associated with the user, or a change in reflected signal strength (e.g., due to a change in radar cross section). Typically, triggers indicating a lower probability of user interaction with the electronic device 102 or a preference to collect data using longer response delays may cause a lower power mode to be activated to conserve power.

[0068] Power manager 320 can also save power by shutting down one or more components within transceiver 306 (e.g., a voltage-controlled oscillator, a multiplexer, an analog-to-digital converter, a phase-locked loop, or a crystal oscillator) during periods of inactivity. These periods of inactivity occur when radar system 104 is not actively transmitting or receiving radar signals and can be on the order of microseconds (μs), milliseconds (ms), or seconds (s). Furthermore, power manager 320 can modify the transmit power of the radar signal by adjusting the amount of amplification provided by the signal amplifier. Furthermore, power manager 320 can control the use of various hardware components within radar system 104 to save power. For example, if processor 308 includes a lower-power processor and a higher-power processor (e.g., processors with different amounts of memory and computational capabilities), power manager 320 can switch between utilizing the lower-power processor for low-level analysis (e.g., implementing idle mode, detecting motion, determining a user's location, or monitoring the environment) and utilizing the higher-power processor for situations where awareness manager 106 requests high-fidelity or accurate radar data (e.g., for implementing awareness mode, engagement mode, or activity mode, gesture recognition, or user orientation).

[0069] Additionally, power manager 320 can determine the context of the environment surrounding electronic device 102. Based on this context, power manager 320 can determine which power states will become available and how they should be configured. For example, if electronic device 102 is in a user's pocket, radar system 104 need not operate in a higher power mode with a high duty cycle even though user 112 is detected as being in proximity to electronic device 102. Thus, even if a user is detected as being in proximity to electronic device 102, power manager 320 can cause radar system 104 to remain in a lower power mode and cause display 114 to remain off or in another lower power state. Electronic device 102 can use any suitable non-radar sensor 108 (e.g., a gyroscope, accelerometer, light sensor, proximity sensor, capacitive sensor, etc.) in conjunction with radar system 104 to determine the context of its environment. This context can include the time of day, calendar day, brightness / darkness, the number of users in the vicinity of user 112, the surrounding noise level, the speed at which surrounding objects (including user 112) are moving relative to electronic device 102, and the like.

[0070] Figure 5Additional details of an example implementation 500 of a radar system 104 within an electronic device 102 are shown. In example 500, antenna array 304 is positioned beneath an outer housing of electronic device 102, such as a glass cover or housing. Depending on its material properties, the outer housing can act as an attenuator 502, attenuating or distorting radar signals transmitted and received by radar system 104. Attenuator 502 can comprise different types of glass or plastic, some of which can be found within a display screen, housing, or other components of electronic device 102, and have a dielectric constant (e.g., relative permittivity) between approximately 4 and 10. Thus, attenuator 502 is opaque or translucent to radar signal 506 and can cause a portion of transmitted or received radar signal 506 to be reflected (as shown by reflected portion 504). For conventional radar, attenuator 502 can reduce the effective range that can be monitored, prevent small targets from being detected, or reduce overall accuracy.

[0071] Assuming that the transmit power of radar system 104 is limited and redesigning the enclosure is undesirable, one or more attenuation-related properties of radar signal 506 (e.g., frequency sub-spectrum 508 or steering angle 510) or attenuation-related characteristics of attenuator 502 (e.g., distance 512 between attenuator 502 and radar system 104 or thickness 514 of attenuator 502) are adjusted to mitigate the effects of attenuator 502. Some of these characteristics may be set during manufacturing or adjusted by attenuation mitigator 314 during operation of radar system 104. Attenuation mitigator 314 may, for example, cause transceiver 306 to transmit radar signal 506 using a selected frequency sub-spectrum 508 or steering angle 510, cause the platform to move radar system 104 closer to or further away from attenuator 502 to change distance 512, or prompt a user to apply another attenuator to increase thickness 514 of attenuator 502.

[0072] Appropriate adjustments can be made by the fade mitigator 314 based on predetermined characteristics of the attenuator 502 (e.g., characteristics stored in the computer-readable medium 204 of the electronic device 102 or within the system medium 310) or by processing the return of the radar signal 506 to measure one or more characteristics of the attenuator 502. Even if some of the attenuation-related characteristics are fixed or constrained, the fade mitigator 314 can take these constraints into account to balance each parameter and achieve the desired radar performance. As a result, the fade mitigator 314 enables the radar system 104 to achieve enhanced accuracy and a greater effective range for detecting and tracking users located on opposite sides of the attenuator 502. These techniques provide an alternative to increasing the transmit power, which increases the power consumption of the radar system 104, or to changing the material properties of the attenuator 502, which can be difficult and expensive once the device is in production.

[0073] Figure 6 An example scheme 600 implemented by radar system 104 is shown. Portions of scheme 600 may be performed by processor 308, computer processor 202, or other hardware circuitry. Scheme 600 may be customized to support different types of electronic devices and radar-based applications (e.g., awareness manager 106), and enables radar system 104 to achieve target angle accuracy despite design constraints.

[0074] Transceiver 306 generates raw data 602 based on the individual responses of receive antenna elements 402 to the received radar signals. The received radar signals may be associated with one or more frequency sub-spectra 604 selected by angle estimator 318 to facilitate angular ambiguity resolution. For example, frequency sub-spectra 604 may be selected to reduce the number of side lobes or reduce the amplitude of the side lobes (e.g., reducing the amplitude by 0.5 dB, 1 dB, or more). The number of frequency sub-spectra may be determined based on the target angle accuracy or computational limitations of radar system 104.

[0075] Raw data 602 includes digital information (e.g., in-phase and quadrature data) for a period of time, for different wave numbers and multiple channels, respectively, associated with receive antenna elements 402. A fast Fourier transform (FFT) 606 is performed on raw data 602 to generate pre-processed data 608. Pre-processed data 608 includes digital information across a period of time, for different ranges (e.g., intervals), and for multiple channels. Doppler filtering 610 is performed on pre-processed data 608 to generate range-Doppler data 612. Doppler filtering 610 may include another FFT that generates amplitude and phase information for multiple range bins, multiple Doppler frequencies, and multiple channels. Digital beamformer 316 generates beamformed data 614 based on range-Doppler data 612. Beamformed data 614 includes digital information for a set of azimuth and / or elevation angles representing the field of view from which different steering angles or beams are formed by digital beamformer 316. Although not shown, the digital beamformer 316 may alternatively generate beamformed data 614 based on the pre-processed data 608, and the Doppler filtering process 610 may generate range-Doppler data 612 based on the beamformed data 614. To reduce computational effort, the digital beamformer 316 may process a portion of the range-Doppler data 612 or the pre-processed data 608 based on a range, time, or Doppler frequency interval of interest.

[0076] The digital beamformer 316 can be implemented using a single-look beamformer 616, a multi-look interferometer 618, or a multi-look beamformer 620. Typically, the single-look beamformer 616 can be used for deterministic objects (e.g., point source targets with a single phase center). For non-deterministic targets (e.g., targets with multiple phase centers), a multi-look interferometer 618 or a multi-look beamformer 620 is used to improve accuracy relative to the single-look beamformer 616. A human is an example of a non-deterministic target and has multiple phase centers 622 that can change based on different field of view angles, as shown at 624-1 and 624-2. The variations in constructive or destructive interference generated by the multiple phase centers 622 can be challenging for conventional radars to accurately determine angular position. However, the multi-look interferometer 618 or the multi-look beamformer 620 performs coherent averaging to increase the accuracy of the beamformed data 614. Multi-look interferometer 618 coherently averages two channels to generate phase information that can be used to accurately determine angular information. Multi-look beamformer 620, on the other hand, can use a linear or nonlinear beamformer, such as Fourier, capon, multiple signal classification (MUSIC), or minimum variance distortionless response (MVDR), to coherently average two or more channels. The increased accuracy provided by multi-look beamformer 620 or multi-look interferometer 618 enables radar system 104 to recognize small gestures or distinguish between multiple parts of a user.

[0077] Angle estimator 318 analyzes beamforming data 614 to estimate one or more angular positions. Angle estimator 318 may utilize signal processing techniques, pattern matching techniques, or machine learning. Angle estimator 318 also resolves angular ambiguities that may be caused by the design of radar system 104 or the field of view of a radar system 104 monitor. Exemplary angular ambiguities are shown in amplitude plot 626 (e.g., amplitude response).

[0078] The amplitude graph 626 depicts the amplitude differences that may occur for different angular positions of the target and for different steering angles 510. A first amplitude response 628-1 (shown as a solid line) is shown for a target at a first angular position 630-1. Similarly, a second amplitude response 628-2 (shown as a dashed line) is shown for a target at a second angular position 630-2. In this example, differences in angles between -180 degrees and 180 degrees are considered.

[0079] As shown in amplitude graph 626, there are ambiguity regions for two angular positions 630-1 and 630-2. A first amplitude response 628-1 has the highest peak at first angular position 630-1 and a smaller peak at second angular position 630-2. While the highest peak corresponds to the actual location of the target, the smaller peak causes ambiguity in first angular position 630-1 because it is within some threshold where a conventional radar might not be able to confidently determine whether the target is at first angular position 630-1 or second angular position 630-2. Conversely, a second amplitude response 628-2 has a smaller peak at second angular position 830-2 and a higher peak at first angular position 630-1. In this case, the smaller peak corresponds to the location of the target.

[0080] While conventional radars may be limited to using the highest peak amplitude to determine angular position, angle estimator 318 instead analyzes subtle differences in the shapes of amplitude responses 628-1 and 628-2, characteristics of which may include, for example, roll-off, peak or null width, angular position of peaks or nulls, height or depth of peaks and nulls, sidelobe shape, symmetry within amplitude response 628-1 or 628-2, or asymmetry within amplitude response 628-1 or 628-2. Thus, angle estimator 318 maps a unique angular signature or pattern to an angular position.

[0081] Angle estimator 318 may include a set of algorithms or tools that can be selected based on the type of electronic device 102 (e.g., computing power or power constraints) or the target angular resolution of awareness manager 106. In some embodiments, angle estimator 318 may include a neural network 632, a convolutional neural network (CNN) 634, or a long short-term memory (LSTM) network 636. Neural network 632 may have various depths or numbers of hidden layers (e.g., 3 hidden layers, 5 hidden layers, or 10 hidden layers) and may also include different numbers of connections (e.g., neural network 632 may include a fully connected neural network or a partially connected neural network). In some cases, CNN 634 may be used to increase the computational speed of angle estimator 318. LSTM network 636 may be used to enable angle estimator 318 to track a target. Using machine learning techniques, angle estimator 318 employs a nonlinear function to analyze the shape of amplitude response 628-1 or 628-2 and generates angle probability data 638, which indicates the likelihood that the user or a portion of the user is within the angle bin. Angle estimator 318 may provide angle probability data 638 for a few angle bins—such as two angle bins providing probabilities that the target is to the left or right of electronic device 102—or for thousands of angle bins (e.g., providing angle probability data 638 for consecutive angle measurements).

[0082] Based on the angular probability data 638, the tracker module 640 generates angular position data 642 that identifies the angular position of the target. The tracker module 640 can determine the angular position of the target based on the angular bin with the highest probability in the angular probability data 638 or based on predicted information (e.g., previously measured angular position information). The tracker module 640 can also keep track of one or more moving targets so that the radar system 104 can reliably distinguish or identify the targets. Other data can also be used to determine the angular position, including range, Doppler, velocity, or acceleration. In some cases, the tracker module 640 can include an alpha-beta tracker, a Kalman filter, a multi-hypothesis tracker (MHT), etc.

[0083] Quantizer module 644 obtains angular position data 642 and quantizes the data to produce quantized angular position data 646. Quantization can be performed based on the target angular resolution of awareness manager 106. In some cases, a fewer number of quantization levels can be used, such that quantized angular position data 646 indicates whether the target is to the right or left of electronic device 102, or identifies the 90-degree quadrant in which the target is located. This may be sufficient for some radar-based applications, such as user proximity detection. In other cases, a larger number of quantization levels can be used, such that quantized angular position data 646 indicates the angular position of the target within an accuracy of a fraction of a degree, 1 degree, 5 degrees, etc. This resolution can be used for higher-resolution radar-based applications, such as gesture recognition, or in implementations that recognize zones, awareness modes, engagement modes, or activity modes as described herein. In some embodiments, digital beamformer 316, angle estimator 318, tracker module 640, and quantizer module 644 are implemented together in a single machine learning module.

[0084] These and other capabilities and configurations are described below, and Figures 1 to 6 The entities described may be further divided, combined, used with other sensors or components, etc. In this manner, radar-based authentication status feedback may be implemented using different embodiments of electronic devices 102 with different configurations of radar systems 104 and non-radar sensors. Figure 1 An example operating environment 100 and Figures 2 to 6 The detailed diagrams illustrate some of the many possible environments and devices in which the described techniques can be employed.

[0085] Example Method

[0086] Figures 7 to 9Depicted is an example method 700 that enables radar-based authentication status feedback. Method 700 can be performed using an electronic device that uses a radar system to provide a radar field. The radar field is used to detect user interaction with the electronic device, such as the user's presence in the radar field and the user's movement relative to the electronic device. The user's movement can also be used to detect user actions that are categorized as indications of the user's intent to interact (or not interact) with the electronic device. For example, the electronic device may have access to a library (e.g., in a storage device included with or associated with the electronic device) of actions categorized as indicators of the user's intent to interact or not interact with the device (reaching out to pick up the electronic device, turning or moving toward or away from the electronic device, leaning over or looking at the electronic device). Actions categorized as indications of user intent to interact with the electronic device may be referred to herein as user actions indicating user intent, indications of user intent, or indications of user interaction intent and may include reaching out, moving, or turning toward the electronic device 102, looking at or leaning over the electronic device, and so on. Based on the detection of the user's presence, movement, and indications of user intent, the electronic device can enter and exit different modes of functionality based on these modes and present different visual elements on the display. These modes can enable different functionalities for electronic devices, relative to Figure 10-17 Describe its example.

[0087] The method 700 is shown as a collection of blocks that specify the operations performed but is not necessarily limited to the order or combination shown for performing the operations by the corresponding blocks. In addition, any one or more operations may be repeated, combined, reorganized, or chained to provide a variety of additional and / or alternative methods. In the following discussion, reference may be made to Figure 1 An example operating environment 100 or Figures 2 to 6 Where entities or processes are described in detail herein, reference thereto is by way of example only. The techniques are not limited to performance by one entity or multiple entities operating on one device.

[0088] At 702, a radar field is provided. The radar field can be provided by any of a variety of electronic devices (e.g., electronic device 102 described above), which includes or is associated with a radar system (e.g., radar system 104) and an awareness manager (e.g., awareness manager 106). Furthermore, the radar field can be any of a variety of radar fields, such as radar field 110 described above.

[0089] At 704, reflections from objects in the radar field are sensed by the radar system. The objects can be any of a variety of objects such as wood, plastic, metal, fabric, or organic materials (e.g., a person, such as user 112 described above, or a body part of a person, such as a user's hand). For clarity, the objects will be referred to as "users" or "users" while describing method 700.

[0090] At 706, reflections from objects in the radar field are analyzed. The analysis may be performed by any of a variety of entities (e.g., radar system 104, awareness manager 106, or another entity) and may include various operations or determinations, such as reference Figure 1-6 Those described.

[0091] At 708, based on the analysis of the reflections, radar data (e.g., reference Figure 1-6 The radar data described herein may be provided by any of a variety of entities, such as the radar system 104, the awareness manager 106, or another entity. In some embodiments, the radar system may provide the radar data and communicate the radar data to other entities (e.g., any of the radar-based applications, awareness managers, modules, or non-radar sensors described herein). The method 700 is described in Figure 8 Continue, as Figure 7 The letter "A" after block 708 corresponds to Figure 8 The letter "A" before block 710.

[0092] At 710, the electronic device is maintained in a sleep mode. For example, the awareness manager 106 may maintain the electronic device 102 in a sleep mode. The sleep mode is a persistent lower power mode, as described in reference to FIG. Figure 1 In the sleep mode, the display may be on or off, and where the display provides a touch interface, the touch interface may remain functional.

[0093] At 712, radar data is obtained from the radar system. For example, radar-based awareness manager 106 may obtain radar data from radar system 104. Radar data may be obtained in any suitable manner. Although indicated as a specific step in method 700, electronic device 102, awareness manager 106, or other systems and components of the electronic device may continue to obtain radar data (including multiple subsets of radar data) as long as the radar system is powered on.

[0094] At 714, based on the radar data and during the sleep mode, the presence of the user within the recognition area of ​​the electronic device is determined. For example, based on one or more subsets of the radar data, the radar-based awareness manager 106 can determine the presence of the user 112 within the recognition area 116 of the electronic device 102. The recognition area can take on a variety of shapes and sizes, as shown in FIG. Figure 1As described. For example, the recognition zone can be an area approximately contiguous with the radar field, a radius extending from the radar system, a volume around the radar system, or a non-uniform shape. The recognition zone can extend various distances from the radar system (e.g., three feet, seven feet, ten feet, or fourteen feet, but for larger devices, such as large screen televisions, the recognition zone can be extended to four, five, six, or seven meters). In addition, the recognition zone can be a static size or shape that is predefined, user-selectable, or determined via another method. In some cases, the recognition zone can be dynamic and automatically adjusted (e.g., by the awareness manager 106, as described in reference Figure 1 described).

[0095] At 716, in response to determining the presence of the user within the recognition area, the electronic device exits the sleep mode and enters the awareness mode, wherein the display presents one or more visual elements indicating the state of the electronic device. For example, in response to determining that the user 112 is within the recognition area 116, the awareness manager 106 may cause the electronic device 102 to exit the sleep mode and enter the awareness mode. Figure 1 As described, awareness mode is another lower power mode in which a display of the electronic device (e.g., display 114) presents one or more visual elements that may indicate a status or functionality level of the electronic device. Visual elements presented in awareness mode may include visual elements of contextual information or simple status information about the electronic device, such as those shown in FIG. Figure 1 Described. Reference Figure 10-12 Some example visual elements are described in more detail. When the user leaves the recognition area, the electronic device can return to sleep mode (immediately after determining that the user has exited the recognition area or after the user has been outside the recognition area for a period of time). For example, awareness manager 106 can determine that user 112 has turned away from electronic device 102 and / or is moving away from electronic device 102 and immediately return to sleep mode.

[0096] At 718, based on the radar data and during the awareness mode, a user action that is classified as an indication of the user's intent to interact with the electronic device is detected. For example, based on one or more other subsets of the radar data provided by radar system 104, awareness manager 106 may detect that user 112 is performing an action (e.g., reaching toward electronic device 102) and determine that the action is an indication of the user's intent to interact with electronic device 102 (e.g., by referencing the action library, as described above and referring to the user's actions). Figure 1 ). Actions that may be indicative of a user's intent to interact with an electronic device include reaching toward or turning toward the electronic device, leaning over the electronic device, and the like, as shown in FIG. Figure 1Note that different actions that are classified as indicative of user interaction intent may have different thresholds to be considered. For example, if a portion of the user's body moves within a threshold distance of the electronic device (e.g., three inches, five inches, eight inches, or twelve inches), then only the reach distance toward the electronic device may be determined to indicate the user's intent.

[0097] In some embodiments, the electronic device may further include machine learning technology that enables the electronic device to learn different or additional indications of user intent and add, remove, or modify actions stored in the library (e.g., based on the user's history and behavior with the device). In addition, if the electronic device determines that the user is in the recognition area, but no indication of interaction intent is detected, the electronic device may remain in awareness mode until such an action is detected. In other cases, even if the user remains in the recognition area, the electronic device may remain in awareness mode for a selectable and adjustable duration (e.g., 60, 30, or 10 seconds) and then transition back to sleep mode. A description of method 700 is provided in Figure 9 Continue, as Figure 8 The block 718 is indicated by the letter "B" after which corresponds to Figure 9 The letter "B" before block 720.

[0098] At 720, in response to detecting a user action indicating a user's intent to interact with the electronic device, the electronic device automatically prepares the authentication system to perform an authentication process. For example, in response to detecting an indication of a user's (e.g., user 112) intent to interact with the electronic device 102, the awareness manager 106 can cause the electronic device 102 to prepare the authentication system 118.

[0099] In some embodiments, in response to detecting a user action indicating the user's intention to interact with the electronic device, the electronic device further exits the awareness mode and enters the engagement mode, wherein the display of the electronic device presents one or more other visual elements indicating another state (or change of state) of the electronic device, such as reference Figure 1 As described in reference Figure 1 As described, the engaged mode is a higher power mode of the electronic device and the radar system. When the electronic device enters the engaged mode, the display of the electronic device (e.g., display 114) may present one or more other or additional visual elements that may indicate a new or enhanced state or level of functionality of the electronic device. The visual elements presented in the engaged mode may include a visual element (e.g., a locked padlock icon) indicating that access to the electronic device is locked and authentication is required for access. Figure 10-12 Describe other visual elements in more detail.

[0100] In the awareness or engagement mode, in response to detecting a user action indicating the user's intent to interact with the electronic device, the electronic device may also automatically prepare an authentication system (e.g., authentication system 118) to perform an authentication process. The authentication system may use any suitable authentication technology. For example, the authentication system may include or be associated with a camera (e.g., one of the non-radar sensors 108) and a facial recognition module (e.g., facial recognition module 120), which may be used to authenticate a user based on an image of the user's face. Automatic preparation may include steps such as placing the camera in a state in which it can capture images (e.g., a "warm-up" process that prepares the camera and any associated programs to capture images). The preparation may also include placing the facial recognition module in a state in which it can use the captured image to verify the user, as described in reference to FIG. Figure 1 As previously mentioned, this preparation can reduce the delay between the user's initial action for verification and the completion of the authentication process.

[0101] At 722, based on the triggering event, the prepared authentication system performs an authentication process on the user. For example, based on the triggering event, the awareness manager 106 may cause the prepared authentication system 118 to perform an authentication process on the user 112. The triggering event is an interaction that is separate from or in addition to the action of transitioning the electronic device to the optional engaged state. The triggering event indicates that the user is ready to authenticate. For example, the triggering event can be a change in the user's position or orientation, a change in the orientation of the electronic device (e.g., the user touches, picks up, leans over, or rotates the electronic device), or an explicit user action, such as a touch input, as referenced in FIG. Figure 1 Radar data or non-radar data can be used to detect a triggering event, and in the absence of a triggering event (e.g., indicating that the user does not intend to interact with the electronic device), the user is not authenticated and, for example, if the electronic device 102 is in the engaged mode, the electronic device can remain in the engaged mode for a selectable duration and then transition back to the awareness mode.

[0102] At 724, in response to the user being authenticated, the radar-based awareness manager causes the electronic device to enter an active mode in which the user has full rights and access to the electronic device. For example, in response to successfully authenticating the user 112, the awareness manager 106 may cause the electronic device 102 to exit the awareness or engagement mode and enter the active mode. In the active mode, as described in reference Figure 1 As described, the user has full rights and access to the electronic device (eg, the electronic device is unlocked and fully functional).

[0103] Optionally, at 726, when the electronic device enters the active mode, the display may present one or more different or additional visual elements that indicate another state or change of state of the electronic device. For example, in the active mode, the display 114 may present a visual element that indicates a new or enhanced state or level of functionality of the electronic device 102. The visual elements presented in the active mode may include a background image or a visual element that indicates that the electronic device 102 is unlocked (e.g., an unlocked padlock icon), as described with reference to FIG. Figure 1 As described in reference Figure 1 As described above, the visual elements displayed in the active mode may be determined based at least in part on the number of users detected within the recognition area. In some cases, the background image or visual element may be presented for a duration and then fade out or disappear. Figure 10-12 Examples of these additional or other visual elements are described in more detail.

[0104] These techniques for radar-based authentication state feedback can be more secure than other authentication and feedback techniques. For example, a user's location, orientation, or 3D gestures (especially user-defined gestures, micro-gestures, and gestures or location-based gestures) are generally not copyable or obtainable by unauthorized persons (unlike, for example, a password). In addition, a radar image of a user (e.g., based on the radar data described above), even if it includes the user's face, cannot visually identify the user like a photo or video. Even so, further to the above description, the user can be provided with controls that allow the user to choose whether and when any of the systems, programs, managers, modules, or features described in this document can implement the collection of user information (e.g., an image of the user, radar data describing the user, information about the user's social network, social actions or activities, occupation, user preferences, or the user's current location) and whether and when to send content or communications from a server to the user. In addition, certain data can be processed in one or more ways before it is stored or used so that personally identifiable information is removed. For example, the user's identity can be processed so that no personally identifiable information can be determined for the user, or the user's geographic location can be summarized (such as to a city, zip / postal code, or state level) in the case of location information so that the user's specific location cannot be determined. Thus, the user can control what information is collected about the user, how that information is used, and what information is provided to or about the user.

[0105] Example visual elements

[0106] As described above, the techniques and systems described herein can enable multiple modes of the electronic device 102 that provide different services and functionality. The mode the device is in is changed based on radar data (e.g., from a radar field such as radar field 11), which can indicate the user's current location, interaction level, and expected changes in the user's interaction level. The electronic device 102 also presents visual elements on the display 114 that indicate the functions and services that the device can provide in different modes. These visual elements also change based on the mode to help the user understand the mode in which the electronic device is operating and the services and functionality that are available. Figure 10-12 The electronic device 102 is illustrated as operating in multiple modes and describes examples of visual elements that may be presented on a display in different modes.

[0107] For example, when no user 112 is detected near the electronic device 102 (e.g., within the radar field 110 or the recognition area 116), the device operates in a sleep mode. In the sleep mode, the display 114 may present fewer visual elements than in other modes, or no visual elements. As described above, the display 114 may be turned on or off. When the awareness manager 106 determines (e.g., using radar data or one or more subsets of radar data from the radar system 104) that the user 112 is present in the recognition area 116, the electronic device 102 exits the sleep mode and enters the awareness mode. In the awareness mode, the display 114 presents one or more visual elements that may indicate the status or functionality level of the electronic device 102.

[0108] When electronic device 102 is in awareness mode, awareness manager 106 may detect a user action categorized as an indication of a user's intent to interact with electronic device 102. In response to detecting the user action, electronic device 102 may prepare authentication system 118 to perform an authentication process. In some embodiments, when awareness manager 106 detects an indication of a user's intent to interact with electronic device 102, awareness manager 106 may also cause electronic device 102 to exit awareness mode and enter engaged mode. In engaged mode, display 114 may present one or more additional or alternative visual elements that may indicate a change in the state or functionality level of electronic device 102. Awareness manager 106 may also detect a triggering event and, based on the triggering event, cause authentication system 118 to authenticate user 112. In response to detecting that user 112 is authenticated, electronic device 102 exits awareness or engaged mode and enters active mode. In active mode, display 114 may present one or more additional or alternative visual elements that may indicate a change in the state or functionality level of electronic device 102.

[0109] Figure 10An example 1000 of an electronic device transitioning from a sleep mode to an awareness mode is shown. Detailed view 1000-1 shows that the electronic device 102 is in sleep mode when the user 112 is outside the recognition area 116. In this example, the recognition area 116 has a wedge shape, but as shown in FIG. Figure 1 As noted, the recognition area can take on any suitable shape or size. Continuing with this example, in this case, display 114 does not present any visual elements in sleep mode, as shown on example display 114-1. In another detailed view 1000-2, user 112 has moved closer to electronic device 102 and awareness manager 106 has determined that user 112 has entered recognition area 116. Based on this determination, electronic device 102 exits sleep mode and enters awareness mode, as shown by arrow 1002.

[0110] In the detailed view 1000-2, a plurality of visual elements are presented on the example display 114-2. For example, in the awareness mode, the example display 114-2 presents a time of day element 1004 (clock), a date element 1006, a connectivity status element 1008 (e.g., Wi-Fi, cellular, or other network connectivity), and a battery level indicator element 1010 (including a graphical element and a percentage indicator). In the detailed view 1000-2, the rest of the example display 114-2 is blank. However, in some embodiments, additional elements may be displayed, including background images, such as wallpaper or other images. Although not shown in the Figure 10 , if the user 112 exits the recognition area 116 , the awareness manager 106 may cause the electronic device 102 to stop displaying visual elements and return to sleep mode (either immediately or after the user 112 has been in the recognition area 116 after a selectable predetermined amount of time).

[0111] Figure 11 An example 1100 of an electronic device transitioning from an awareness mode to an optional engagement mode is shown. Detailed view 1100-1 shows the electronic device 102 in the awareness mode, as shown in FIG. Figure 10 1 . The embodiment of the present invention is described, including displaying a plurality of visual elements on the example display 114-3. Another detailed view 1100-2 shows the user 112 reaching out to grab the electronic device 102. The awareness manager 106 detects the reaching out (e.g., using one or more subsets of the radar data) as a user action that is indicative of the user's intent to interact with the electronic device 102. In response to detecting this user action, the electronic device 102 exits the awareness mode and enters the engagement mode, as indicated by arrow 1102.

[0112] In detailed view 1100-2, additional visual elements are presented on example display 114-4. For example, in engagement mode, example display 114-4 presents a background image 1104 (in this case, an image of the Golden Gate Bridge). Background image 1104 can have dynamic features that adjust to the user's context, such as animation, or varying brightness or transparency levels that change depending on the distance or speed of reach. As described above, while in engagement mode, electronic device 102 also prepares authentication system 118 to perform the authentication process (note that in some cases, electronic device 102 does not enter engagement mode and prepares authentication system 118 while in awareness mode). Accordingly, example display 114-4 also presents a lock icon 1106 that indicates that full access to electronic device 102 is unavailable until user 112 is authenticated. In some embodiments, additional visual elements may be displayed on example display 114-4, and some or all of the visual elements presented on example display 114-3 may cease to be presented. Although not in Figure 11 , if user 112 withdraws the reach gesture, awareness manager 106 may cause electronic device 102 to exit engagement mode and return to awareness mode (either immediately or after the reach has been withdrawn within a selectable predetermined amount of time).

[0113] Figure 12 An example 1200 illustrating an electronic device transitioning from an engaged mode to an active mode after a user 112 is authenticated (note that in some embodiments, the electronic device may transition from an aware mode to an active mode). Detailed view 1200-1 shows the electronic device 102 in the engaged mode, as shown in FIG. Figure 11 As described, including displaying a plurality of visual elements on the example display 114-5. Figure 11 As shown, when the user reaches out to grab the electronic device 102, the authentication system 118 is ready to authenticate the user 112. Figure 12 , another detailed view 1200-2 shows that user 112 has picked up electronic device 102. Awareness manager 106 can determine that picking up electronic device 102 is a triggering event, as described above, and authenticate user 112. If user 112 is authenticated, electronic device 102 exits engaged mode and enters active mode, as shown by arrow 1202.

[0114] Additional visual elements associated with the active mode may also be presented on the example display 114-6, as shown in the detailed view 1200-2. For example, in the active mode, the example display 114-6 continues to present visual elements associated with the awareness mode, but the background image 1104 (associated with the engagement mode) has changed to another background image 1204, a beach silhouette (note that because the background image 1204 has a different color scheme, some of the visual elements have changed color so that they remain visible to the user 112). In addition, the engagement mode lock icon 1106 has been transformed into an unlock icon 1206, which indicates that the user 112 is authenticated. In some embodiments, the unlock icon 1206 may be presented for a duration and then fade out. Although Figure 12 Not shown, additional visual elements may be displayed on the example display 114-6 after the unlock icon 1206 fades out, such as instructions (e.g., "slide or tap to open"), one or more application launch icons, or other available visual elements available to the electronic device 102.

[0115] In some embodiments, user 112 may remain authenticated as long as user 112 remains within recognition zone 116 (or another defined area where the radar system can detect the presence of user 112). In these embodiments, display 114 may remain charged and able to receive input and present content, or the screen may be turned off to conserve battery power. Because user 112 remains authenticated, even when the screen is off, the user may access electronic device 102 by touching the screen, picking up the device, or other actions without having to reauthenticate. In this way, the user's enjoyment and experience of electronic device 102 may be improved while preserving battery power.

[0116] Furthermore, the described progression between modes (e.g., from sleep mode, through awareness and engagement mode, to authenticated and active mode) can operate in the opposite direction. For example, when electronic device 102 is in active mode and user 112 puts it down (e.g., another triggering event occurs), awareness manager 106 can instead lock electronic device 102 (deauthenticating user 112) and place electronic device 102 in engaged mode, as described above. Thus, if the user's hand remains near electronic device 102 (e.g., held in an "reach" position), awareness manager 106 can leave electronic device 102 in engaged mode. Conversely, if the user's hand is withdrawn, awareness manager 106 can transition electronic device 102 from engaged mode to awareness mode. Then, as noted, electronic device 102 can remain in awareness mode while the user is in a recognized area. During this progression between areas, display 114 can present the aforementioned visual elements for each area to indicate to user 112 the changing state of electronic device 102.

[0117] In some implementations, an application running on the electronic device 102 (e.g., in active mode) may be able to receive input via radar-based, remote, three-dimensional (3D) gestures. In this case, the display in awareness mode may present a visual element indicating the availability of the application to receive this type of gesture input. For example, when a radar gesture subscription application (gesture subscription application) is running, the display 114 may present an icon, a contrasting lighting area (e.g., an area that is brighter or darker than the surrounding area), or an area of ​​a different or contrasting color.

[0118] consider Figure 13 , which generally illustrates at 1300 an example visual element that can be used to indicate that a gesture-subscribing application is running on electronic device 102. Detailed view 1300-1 illustrates example display 114-7, shown in a state where no gesture-subscribing application is running. Another state is shown on example display 114-8, in which at least one gesture-subscribing application is running. As described above, visual element 1302 at the top of example display 114-8 indicates that electronic device 102 can receive input via remote 3D gestures. Visual element 1302 is shown as an illuminated line, but can be presented in another location, at a different illumination level (e.g., only partially illuminated), or as an element of another shape or type. For example, in another detailed view 1300-2, example display 114-9 is shown in a state where no gesture-subscribing application is running. Another state is shown on example display 114-10, in which at least one gesture-subscribing application is running. Visual element 1304 at the top of example display 114-10 indicates that electronic device 102 can receive input via remote 3D gestures. Visual element 1304 is shown as an illuminated area (e.g., a luminous area) having less defined edges than visual element 1302. Like visual element 1302, visual element 1304 can be presented at another location 10 on display 114. Note that for clarity, other elements (e.g., time, date, or application launch icons) are not shown on example displays 114-7 to 114-10. However, visual element 1302 or 1304 can be displayed with other content on the display and while electronic device 102 is in awareness mode, engagement mode, activity mode, or other modes.

[0119] In some embodiments, when a user interacts with the electronic device 102 using input other than remote 3D gestures (e.g., touch or voice input), a visual element indicating that a gesture-subscribing application is running (e.g., example visual element 1302 or 1304) can fade out or disappear completely. For example, when a gesture-subscribing application is operating on the electronic device 102, the user can decide to use a touch command on the electronic device to start another application (or decide to touch or press the power or lock control). In this case, when the user picks up the electronic device 102 or touches the display 114 (or the power / lock control), the visual element 1302 or 1304 can fade out or disappear. When the user stops touching the display 114 or puts down the electronic device 102, the visual element will reappear (or brighten) if one or more gesture-subscribing applications are operating on the electronic device 102. The visual element can reappear or brighten immediately or reappear or brighten after a selectable duration.

[0120] Additionally or alternatively, the display 114 may present or not present visual elements based on the operating state, screen state, power state, or functional mode of the electronic device 102, such as a sleep mode, an awareness mode, an engagement mode, or an activity mode. For example, in a sleep mode or another mode or state where the user may interact with the electronic device less frequently, or where visual elements will be presented for a longer duration without changing or hiding, the risk of screen burn-in increases (e.g., leaving ghost images where the visual elements were presented because infrequent interactions mean that the visual elements remain in the same location for longer periods of time). In these cases, the visual elements may be presented at locations that change over time (e.g., the location shifts over time to avoid being presented in one location for a long period of time).

[0121] In some cases, visual elements may also or alternatively be presented and hidden at intervals that can help prevent screen burn-in. The intervals may be of any duration appropriate to the type of display, such as one second, three seconds, five seconds, and so on. Similarly, when the electronic device 102 is in a mode or state where there is frequent interaction of different types or modes (e.g., touch, voice, and remote 3D gestures), visual elements may be presented for longer durations because the user's interaction inherently causes the electronic device 102 to show and hide visual elements, so there is less risk of screen burn-in.

[0122] The electronic device 102 may also provide more detailed visual feedback regarding the availability of remote 3D gestures. Figure 14, which generally illustrates at 1400 an example visual element that can be used to indicate that a user's hand is within a gesture zone that enables a gesture-subscribing application to receive remote 3D gesture input. The gesture zone is a region around electronic device 102 (or radar system 104) within which electronic device 102 can receive, interpret, and act upon remote 3D gestures, such as a swipe or pinch. The gesture zone can extend any suitable distance from electronic device 102 (e.g., approximately three, five, seven, or ten inches).

[0123] exist Figure 14 , an example display 114-11 is shown in a state where at least one gesture subscription application is running (e.g., similar to reference Figure 13 1004 ). The user's hand 1402 is shown near the example display 114-11, but outside the gesture area (the boundary of the gesture area is shown as dashed line 1404). Visual element 1406 is shown as an illuminated line near the top of the example display 114-11, but visual element 1406 can be presented at another location or as another shape or type of element. When the user's hand 1402 moves to cross the boundary of the gesture area 1004, as shown by arrow 1408, another visual element 1410 replaces visual element 1406, as shown on example display 114-12. In this example, visual element 1110 is a brighter line near the top of the display (e.g., brighter or more fully illuminated), but visual element 1410 can also be presented at another location or as another shape or type of element. When the user's hand 1402 is within the boundaries of gesture area 1404, visual element 1408 may move back and forth to represent a smaller, non-gestural movement of the user's hand 1402, as indicated by arrow 1412. A corresponding movement of visual element 1410 is seen on example display 114-13. When the user's hand 1402 is moved back outside the boundaries of gesture area 1404, as indicated by arrow 1416, the display returns to the state shown in example display 114-11, where visual element 1406 is displayed near the top of example display 114-11. Visual elements 1406 and 1410, along with the movement of visual element 1408, as indicated by double-headed arrow 1414, can help user 112 understand when gestures are available and provide feedback indicating that electronic device 102 is aware of the relative position of the user's hand, which can improve the user's experience with electronic device 102.

[0124] In some embodiments ( Figure 14(not shown in the figure), non-gestural motion can be represented by other visual elements or changes to visual elements. For example, where the visual element is a "luminous" shape (e.g., a shape with varying brightness, color, or other attributes), the focus of the shape can be shifted to represent motion while the shape itself remains stationary. Additionally or alternatively, changes in the intensity of brightness or color can be used to represent motion (e.g., the brightness or color of a shape or portion of a shape changes in accordance with the non-gestural motion).

[0125] As reference Figure 14 As described, the electronic device 102 may provide visual feedback to indicate smaller, non-gesture movements of the user's hand 1402 in the gesture area, such as moving back and forth in accordance with the movement of the user's hand. Similarly, when the user 112 makes a remote 3D gesture (e.g., a swipe gesture to skip a song or an omnidirectional gesture to dismiss an alert or notification), the electronic device 102 may provide feedback to notify the user 112 that the gesture was successfully received or that a gesture attempt was received but was not clear enough to be confirmed as a remote 3D gesture. For example, Figure 15 A sequence of example visual elements that may be used to notify the user 112 that a gesture has been successfully received is illustrated generally at 1500 .

[0126] exist Figure 15 , an example display 114-14 is shown in a state where at least one gesture-subscribing application is running and a user's hand 1502 is within the boundaries of a gesture area, such as gesture area 1404 (e.g., similar to reference 1404). Figure 14 12). The example display 114-14 is presenting a visual element 1504, shown as an illuminated line near the top of the example display 114-14, to indicate the availability of remote 3D gestures and that the user's hand 1502 is within the gesture area. Figure 15 In the example of FIG14 , user 112 performs a sliding or swiping gesture from left to right, as indicated by arrow 1506. In response to the movement of user's hand 1502, indicated by arrow 1506, visual element 1504 also moves, as shown in sequence 1508 (shown within a dashed rectangle). In sequence 1508, example display 114-15 illustrates the beginning of sequence 1508 when visual element 1504 begins to move to the right, as indicated by arrow 1510.

[0127] Sequence 1508 continues in another example display 114-16, where visual element 1504 bends around the corner of example display 114-16, as indicated by arrow 1512. Visual element 1504 can continue down a side of display 114 for a variable distance (e.g., once the trailing end of the visual element completes the bend or after the trailing end has traveled a particular distance along the side) and then disappear. In another example display 114-17, visual element 1504 reappears or regenerates from the left side of example display 114-17 and moves toward a center position, as indicated by arrow 1514. When sequence 1508 is complete, the display returns to the state shown in example display 114-14, where visual element 1504 is displayed near the top of example display 114-14 and subtly tracks the user's hand 1502 while the user's hand remains within the boundaries of gesture area 1404. The movement of the visual element 1504 may help the user 112 understand when a gesture has been accepted and when the gesture is complete, which may improve the user's experience with the electronic device 102 .

[0128] Note that sequence 1508 begins when user 112 begins the gesture, but the gesture and sequence 1508 may be completed at different times. Also, as described above, while visual element 1504 is shown as a partially illuminated line near the top of display 114, visual element 1504 may be presented at another location or as another shape or type of element. Sequence 1508 may also begin at another location on display 114 and continue in another direction, such as from right to left, top to bottom, or bottom to top (e.g., if the Remote 3D gesture moves from right to left, top to bottom, or bottom to top). Other examples of sequences showing successful Remote 3D gestures ( Figure 15 Examples (not shown) include visual elements that fold over on themselves, temporarily disappear, and then regenerate (e.g., from their original positions). Other examples include visual elements that bend or flex (e.g., at one end, both ends, in the middle, or at another location) to illustrate a successful remote 3D gesture, such as a gesture made in a direction perpendicular to display 114 or a gesture having a component perpendicular to display 114.

[0129] In some cases, as described herein, visual elements can be hidden even when Remote 3D gestures are available (e.g., because the user is interacting with voice or touch input, or to reduce the risk of screen burn-in). In this case, the visual elements can still be shown when the user makes a successful Remote 3D gesture. Consider an example where the visual elements are hidden when the user is listening to music and uses voice input to open another application. In this example, the user performs a Remote 3D gesture to skip a song, and the display presents sequence 1508 to notify the user that the Remote 3D gesture was successful.

[0130] Figure 16 A sequence of example visual elements that may be used to notify the user 112 that a gesture was not successfully received is illustrated generally at 1600. Figure 16 , an example display 114-18 is shown in a state where at least one gesture-subscribing application is running and a user's hand 1602 is within the boundaries of a gesture area (such as gesture area 1404) (e.g., similar to reference 1404). Figure 14 Example display 114-12 is depicted in the example display 114-18. Example display 114-18 is presenting a visual element 1604, shown as an illuminated line near the top of example display 114-18, to indicate the availability of remote 3D gestures and that the user's hand 1602 is within the gesture area. Figure 16 In the example of , user 112 attempts to perform a sliding or swiping gesture from left to right, but fails to meet sufficient criteria for a swipe gesture. For example, as shown by curved arrow 1606, the user's hand 1602 may not be able to travel a sufficient distance in the relevant direction before withdrawing. In this case, when electronic device 102 (or radar system 104) detects movement 1606 of user's hand 1602, it lacks sufficient definition to be successfully determined as a long-range 3D gesture, and visual element 1604 moves as shown in sequence 1608 (shown within the dashed rectangle). In sequence 1608, example display 114-19 illustrates the beginning of sequence 1608 when visual element 1604 begins to move to the right, as shown by arrow 1610.

[0131] Continuing with sequence 1608, another example display 114-20 shows that the visual element 1604 has stopped and shrunk (compared to its starting length as shown in example display 114-18) before reaching the edge of example display 114-20. In another example display 114-21, the visual element 1604 reverses direction, as shown by another arrow 1612, begins to move back toward its original position (center in this example), and begins to recover to its original length. In other embodiments, instead of stopping and shrinking, the visual element 1604 can slow down and rebound before reversing direction. When sequence 1608 is completed, the display returns to the state shown in example display 114-18, where the visual element 1604 is displayed near the top of the example display 114-18 and subtly tracks the user's hand 1602 while the user's hand 1602 remains within the boundaries of the gesture area 1404. The movement of the visual element 1604 can help the user 112 understand when a gesture has not been successfully completed, so that the user can learn techniques for making successful remote 3D gestures and become aware when an attempted gesture fails (e.g., so that it can be tried again if necessary), which can improve the user's experience with the electronic device 102.

[0132] Note that sequence 1608 may begin when electronic device 102 (or awareness manager 106) (e.g., using one or more subsets of radar data) detects that user 112 has attempted a remote 3D gesture, but also determines that the gesture failed to meet at least one criterion necessary for acceptance. Thus, depending on the nature of the attempted gesture and the speed of sequence 1608, the attempted gesture and sequence 1608 may complete at different times. Furthermore, as described above, while visual element 1604 is shown as a partially illuminated line near the top of display 114, visual element 1604 may be presented at another location or as another shape or type of element. Sequence 1608 may also begin at another location on display 114 and continue in another direction, such as from right to left, top to bottom, or bottom to top (e.g., if the attempted remote 3D gesture moves from right to left, top to bottom, or bottom to top). Other examples of sequences illustrating unsuccessful remote 3D gesture attempts include a visual element that partially collapses on itself, such as by briefly contracting, and then returns to its original size and position.

[0133] In some embodiments, electronic device 102 includes a gesture pause mode that can disable or pause the Remote 3D gesture capabilities of electronic device 102 when conditions indicate the system may be inefficient or ineffective at receiving or interpreting gestures. For example, when electronic device 102 moves at a speed above a threshold, or when the direction of movement of electronic device 102 changes rapidly and repeatedly, the electronic device can enter gesture pause mode and provide visual feedback to the user. Electronic device 102 can determine to enter gesture pause mode based on input from any of a variety of sensors, including a radar sensor (e.g., radar system 104), an inertial measurement unit (IMU), a proximity sensor (e.g., an active infrared proximity sensor), and the like. For example, if user 112 is walking and listening to audio content, with electronic device 102 swinging back and forth in the user's hand, this motion can resemble a Remote 3D-based swipe gesture, but user 112 does not intend to skip tracks or adjust the volume. Therefore, because the motion of electronic device 102 introduces ambiguity into the gesture interpretation process, electronic device 102 can determine to enter gesture pause mode until the ambiguity is resolved (e.g., user 112 stops walking).

[0134] consider Figure 17 , which illustrates generally at 1700 an example visual element that can be used to indicate that a gesture-subscribing application is available to receive remote 3D gesture input but gestures are currently paused. Gesture pause mode can be activated whenever remote 3D gestures are available, regardless of whether the user's hand is within or outside the gesture area. Figure 17, an example display 114-22 is shown in a state where at least one gesture-subscribing application is running and a user's hand 1702 is within the boundaries of a gesture area 1704 (e.g., similar to reference 1704). Figure 14 17. Example display 114-12 is depicted in FIG. 17. Example display 114-22 is presenting visual element 1708, shown as an illuminated line near the top of example display 114-22, to indicate the availability of remote 3D gestures and that the user's hand 1702 is within the gesture area. If user 112 takes an action that causes electronic device 102 to enter gesture pause mode (e.g., user's hand 1702 begins moving back and forth while user 112 walks, as shown by arrow 1706), visual element 1708 changes, as shown in sequence 1710 (within the dashed rectangle).

[0135] Example display 114-23 illustrates the beginning of sequence 1710, with another visual element 1712 replacing visual element 1708 in response to electronic device 102 detecting motion indicated by arrow 1706. As shown on example display 114-23, visual element 1712 is another line that is shorter and darker than visual element 1708. Sequence 1710 continues in another example display 114-24, where visual element 1712 begins moving to the right, as indicated by arrow 1714. Sequence 1710 continues in another example display 114-25, where visual element 1712 moves to the left, as indicated by arrow 1716. During sequence 1710, visual element 1712 may stop and reverse direction before reaching a side of the display, or may go all the way to the edge before reversing direction. In some embodiments, visual element 1712 may further shrink to reverse direction when it stops, and then return to its original size upon, after, or as it begins moving in the opposite direction. In addition, the oscillation of the visual element 1712 can match the conditions on which the gesture pause mode is based. For example, if the user's arm is swinging, the speed or frequency of the oscillation of the visual element 1712 can approximately match the speed or frequency of the user's hand movement.

[0136] As described above, although visual element 1712 is shown as a partially illuminated line near the top of display 114, visual element 1712 may be presented in another location or as another shape or type of element. Sequence 1710 may also begin at another location on display 114 and continue in another direction, such as from right to left, top to bottom, or bottom to top (e.g., depending on the orientation of any content on display 114 or another factor).

[0137] When the electronic device 102 exits gesture pause mode, the sequence 1710 is completed and the display returns to the appropriate state, depending on whether a gesture-subscribing application is running and the position of the user's hand 1702. This sequence 1710 of motion of the visual element 1712 can help the user 112 understand when it is time to pause a gesture and allow the user 112 to adjust how they use the electronic device to avoid or take advantage of gesture pause mode, which can improve the user's experience with the electronic device 102.

[0138] In some embodiments, the electronic device 102 can determine the background color of a region of the display 114 on which the reference image is displayed. Figures 13 to 17 The visual element described is displayed or will be displayed. In response to determining the background color, the electronic device 102 can cause the display 114 to present the visual element in another color different from the background color, and can provide a human-discernible contrast between the visual element and the background color to make it easier for the user 112 to see the visual element. In some cases, the electronic device 102 can continuously, automatically, and dynamically adjust the color of the visual element based on changes to the background color.

[0139] Example computing system

[0140] Figure 18 Illustrated are various components of an example computing system 1800 that can be used as described with reference to the previous Figures 1 to 17 Any type of client, server, and / or electronic device described is implemented to implement radar-based authentication status feedback.

[0141] Computing system 1800 includes a communication device 1802 that can implement wired and / or wireless communication of device data 1804 (e.g., radar data, authentication data, reference data, received data, data being received, data scheduled for broadcast, and data packets of the data). Device data 1804 or other device content can include configuration settings for the device, media content stored on the device, and / or information associated with the user of the device (e.g., the identity of a person within the radar field or customized gesture data). Media content stored on computing system 1800 can include any type of radar, biometric, audio, video, and image data. Computing system 1800 includes one or more data inputs 1806 that can receive any type of data, media content, and / or input, such as human speech, interaction with the radar field (e.g., radar 3D gestures), touch input, user-selectable input or interaction (explicit or implicit), messages, music, television media content, recorded video content, and any other type of audio, video, and / or image data received from any content and / or data source.

[0142] Computing system 1800 also includes a communication interface 1808, which can be implemented as any one or more of a serial and / or parallel interface, a wireless interface, any type of network interface, a modem, and any other type of communication interface. Communication interface 1808 provides a connection and / or communication link between computing system 1800 and a communication network through which other electronic, computing, and communication devices communicate data with computing system 1800.

[0143] Computing system 1800 includes one or more processors 1810 (e.g., any of a microprocessor, controller, or other controller) capable of processing various computer-executable instructions to control the operation of computing system 1800 and enable techniques for radar-based authentication status feedback or in which radar-based authentication status feedback can be implemented. Alternatively or additionally, computing system 1800 can be implemented using any one or a combination of hardware, firmware, or fixed logic circuitry, along with processing and control circuitry generally indicated at 1812. Although not shown, computing system 1800 can include a system bus or data transfer system that couples various components within the device. The system bus can include any one or a combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus utilizing any of a variety of bus architectures. Also not shown, computing system 1800 can include one or more non-radar sensors, such as non-radar sensor 108.

[0144] The computing system 1800 also includes computer-readable media 1814, such as one or more storage devices capable of persistent and / or non-transitory data storage (e.g., as opposed to mere signal transmission), examples of which include random access memory (RAM), non-volatile memory (e.g., any one or more of read-only memory (ROM), flash memory, EPROM, EEPROM, etc.), and disk storage devices. The disk storage device can be implemented as any type of magnetic or optical storage device, such as a hard drive, a recordable and / or rewritable compact disk (CD), any type of digital versatile disk (DVD), etc. The computing system 1800 can also include a mass storage media device (storage media) 1816.

[0145] Computer-readable media 1814 provides a data storage mechanism for storing device data 1804, as well as various device applications 1818 and any other type of information and / or data related to the operation of computing system 1800. For example, operating system 1820 can be maintained as a computer application on computer-readable media 1814 and executed on processor 1810. Device applications 1818 may include device managers, such as any form of control applications, software applications, signal processing and control modules, device-specific code, abstraction modules, gesture recognition modules, and / or other modules. Device applications 1818 may also include system components, engines, modules, or managers for implementing radar-based authentication status feedback, such as radar system 104, awareness manager 106, authentication system 118, or facial recognition module 120. Computing system 1800 may also include or have access to one or more machine learning systems.

[0146] Several examples are described below.

[0147] Example 1. An electronic device comprising:

[0148] monitor,

[0149] A radar system, the radar system being implemented at least partially in hardware, the radar system being configured to:

[0150] providing radar fields;

[0151] sensing a reflection from a user in the radar field;

[0152] analyzing the reflection of the user in the radar field; and

[0153] providing radar data based on analysis of the reflections;

[0154] one or more computer processors; and

[0155] One or more computer-readable media having instructions stored thereon that, in response to execution by the one or more computer processors, implement a radar-based awareness manager, the radar-based awareness manager configured to:

[0156] maintaining the electronic device in a sleep mode;

[0157] determining, based on the first subset of the radar data, the presence of the user within a recognition area of ​​the electronic device;

[0158] In response to determining the presence of the user within the recognition area, causing the electronic device to enter an awareness mode in which the display presents a first visual element indicating a first state of the electronic device;

[0159] detecting, based on a second subset of the radar data, a user action categorized as indicative of a user intent to interact with the electronic device;

[0160] in response to detecting the user action categorized as an indication of the user's intent to interact with the electronic device, causing the electronic device to prepare an authentication system to perform an authentication process;

[0161] causing the prepared authentication system to perform the authentication process on the user based on a trigger event; and

[0162] In response to the user being authenticated, the electronic device is caused to enter an active mode.

[0163] Example 2. An electronic device according to Example 1, wherein the first visual element indicating the first state of the electronic device in the awareness mode is one or more of a displayed time, a displayed date, a connectivity status, or a battery charge indicator.

[0164] Example 3. An electronic device according to Example 1 or 2, wherein the radar-based awareness manager is further configured to, in response to detection of the user action that is classified as an indication of the user intent to interact with the electronic device, cause the electronic device to enter an engagement mode in which the display presents a second visual element indicating a second state of the electronic device.

[0165] Example 4. An electronic device according to Example 3, wherein the second visual element indicating that the electronic device is in the second state of the engagement mode is one or more of a background image or a visual element indicating that the electronic device is locked.

[0166] Example 5. The electronic device of Example 3 or 4, wherein the radar-based awareness manager is further configured to cause the electronic device to exit the engagement mode before entering the activity mode in response to the user being authenticated.

[0167] Example 6. The electronic device of any of Examples 1-5, wherein the authentication system is associated with a camera and a facial recognition module, and preparing the authentication system to perform the authentication process comprises:

[0168] causing the camera to enter a state in which the camera can capture images; and

[0169] The facial recognition module is caused to enter a state in which the facial recognition module can authenticate the user using the captured image. Among the advantages of the described embodiments, include embodiments in which radar sensing is used to detect the presence of a user within a recognition area, and further include embodiments in which radar is used to detect user actions classified as indications of the user's intent to interact with the electronic device, either of which can alternatively be implemented using an on-device camera provided with most modern smartphones, with the power usage of the radar facility being significantly less than that of the camera facility, while the adequacy of the radar facility's results can generally be better than the adequacy of the camera facility's results. For example, using the radar device described above, the desired user presence or user intent detection can be achieved at average powers ranging from single-digit milliwatts to only tens of milliwatts (e.g., 10mW, 20mW, 30mW, or 40mW), even including the processing power used to process radar vector data to make the determination. At these low power levels, it is readily acceptable to have the radar facility in an always-on state. Thus, for example, with the smartphone radar facility in an always-on state, a user who has been sitting across the room from their smartphone for many hours can still be provided with the desired enjoyable and seamless experience described herein. In contrast, the optical cameras included with most current smartphones typically operate at power levels of several hundred milliwatts (e.g., on the order of 400 mW, above 40 mW). At these power levels, optical cameras would be disadvantageous because they would significantly reduce the battery life of most current smartphones, making it highly impractical, if not prohibitive, to have them always on. Another advantage of radar devices is that their field of view can be quite large, even when the user is lying flat and face-up on a tabletop, easily enough to detect a user walking upward from any direction (in many typical implementations, the radar chip faces outward in the same general direction as the selfie camera). Furthermore, due to their Doppler processing capabilities, they can be highly effective at detecting even relatively subtle movements of moving objects from all directions (particularly at operating frequencies near 60 GHz). Furthermore, radar systems can operate in environments where the performance of camera systems is degraded or limited. For example, in low-light environments, camera systems may have a reduced ability to detect shapes or movement. Conversely, radar systems perform just as well in low-light conditions as in full light. The radar system can also detect presence and gestures through some obstructions. For example, if a smartphone is in a pocket, jacket, or pants, the camera system cannot detect the user or gestures. However, the radar system can still detect objects in its field of view even through fabric that would obstruct the camera system. A further advantage of using a radar system over a smartphone's onboard camera system is privacy, allowing users to have the benefits of the enjoyable and seamless experience described herein without having to worry about the presence of cameras recording them for such purposes.

[0170] Example 7. An electronic device according to any one of Examples 1-6, wherein causing the electronic device to enter the active mode further includes causing the display to present a third visual element indicating a third state of the electronic device, the third visual element being at least one of: a background image, a wallpaper, a home screen, a screen having a visual element indicating that the electronic device is in an unlocked state, or a screen of an application that was most recently opened before the electronic device entered the locked state.

[0171] Example 8. The electronic device of Example 7, wherein the radar-based awareness manager is further configured to:

[0172] determining whether another user is present within a recognition area of ​​the electronic device based on at least one of the first subset of radar data or another subset of radar data; and

[0173] The third visual element indicative of a third state of the electronic device is determined based at least in part on a determination of whether the other user is present within the recognition area.

[0174] Example 9. An electronic device according to any of Examples 1-8, wherein the user action classified as an indication of the user's intent to interact with the electronic device is a reach-out gesture.

[0175] Example 10. An electronic device according to any of Examples 1-9, wherein the triggering event is at least one of: a change in user position, a change in orientation of the electronic device, or an explicit touch input.

[0176] Example 11. An electronic device according to any of Examples 1-10, wherein the first subset of radar data and the second subset of radar data are based on reflections from the user in the radar field at separate times.

[0177] Example 12. A method implemented in an electronic device including a radar system and a radar-based awareness manager, the method comprising:

[0178] providing a radar field by the radar system;

[0179] sensing, by the radar system, a reflection from a user in the radar field;

[0180] analyzing, by the radar system, the reflection of the user in the radar field;

[0181] providing radar data by the radar system and based on the reflection analysis;

[0182] maintaining, by the radar-based awareness manager, the electronic device in a sleep mode;

[0183] determining, based on the first subset of the radar data, the presence of the user within a recognition area of ​​the electronic device;

[0184] In response to determining the presence of the user within the identification area, causing the electronic device to enter an awareness mode, wherein the display presents a first visual element indicative of a first state of the electronic device;

[0185] detecting, based on a second subset of the radar data, a user action categorized as indicative of a user intent to interact with the electronic device;

[0186] in response to detecting the user action categorized as the indication of the user's intent to interact with the electronic device, causing the electronic device to prepare an authentication system to perform an authentication process;

[0187] causing the prepared authentication system to perform the authentication process on the user based on a triggering event by the radar-based awareness manager; and

[0188] In response to the user being authenticated, the electronic device is caused by the radar-based awareness manager to enter an active mode.

[0189] Example 13. The method according to Example 12 further includes: in response to detecting the user action that is classified as an indication of the user intention to interact with the electronic device, causing the electronic device to enter an engagement mode, in which the display presents a second visual element indicating a second state of the electronic device.

[0190] Example 14. The method of Example 12 or 13, wherein causing the electronic device to enter the active mode by the radar-based awareness manager further comprises causing the electronic device to exit the engaged mode before entering the active mode.

[0191] Example 15. The method of any of Examples 12-14, wherein:

[0192] The first visual element indicating the first state of the electronic device in the awareness mode is at least one of a displayed time, a displayed date, a connectivity status, or a battery level indicator; and

[0193] The second visual element indicating that the electronic device is in the second state of the engagement mode is one or more of a background image or a visual element indicating that the electronic device is locked.

[0194] Example 16. The method of any of Examples 12-15, wherein the authentication system is associated with a camera and a facial recognition module, and preparing the authentication system to perform the authentication process comprises:

[0195] causing the camera to enter a state in which the camera can capture images; and

[0196] The facial recognition module is brought into a state where the facial recognition module can authenticate the user using the captured image.

[0197] Example 17. The method of any of Examples 12-16, wherein causing the electronic device to enter the active mode further comprises causing the display to present a third visual element indicative of a third state of the electronic device.

[0198] Example 18. The method of Example 17, wherein:

[0199] The third visual element is at least one of a background image or a visual element indicating that the electronic device is unlocked; and

[0200] The background image or the visual element is presented for a duration and stops being presented.

[0201] Example 19. The method of any of Examples 12-18, wherein the user action classified as the indication of the user intent to interact with the electronic device is a reach-out gesture.

[0202] Example 20. The method of any one of Examples 12-19, wherein the triggering event is at least one of: a change in user position, a change in orientation of the electronic device; and explicit touch input.

[0203] in conclusion

[0204] Although embodiments of techniques and apparatus for enabling radar-based authentication state feedback have been described in language specific to features and / or methods, it is to be understood that the subject matter of the appended claims is not necessarily limited to the specific features or methods described. Rather, these specific features and methods are disclosed as example embodiments of enabling radar-based authentication state feedback.

Claims

1. An electronic device comprising: monitor, A radar system, the radar system being implemented at least partially in hardware, the radar system being configured to: providing radar fields; sensing reflections from users in the radar field; analyzing the reflections from the user in the radar field; as well as providing radar data based on said analyzing of said reflections; one or more computer processors; as well as One or more computer-readable media having instructions stored thereon that, in response to execution by the one or more computer processors, implement a radar-based awareness manager, the radar-based awareness manager configured to: maintaining the electronic device in a sleep mode, wherein the sleep mode reduces power consumption; determining, by the radar-based awareness manager and based on a first subset of the radar data, a presence of the user within a recognition area of ​​the electronic device; In response to determining the presence of the user within the recognition area, causing the electronic device to enter an awareness mode in which the display presents a first visual element indicative of a first state of the electronic device and subsequently: detecting, by the radar-based awareness manager and based on a second subset of the radar data, a user action categorized as indicative of the user's intent to interact with the electronic device; causing the electronic device to prepare an authentication system to perform an authentication process in response to detecting the user action categorized as indicative of the user's intent to interact with the electronic device; enabling the prepared authentication system to perform the authentication process on the user based on a triggering event; and In response to the user being authenticated, the electronic device is caused to enter an active mode, the active mode being a higher power mode in which the user has full rights and access to the electronic device.

2. The electronic device according to claim 1, wherein The first visual element indicating the first state of the electronic device in the awareness mode is one or more of a displayed time, a displayed date, a connectivity status, and a battery level indicator.

3. The electronic device according to claim 1, wherein The radar-based awareness manager is further configured to, in response to detecting the user action that is classified as indicative of the user's intent to interact with the electronic device, cause the electronic device to enter an engagement mode in which the display presents a second visual element indicative of a second state of the electronic device.

4. The electronic device according to claim 3, wherein The second visual element indicating the second state of the electronic device in the engagement mode is one or more of a background image and a visual element indicating that the electronic device is locked.

5. The electronic device according to claim 3, wherein The radar-based awareness manager is further configured to, in response to the user being authenticated, cause the electronic device to exit the engaged mode before entering the active mode. The electronic device according to claim 1 , wherein: The authentication system is associated with a camera and a facial recognition module, and preparing the authentication system to perform the authentication process includes: causing the camera to enter a state in which the camera is capable of capturing images; and The facial recognition module is caused to enter a state in which the facial recognition module is able to authenticate the user using the captured image.

7. The electronic device according to claim 1, wherein Causing the electronic device to enter the active mode further includes causing the display to present a third visual element indicating a third state of the electronic device, the third visual element being at least one of: a background image, a wallpaper, a home screen, a screen having a visual element indicating that the electronic device is in an unlocked state, and a screen of an application that was most recently opened before the electronic device entered the locked state.

8. The electronic device according to claim 7, wherein: The radar-based awareness manager is further configured to: determining whether another user is present within the identification area of ​​the electronic device based on at least one of the first subset of radar data and the other subset of radar data; as well as The third visual element indicative of the third state of the electronic device is determined based at least in part on a determination of whether the other user is present within the recognition area.

9. The electronic device according to claim 1, wherein The triggering event includes at least one of the following: Changes in user location; a change in the orientation of the electronic device; and Explicit touch input.

10. The electronic device according to claim 1, wherein The first subset of the radar data and the second subset of the radar data are based on reflections from the user in the radar field at separate times.

11. The electronic device according to claim 1 , wherein: The electronic device includes machine learning technology configured to generate a machine learning model trained at least in part based on the user's history and behavior with the device; and The radar-based awareness manager is further configured to: detecting, via a library associated with the electronic device, a user action categorized as indicative of the user's intent to interact with the electronic device, the library including actions categorized as indicative of the user's intent to interact with the device or not to interact with the device; as well as Determining, using the machine learning model, the addition, removal, or modification of actions in the library that are categorized as indicative of the user's intent to interact with the device or not to interact with the device.

12. A method implemented in an electronic device including a radar system and a radar-based awareness manager, the method comprising: providing a radar field by the radar system; sensing, by the radar system, a reflection from a user in the radar field; analyzing, by the radar system, the reflection from the user in the radar field; providing radar data by the radar system and based on the analysis of the reflections; maintaining, by the radar-based awareness manager, the electronic device in a sleep mode, wherein the sleep mode reduces power consumption; determining, by the radar-based awareness manager and based on the first subset of the radar data, the presence of the user within a recognition area of ​​the electronic device and then: In response to determining the presence of the user within the recognition area, causing the electronic device to enter an awareness mode in which a display of the electronic device presents a first visual element indicating a first state of the electronic device; detecting, by the radar-based awareness manager and based on a second subset of the radar data, a user action categorized as indicative of the user's intent to interact with the electronic device; causing the electronic device to prepare an authentication system to perform an authentication process in response to detecting the user action categorized as indicative of the user's intent to interact with the electronic device; The radar-based awareness manager causes the prepared authentication system to perform the authentication process on the user based on a trigger event; as well as In response to the user being authenticated, the electronic device is caused by the radar-based awareness manager to enter an active mode, wherein the active mode is a higher power mode in which the user has full rights and access to the electronic device.

13. The method according to claim 12, further comprising: In response to detecting the user action categorized as indicative of the user's intent to interact with the electronic device, the electronic device is caused to enter an engagement mode in which the display presents a second visual element indicative of a second state of the electronic device.

14. The method according to claim 13, wherein: the first visual element indicating the first state of the electronic device in the awareness mode is at least one of a displayed time, a displayed date, a connectivity status, and a battery level indicator; as well as The second visual element indicating the second state of the electronic device in the engagement mode is one or more of a background image and a visual element indicating that the electronic device is locked.

15. The method according to claim 12, wherein: The authentication system is associated with a camera and a facial recognition module, and preparing the authentication system to perform the authentication process includes: causing the camera to enter a state in which the camera is capable of capturing images; and The facial recognition module is caused to enter a state in which the facial recognition module is able to authenticate the user using the captured image.

16. The method according to claim 12, wherein Causing the electronic device to enter the active mode further includes causing the display to present a third visual element indicative of a third state of the electronic device.

17. The method according to claim 16, wherein: The third visual element is at least one of a background image and a visual element indicating that the electronic device is unlocked; and The background image or the visual element is presented for a duration and then stops being presented.

18. The method according to claim 12, wherein The triggering event is at least one of the following: Changes in user location; a change in the orientation of the electronic device; and Explicit touch input.

19. The method of claim 12, wherein: The electronic device further includes machine learning technology configured to generate a machine learning model, the machine learning model trained at least in part based on the user's history and behavior with the device; as well as The method further comprises: detecting user actions categorized as indicative of the user's intent to interact with the electronic device using a library associated with the electronic device, the library including actions categorized as indicative of the user's intent to interact with the device or not to interact with the device; as well as Determining, using the machine learning model, the addition, removal, or modification of actions in the library that are categorized as indicative of the user's intent to interact with the device or not to interact with the device.

20. A method implemented in an electronic device including a radar system and a radar-based awareness manager, the method comprising: providing a radar field by the radar system; sensing, by the radar system, a reflection from a user in the radar field; analyzing, by the radar system, the reflection from the user in the radar field; providing radar data by the radar system and based on the analysis of the reflections; maintaining, by the radar-based awareness manager, the electronic device in a sleep mode, wherein the sleep mode reduces power consumption; determining, by the radar-based awareness manager and based on a first subset of the radar data, a presence of the user within a recognition area of ​​the electronic device; In response to determining the presence of the user within the recognition area, causing the electronic device to enter an awareness mode in which a display of the electronic device presents a first visual element indicating a first state of the electronic device and subsequently: detecting, by the radar-based awareness manager and based on a second subset of the radar data, a user action categorized as indicative of the user's intent to interact with the electronic device and the detecting being performed using a library associated with the electronic device and including user actions categorized as indicative of the user's intent to interact with the device or not to interact with the device; causing the electronic device to prepare an authentication system to perform an authentication process in response to detecting the user action categorized as indicative of the user's intent to interact with the electronic device; The radar-based awareness manager causes the prepared authentication system to perform the authentication process on the user based on a trigger event; in response to the user being authenticated, causing the electronic device to enter, by the radar-based awareness manager, an active mode, wherein the active mode is a higher power mode in which the user has full rights and access to the electronic device; as well as Additions, removals, or modifications to actions in the library that are categorized as indications of the user's intent to interact with or not interact with the device are determined using a machine learning model trained at least in part based on the user's history and behavior with the device.

Images