Smartphone, system and method implemented in an electronic device

Abstract

The present invention relates to smartphones, systems and methods implemented in electronic devices. Described herein are techniques and systems that enable smartphones, systems and methods implemented in electronic devices. The techniques and systems use a radar field to accurately determine a user's position and body orientation relative to an electronic device, such as a smartphone. The radar field also enables the device to receive 3D gestures from the user to interact with the device. The techniques allow the device to provide functionality based on the user's presence and orientation, and to appropriately adjust the timing, content and format of the device's interaction with the user.

Description

[0001] Divisional

[0002] This application is a divisional of China Patent Application No. 201980017923.9, filed August 5, 2019, which has a priority date of August 5, 2019. BACKGROUND

[0003] Applications on smartphones and other electronic devices provide a growing array of productivity, entertainment, and communication functions that have become nearly essential. We listen to music, watch videos, and share presentations and documents. These applications remind us of appointments (and sometimes even schedule them), notify us when someone is texting or calling, and keep track of projects in work, school, and home. Moreover, as these applications, and the devices that run them, become more familiar with our schedules and preferences, they suggest alternative routes of transportation, suggest restaurants and otherwise communicate independently with us. But, despite all the help our applications provide us, with all their computing power and artificial intelligence, they remain socially unaware and can become intrusive. That is, no matter how “smart” a smartphone is, when it displays or plays a reminder, alert, or suggestion, it does not know what its user is doing (or even if the user is near the device). So, if we move to the other side of the device, or try to share the displayed content with others across a table, the content can be upside down. If we rotate the device, it can not recognize what we want to do until we pick it up, orient it to where we want it, and put it down again. In other cases, the device can interrupt us at an inconvenient time, or display a reminder or notification in an inappropriate location or at an embarrassing volume or brightness, which can include personal and private information. So, taking advantage of the powerful and interesting features of our applications can be inconvenient, embarrassing, and frustrating, and we can not realize the full potential of electronic devices and applications because of their limited awareness. SUMMARY

[0004] This document describes techniques and systems that enable smartphones, systems, and methods to be implemented in electronic devices. These techniques and systems use a radar field to accurately determine a user’s position and body orientation relative to an electronic device, such as a smartphone. The radar field also enables the device to receive three-dimensional (3D) gestures from the user to interact with the device. These techniques allow the device to provide functionality based on the user’s presence and orientation, and allow the timing, content, and format of the device’s interaction with the user to be adjusted appropriately.

[0005] Aspects described below include a system that includes a smartphone, a display, a radar system, one or more computer processors, and one or more computer-readable media. The radar system is implemented at least in part in hardware and provides a radar field. The radar system also senses reflections from objects in the radar field and analyzes the reflections from the objects in the radar field. The radar system further provides radar data based on the analysis of the reflections. The one or more computer processors include stored instructions that, when executed by the one or more computer processors, perform operations. The operations include determining an orientation of the smartphone relative to the objects based on a first subset of the radar data and providing an orientation-based function of the smartphone in response to determining the orientation of the smartphone. The operations also include determining a change in the orientation of the smartphone relative to the objects based on a second subset of the radar data and modifying the orientation-based function of the smartphone in response to the change in the orientation.

[0006] Aspects described below also include a system that includes an electronic device, a radar system, one or more computer processors, and one or more computer-readable media. The radar system is implemented at least in part in hardware and provides a radar field. The radar system also senses reflections from objects in the radar field and analyzes the reflections from the objects in the radar field. The radar system further provides radar data based on the analysis of the reflections. The one or more computer-readable media include stored instructions that, when executed by the one or more computer processors, perform operations. The operations include determining a presence of an object within a perception distance of the electronic device based on a first subset of the radar data and providing a presence-based function of the electronic device in response to determining the presence of the object within the perception distance. The operations also include determining that the object is outside the perception distance of the electronic device based on a second subset of the radar data and ceasing to provide the presence-based function in response to determining that the object is outside the perception distance.

[0007] Aspects described below also include a method implemented in an electronic device that includes a radar system and a radar-based application. The method includes providing, by the radar system, a radar field and sensing, by the radar system, reflections from objects in the radar field. The method also includes analyzing the reflections from the objects in the radar field and providing radar data based on the analysis of the reflections. The method further includes determining an orientation of the electronic device relative to the objects based on a first subset of the radar data and providing an orientation-based function of the electronic device in response to determining the orientation of the electronic device. The method also includes determining a change in the orientation of the electronic device relative to the objects based on a second subset of the radar data and modifying the orientation-based function of the electronic device in response to the change in the orientation.

[0008] The aspects described below also include a system that includes an electronic device that includes or is associated with an apparatus to provide a radar field that provides radar data based on sensing and analyzing reflections from objects in the radar field. The system also includes an apparatus to determine an orientation of the electronic device relative to an object in the radar field and to provide an orientation-based function of the electronic device in response to determining the orientation. The system also includes an apparatus to determine a change in the orientation of the electronic device relative to the object in the radar field and to modify the orientation-based function of the electronic device in response to determining the change in the orientation.

[0009] This Summary is provided to introduce simplified concepts relating to smartphones, systems, and methods implemented in electronic devices, which will be further described below in the and drawings. This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it used to determine or limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF DRAWINGS

[0010] Details of one or more aspects of smartphones, systems, and methods implemented in electronic devices are described in this document with reference to the following drawings. In all the Figures, like numbers refer to like features and components:

[0011] Figure 1 An example environment is illustrated in which techniques to enable smartphones, systems, and methods implemented in electronic devices can be implemented.

[0012] Figure 2 An example implementation of a radar system that includes a radar system and can implement smartphones, systems, and methods implemented in electronic devices is illustrated. Figure 1 An example implementation of a smartphone of the

[0013] Figure 3 An example implementation of a radar system of the Figure 1 and Figure 2 is illustrated.

[0014] Figure 4 An example arrangement of receive antenna elements of a radar system of the Figure 3 is illustrated.

[0015] Figure 5 Additional details of an example implementation of a radar system of the Figure 1 and Figure 2 are illustrated.

[0016] Figure 6 Example scenarios that can be implemented by a radar system of the Figure 1 and Figure 2 are illustrated.

[0017] Figure 7FIGURE illustrates another example environment in which techniques enabling smartphones, systems, and methods implemented in electronic devices can be implemented.

[0018] Figures 8-12 FIGURE illustrates an example of presence-based functionality that can be used with techniques for smartphones, systems, and methods implemented in electronic devices as described in Figure 7

[0019] Figure 13 and Figure 14 FIGURE illustrates an example of a modification to presence-based functionality as described in Figures 7-12

[0020] Figures 15-20 FIGURE illustrates an example of 3D gestures that can be used with techniques for smartphones, systems, and methods implemented in electronic devices as described in Figures 7-14

[0021] Figure 21 and 22 depicts an example method of enabling smartphones, systems, and methods implemented in electronic devices.

[0022] Figures 23-25 FIGURE illustrates an example implementation of an electronic device that implements Figure 21 and Figure 22 methods.

[0023] Figure 26 FIGURE illustrates various components of an example computing system that can be implemented as any type of client, server, and / or electronic device as described with reference to Figures 1 to 25 to implement smartphones, systems, and methods implemented in electronic devices or in which techniques enabling smartphones, systems, and methods implemented in electronic devices can be implemented. DETAILED DESCRIPTION

[0024] ​​​This document describes techniques and systems that enable smartphones, systems, and methods implemented in electronic devices. As noted above, because smartphones and other electronic devices are unaware of whether a user is in proximity to the device, or how the user is positioned relative to the device (and the device's display), the device can not always present content in a manner that is convenient for the user. Moreover, and again because of the lack of awareness, the electronic device can disturb the user by displaying reminders or notifications (which can also include personal and private information) at inappropriate or inconvenient times and places, or at an embarrassing volume. It can be frustrating and challenging to properly orient the device to display content (especially for multiple users), or to remember to place the device in "silent" mode. Moreover, users sometimes forget to turn off silent mode, which can result in missing important reminders. Thus, because of the limited recognition of the surrounding environment by the device, the user can not realize the full potential of their smartphone and other devices.

[0025] The described techniques and systems employ a radar system to accurately determine the position and physical orientation of a user relative to an electronic device. The radar field also enables the device to accurately determine three-dimensional (3D) gestures from the user (e.g., including one or more moving gestures in any direction within the 3D space illuminated by the radar field 110), which can be used to interact with the device. Unless otherwise indicated by a particular context, increased accuracy refers to an increased degree of improvement, an increased conformity to truth, or both an increased degree of improvement and an increased conformity to truth. By properly orienting the display and adjusting the timing, content, and format of the device's interaction with the user, these techniques enable the device to provide functionality based on the user's presence and orientation. Thus, the described techniques and systems can improve the quality and effectiveness of the user experience, and thereby increase the user's efficiency, workflow, and entertainment.

[0026] Consider an electronic device that includes a do not disturb (DND) feature that can be activated by orienting the electronic device in a particular way. For example, the DND feature can be activated by orienting the device's display to face the surface on which the device is resting. In this example, placing the electronic device screen down enters a mode in which the device does not provide disturbances, such as calendar reminders, email or text notifications, etc. This type of orientation-based DND mode often prevents the user from even seeing a clock or timer on the screen (e.g., to remind the user when to exit the DND mode). Furthermore, to exit the DND mode, the user must pick up the device, tilt it until the display orientation is correct, and then put the device back down (which can again change the display orientation). When there are multiple viewers, some of the orientations can still be incorrect. Interactions with orientation-based features, such as the DND mode or display orientation, are always inconvenient or frustrating, can reduce efficiency and the quality of the user's experience with the orientation-based features, and can even reduce the likelihood that the user will interact with these features.

[0027] In contrast to these conventional techniques, the systems and techniques described in this document can improve efficiency and usability in a number of areas. For example, in the above example, the user places the electronic device in DND mode, and can have difficulty exiting the DND mode in a simple and convenient manner. In this case, the electronic device can include a radar system that can provide a radar field that extends into an area around the device (e.g., a five, eight, or thirteen foot radius around the device). The radar sensors can use radar signals reflected from objects that enter the radar field to detect the presence and location of the user and the orientation of the device relative to the user. The user can then enter the DND mode by turning the device to a particular orientation (e.g., landscape orientation) with the display facing up. In this case, the user can exit the DND mode using a simple change in orientation, such as rotating the device to a portrait orientation. This easy movement can put the device in the correct orientation relative to the user without requiring the user to have to pick up the device or tilt it, because the radar field allows the device to know the required orientation based on the location of the user.

[0028] In this way, the described techniques and systems allow for simple and convenient interactions with orientation-based features. Users can enjoy the advantages and conveniences of these features without the disruptions and disturbances that can result from attempting to use orientation-based features without the described techniques. This can improve efficiency and reduce the user's frustration, such as having to adjust and re-adjust the orientation of the device to achieve a desired result, thereby increasing the quality of the user's experience.

[0029] This is merely one example of how the techniques and apparatuses described herein can be used to allow users to enjoy directional or presence-based features. Other examples and implementations are described throughout this document. Now, the text turns to an example environment, after which example systems, devices, methods, and components are described.

[0030] Operating Environment

[0031] Figure 1 An example environment 100 is illustrated in which the techniques that enable the smartphones, systems, and methods implemented in electronic devices can be implemented. The example environment 100 includes a smartphone 102 that includes or is associated with a radar system 104 and a radar-based application 106. In some implementations, the smartphone 102 can include a display 108. Some implementations of the radar system 104 are particularly advantageous when applied in the context of smartphones, such as the smartphone 102, that have simultaneous issues such as low power requirements, processing efficiency requirements, spacing and layout limitations of antenna elements, and other issues, and certain embodiments of the radar system 104 are even more advantageous in the context of smartphones that require fine gesture radar detection. Although these embodiments are particularly advantageous in the context of smartphones that require fine gesture radar detection, it should be understood that the applicability of the features and advantages of the present teachings is not necessarily limited thereto, and other embodiments involving other types of electronic devices can also be within the scope of the present teachings.

[0032] In the example environment 100, the radar system 104 provides a radar field 110 by transmitting one or more radar signals or waveforms, as described below with reference to Figures 3 to 6 The radar field 110 is a volume of space from which the radar system 104 can detect reflections of radar signals and waveforms (e.g., radar signals and waveforms reflected from objects in the volume of space). The radar system 104 also enables the smartphone 102 to sense and analyze reflections from objects 112 in the radar field 110. In the examples described with respect to Figure 1 Although the described features and techniques are implemented in the smartphone 102, the described features and techniques can be used with any of a variety of electronic devices (e.g., as described with reference to Figure 2

[0033] The objects 112 can be any of a variety of objects (such as wood, plastic, metal, fabric, or a human body part (e.g., a hand of a user of the smartphone 102)) from which the radar system 104 can sense and analyze reflections. As Figure 1 ​As shown in FIG. 1, the object 112 is a person or a user of the smartphone 102 (e.g., the person 112 or the user 112). Based on the analysis of the reflections, the radar system 104 can provide radar data that includes various types of information associated with the radar field 110 and the reflections from the object 112, as described below with reference to Figures 3-6 As described, the radar system 104 can transmit the radar data to other entities, such as the radar-based application 106.

[0034] It should be noted that the radar data can be provided continuously or periodically over time based on the sensing and analysis of the reflections from the object 112 in the radar field 110. The position of the object 112 can change over time (e.g., the object 112 can move within the radar field 110), and thus, the radar data can change over time corresponding to the changing position, reflections, and analysis. Because the radar data can change over time, the radar system 104 can provide radar data that includes one or more subsets of radar data corresponding to different time periods. For example, the radar system 104 can provide a first subset of radar data corresponding to a first time period, a second subset of radar data corresponding to a second time period, and so on.

[0035] The radar-based application 106 can be any of a variety of radar-based applications that can use the radar data to determine an orientation of the smartphone 102 relative to the object 112 and provide orientation-based features or functions for the smartphone 102 (e.g., via the display 108). Additionally, the radar-based application 106 can use the radar data to determine a change in the orientation of the smartphone 102 relative to the object and modify the orientation-based features or functions based on the change in the orientation. In this way, the orientation of the object 112 within the radar field 110 can be used to provide input or instructions to interact with the smartphone 102.

[0036] The radar-based application 106 can include or be in communication with an orientation module 114 that can store information in internal or external memory related to determining an orientation of the smartphone 102 relative to the object 112 (based on the radar data) and information related to features and functions corresponding to particular orientations of the smartphone 102 with respect to the object 112. In some implementations, the radar-based application 106 can determine the orientation of the smartphone 102 relative to the object 112 using the information stored by the orientation module 114. In other implementations, the orientation module 114 itself can determine the orientation of the smartphone 102 relative to the object 112. As described below, the orientation module 114 can determine the orientation of the smartphone 102 relative to the object 112 based on the radar data. Figure 1 As shown in FIG. 1, the orientation module 114 is included in the radar-based application 106, but the orientation module 114 can be a separate entity that is part of or separate from the radar-based application 106 or the smartphone 102.

[0037] The radar-based application 106 can also include or be in communication with a 3D gesture module 116 that can store both information related to determining a 3D gesture from radar data (e.g., a 3D gesture by the object 112) and information related to an action corresponding to the 3D gesture. Thus, the radar-based application 106 can detect a 3D gesture by the object 112 and determine an action corresponding to the 3D gesture.

[0038] In Figure 1 In this case, the orientation-based function of the smartphone 102 relative to the object 112 (in this case, the user 112) is a do-not-disturb (DND) mode (e.g., a mode in which alerts, notifications, or other device-initiated communications are all muted). Because the display 108 is still visible, non-intrusive content (e.g., a clock, a calendar, or a media player interface) can still be displayed. As shown in the detailed view 100-1, the smartphone 102 is placed in the DND mode by positioning the smartphone 102 in a first orientation (in this case, a landscape orientation) relative to the user 112. As described with reference to Figures 3-6 The radar system 104 can use the radar field 110 to sense and analyze reflections from objects in the radar field 110 in a manner that enables high resolution and high accuracy in identifying device orientation, as described above. For example, because the smartphone 102 knows its own orientation relative to the radar field, the radar data as described above can be used (e.g., by the radar-based application 106) to determine the first orientation of the smartphone 102 relative to the user 112 and provide the DND mode.

[0039] In the detailed view 100-2, the user 112 has rotated the smartphone 102 to a second orientation (in this case, a portrait orientation) relative to the user 112. The rotation of the smartphone 102 is shown by the arrow 118. Again, the radar-based application 106 (or another application) can use the radar data to determine the change from the landscape orientation to the portrait orientation and, in response to the change in orientation, exit the DND mode.

[0040] Note that the user 112 does not necessarily tilt or pick up the smartphone 102. For example, in some implementations, the rotation can be a planar rotation on the surface, a rotation within a plane substantially parallel to a viewing surface of the display 108, or a rotation about an axis substantially perpendicular to a plane containing the smartphone 102, and still effectively cause the smartphone 102 to be in a portrait (second) orientation relative to the user 112. In addition, once out of the DND mode, the radar-based application 106 can display any reminders, notifications, or other device-initiated communications that were muted while the smartphone 102 was in the DND mode. In some implementations, if the rotation exceeds a threshold distance, the radar-based application 106 can determine that the rotation effectively causes a modification of the orientation-based functionality. The threshold distance can be any appropriate distance, such as 25 degrees, 45 degrees, 90 degrees, or 180 degrees, and can be predefined, user-selectable, or determined via a machine learning module included in or associated with the radar system 104 or the radar-based application 106.

[0041] Consider three additional examples (not illustrated) that illustrate how the described techniques can enable orientation-based functionality. In a first example, the orientation-based functionality of the smartphone 102 is to present content on the display 108 in a user-facing orientation. In this way, the smartphone 102 can provide videos, photos, games, and other media that automatically face the user 112 via the radar-based application 106 or another application, without the user 112 having to pick up the smartphone 102. The user 112 can change the orientation of the smartphone 102, and the smartphone 102 can determine via the radar-based application 106 or another application that the orientation has changed to another orientation, and automatically maintain the user-facing orientation of the content when the smartphone 102 is in the other orientation relative to the user 112.

[0042] For example, the user 112 can change the orientation of the smartphone 102 by moving to another location or by rotating the smartphone 102. As described above, the user does not have to tilt or pick up the device. Rather, the change in orientation of the smartphone 102 relative to the user 112 can be a planar rotation on the surface, a rotation within a plane substantially parallel to a viewing surface of the display 108, or a rotation of the smartphone 112 about an axis substantially perpendicular to a plane containing the smartphone 102, and still effectively cause the smartphone 102 to be in another orientation relative to the user 112.

[0043] In a second example, there are two users 112. For example, a first user (Sam) is the owner of the smartphone 102, and a second user (Chris) is a co-worker of Sam. In this example, assume that Sam has a presentation that Chris would like to review. The orientation-based functionality of the smartphone 102 would present the content on the display 108 in an orientation facing Sam (e.g., a first user-facing orientation, similar to the orientation-based functionality described in the previous example). Further assume that Chris and Sam are sitting on opposite sides of a table, and Sam agrees to let Chris review the presentation. Sam can let Chris view the presentation by moving the smartphone 102 toward Chris. As described above, changes in the orientation of the smartphone 102 can be determined using radar data (e.g., by the radar-based application 106). In this case, the change is that the smartphone 102 is farther away from Sam and closer to Chris (e.g., the change in the orientation of the smartphone 102 relative to Sam is a displacement of the smartphone 102 that results in the smartphone 102 being farther away from Sam and closer to Chris).

[0044] Upon determining the change in orientation, the smartphone 102, via the radar-based application 106 or another application, can automatically modify the orientation-based functionality to present the content on the display in an orientation facing Chris (e.g., a second user-facing orientation). Thus, Sam can share the presentation with Chris without having to rotate or flip the smartphone 102. More specifically, as Sam moves the device toward Chris, the displayed content can automatically reorient toward Chris. In this way, the smartphone 102 can provide videos, photos, games, and other media that are automatically oriented in the appropriate direction without the user 112 having to manually adjust the orientation.

[0045] In a third example, the radar-based application 106 or another application can use an attention cue to determine the orientation of the smartphone 102 relative to the user 112. As described with reference to Figures 3-6 The radar system 104 can use the radar field 110 to sense and analyze reflections from objects in the radar field 110 in a way that enables the number and posture of people in the radar field 110 to be identified with high resolution and accuracy, for example, by using angular resolution and digital beamforming. In this way, the smartphone 102 can detect an attention cue from the user 112. An attention cue is a body posture or position of the user 112, such as leaning forward or backward, or an orientation of the torso or head of the user 112 (e.g., toward or away from the smartphone 102). The attention cue can be used to determine whether the user 112 is paying attention while using the smartphone 102 to read a document, listen to music, view photos, etc.

[0046] As described above, attention cues can be determined using radar data. For example, radar data can be used to determine attention cues by determining one or more of: an angle of the torso or head of the user 112 relative to a plane substantially parallel to a viewing surface of the display 108, such as a table on which the smartphone 102 is resting (e.g., whether the user 112 is leaning toward the smartphone 102), or an angular position of the torso or head of the user 112 (e.g., whether the user 112 has turned away from the smartphone 102), or the presence of the user 112 within a threshold distance of the smartphone 102.

[0047] In this example, the orientation-based functionality of the smartphone 102 is to present content on the display 108 in an attention-cue-based content display mode (e.g., a first content display mode). For example, the content display mode can be a media pause mode or a media play mode, a lock screen mode, a sleep or wake mode, a full screen or slideshow mode, or include a particular screen brightness level or volume level. The smartphone 102, using the radar-based application 106 or another application, can determine a change in the attention cue (e.g., a change in the orientation between the user 112 and the smartphone 102), such as a change from leaning forward to leaning backward or a turn of the head of the user 112 from looking at the display 108 to not looking at the display.

[0048] In response to determining the change in the attention cue, the radar-based application 106 (or another application) can present the content in another content display mode (e.g., modify the orientation-based functionality of the smartphone 102 to present the content in a second content display mode). Like the first content display mode, the other content display mode can be a media pause mode or a media play mode, a lock screen mode, a sleep or wake mode, a full screen or slideshow mode, or include a particular screen brightness level or volume level. Continuing the example above, assume that the user 112 is watching a video in full screen mode and then leans forward toward the smartphone 102. In response, the radar-based application 106 can determine that the user 112 is going to interact with the video player and exit the full screen mode.

[0049] In this way, the smartphone 102 can use attention cues to help determine the context in which the user 112 is using the smartphone 102, thereby providing an improved user experience. For example, if the smartphone 102 determines that the user 112 is not paying attention to a video being played, such as by determining that the user 112 has turned away from the display 108 (or has left the room), the smartphone 102 can pause the video and dim the screen brightness so that the user does not miss any content and conserves battery power.

[0050] In some implementations, including implementations of the above examples, the radar-based application 106 can determine that a rotation (e.g., a rotation of the smartphone 102 or a torso or head of the user 112) effectively causes a modification of the directional-based function if the rotation exceeds a threshold. The threshold can be any suitable rotation, such as (25 degrees, 45 degrees, 90 degrees, or 180 degrees). Similarly, a displacement can effectively cause a modification of the directional-based function if the displacement effectively causes the smartphone 102 to become farther from Sam and closer to Chris by at least a threshold distance. The threshold distance can be any suitable distance, such as six, twelve, or eighteen inches. Further, an angle (e.g., an angle at which the user 112 leans forward or backward) can effectively cause a modification of the directional-based function if the angle exceeds a threshold angle. The threshold angle can be any suitable angle, such as 15 degrees, 25 degrees, 35 degrees, or 45 degrees. The threshold rotation, distance, or angle can be predefined, user-selectable, or determined via a machine learning module included in or associated with the radar system 104 or the radar-based application 106.

[0051] Additionally, the radar-based application 106 can determine a directional change that effectively causes a modification of the directional-based function if the directional change is maintained for at least a threshold time. The threshold time can be any suitable time (e.g., 0.5, 1.5, or 2.5 seconds) and can be predefined, user-selectable, or determined via a machine learning module included in or associated with the radar system 104 or the radar-based application 106.

[0052] In more detail, consider Figure 2 which illustrates an example implementation 200 of the smartphone 102 (including the radar system 104, the radar-based application 106, and optionally the display 108), which can be implemented in a smartphone, system, and method implemented in an electronic device. Figure 2 The smartphone 102 of FIG. 1 is illustrated as including other non-limiting example devices that can be implemented in a smartphone, system, and method implemented in an electronic device, including a mobile phone 102-1, a tablet 102-2, a laptop computer 102-3, a desktop computer 102-4, a computing watch 102-5, computing eyewear 102-6, a gaming system 102-7, a home automation control system 102-8, and a vehicle 102-9. Other devices can also include a television, an entertainment system, an audio system, a drone, a trackpad, a drawing pad, a netbook, an e-reader, a home security system, and other home appliances. Note that electronic devices in which the described techniques can be implemented can be wearable, non-wearable but mobile, or relatively immobile (e.g., desktops and appliances).

[0053] An exemplary overall lateral dimension of the smartphone 102 can be, for example, approximately eight centimeters by approximately fifteen centimeters. An exemplary footprint of the radar system 104 can be further limited, such as approximately four millimeters by six millimeters including the antenna. An exemplary power consumption of the radar system 104 can be on the order of a few milliwatts (mW) to several mW (e.g., between approximately 2 mW and 20 mW). The requirement of such limited footprint of the radar system 104 needed to accommodate the many other desirable features of the smartphone 102 (e.g., camera, fingerprint sensor, display 108, etc.) in such a space-limited package in combination with power and processing limitations can result in a tradeoff in accuracy and effectiveness of radar gesture detection, at least some of which can be overcome based on the teachings herein.

[0054] The smartphone 102 also includes one or more computer processors 202 and one or more computer-readable media 204, including memory media and storage media. Applications and / or an operating system (not shown) implemented as computer-readable instructions on the computer-readable media 204 can be executed by the computer processors 202 to provide some of the functionality described herein. The smartphone 102 can also include a network interface 206. The smartphone 102 can use the network interface 206 to communicate data over a wired, wireless, or optical network. By way of example and not limitation, the network interface 206 can communicate data over a local area network (LAN), a wireless local area network (WLAN), a personal area network (PAN), a wide area network (WAN), an intranet, the Internet, a peer-to-peer network, a point-to-point network, or a mesh network.

[0055] Various implementations of the radar system 104 can include a system on a chip (SoC), one or more integrated circuits (ICs), a processor with embedded processor instructions or configured to access processor instructions stored in memory, hardware with embedded firmware, a printed circuit board with various hardware components, or any combination thereof. The radar system 104 operates as a monostatic radar by transmitting and receiving its own radar signals. In some implementations, the radar system 104 can also cooperate with other radar systems 104 within an external environment to implement a bistatic radar, a multistatic radar, or a networked radar. As noted, constraints or limitations of the smartphone 102 can impact the design of the radar system 104. For example, the smartphone 102 can have limited power available to operate the radar, limited computing power, size constraints, layout restrictions, an external housing that attenuates or distorts radar signals, and so on. The radar system 104 includes several features that enable advanced radar functionality and high performance in the presence of these constraints, as described further below with respect to Figure 3 Further described. Note that, in Figure 2In particular embodiments, the radar system 104 is illustrated as part of the smartphone 102. In other embodiments, the radar system 104 can be separate from or remote from the smartphone 102.

[0056] These and other capabilities and configurations, as well as Figure 1 The manner in which the entities act and interact will be set out in more detail below. These entities can be further divided, merged, etc. Figure 1 The environment 100 and Figures 2 to 25 The detailed illustrations of the environment 100 and Figures 3 to 6 Additional details and features of the radar system 104 are described. In Figures 3 to 6 In particular embodiments, the radar system 104 is described in the context of a smartphone 102, but as noted above, the applicability of the features and advantages of the described systems and techniques is not necessarily limited thereto, and other embodiments involving other types of electronic devices can also be within the scope of the present teachings.

[0057] Figure 3 An example embodiment 300 of the radar system 104 is illustrated, which can be used to enable a smartphone, system, and method in an electronic device. In the example 300, the radar system 104 includes at least one of each of the following components: a communication interface 302, an antenna array 304, a transceiver 306, a processor 308, and a system medium 310 (e.g., one or more computer-readable storage media). The processor 308 can be implemented as a digital signal processor, a controller, an application processor, another processor (e.g., the computer processor 202 of the smartphone 102), or some combination thereof. The system medium 310, which can be included in the computer-readable medium 204 of the smartphone 102 or separate therefrom, includes one or more of the following modules: an attenuation mitigator 314, a digital beamformer 316, an angle estimator 318, or a power manager 320. These modules can compensate for or mitigate the effects of integrating the radar system 104 into the smartphone 102, thereby enabling the radar system 104 to recognize small or complex gestures, distinguish different orientations of a user, continuously monitor an external environment, or achieve a target false alarm rate. With these features, the radar system 104 can be implemented in a variety of different devices such as Figure 2 The radar system 104 is illustrated as part of the smartphone 102. In other embodiments, the radar system 104 can be separate from or remote from the smartphone 102.

[0058] Using communication interface 302, radar system 104 can provide radar data to radar-based application 106. Radar system 104 can be implemented separately from or integrated into smartphone 102, and communication interface 302 can be a wireless or wired interface. Depending on the application, radar data may include raw or minimally processed data, in-phase and quadrature (I / Q) data, range-Doppler data, processed data including target location information (e.g., range, azimuth, elevation), cluttered map data, etc. Typically, radar data contains information that radar-based application 106 can use to implement in electronic devices such as smartphones, systems, and methods.

[0059] Antenna array 304 includes at least one transmitting antenna element (not shown) and at least two receiving antenna elements (such as...). Figure 4 (As shown). In some cases, antenna array 304 may include multiple transmit antenna elements to enable a multiple-input multiple-output (MIMO) radar capable of transmitting multiple different waveforms simultaneously (e.g., a different waveform for each transmit antenna element). The use of multiple waveforms can improve the measurement accuracy of radar system 104. In embodiments including three or more receive antenna elements, the receive antenna elements may be positioned in a one-dimensional shape (e.g., a line) or a two-dimensional shape. A one-dimensional shape allows radar system 104 to measure one angular dimension (e.g., azimuth or elevation), while a two-dimensional shape allows for the measurement of two angular dimensions (e.g., both azimuth and elevation). Reference Figure 4 A sample two-dimensional arrangement of the receiving antenna elements is further described.

[0060] Figure 4 The illustration shows an example arrangement 400 of the receiving antenna elements 402. For example, if the antenna array 304 includes at least four receiving antenna elements 402, the receiving antenna elements 402 can be arranged in a rectangular arrangement 404-1, as shown. Figure 4 The middle part is depicted. Alternatively, if the antenna array 304 includes at least three receiving antenna elements 402, a triangular arrangement 404-2 or an L-shaped arrangement 404-3 can be used.

[0061] Due to size or layout constraints of the smartphone 102, the inter-element spacing between the receive antenna elements 402 or the number of receive antenna elements 402 can not be ideal for the angles that the radar system 104 is to monitor. In particular, the inter-element spacing can result in there being angle ambiguities, which present challenges for conventional radars to estimate the angular position of a target. Accordingly, conventional radars can limit the field of view (e.g., the angles to be monitored) to avoid ambiguous regions with angle ambiguities, thereby reducing false detections. For example, a conventional radar can limit the field of view to angles between approximately -45 degrees and 45 degrees to avoid angle ambiguities that occur using a 5 millimeter (mm) wavelength and a 3.5 mm inter-element spacing (e.g., the inter-element spacing is 70% of the wavelength). Accordingly, the conventional radar can not be able to detect targets beyond the 45 degree field of view limit. In contrast, the radar system 104 includes the digital beamformer 316 and the angle estimator 318, which resolve angle ambiguities and enable the radar system 104 to monitor angles beyond the 45 degree limit, such as angles between approximately -90 degrees and 90 degrees, or up to approximately -180 degrees and 180 degrees. These ranges of angles can be applied in one or more directions (e.g., azimuth and / or elevation). Accordingly, the radar system 104 can achieve low false alarm rates for a variety of different antenna array designs, including inter-element spacings that are less than, greater than, or equal to half of the center wavelength of the radar signals.

[0062] Using the antenna array 304, the radar system 104 can form beams that are steered or unsteered, wide or narrow, or shaped (e.g., as a hemisphere, cube, sector, cone, or cylinder). As an example, one or more transmit antenna elements (not shown) can have an unsteered, omnidirectional radiation pattern, or can be capable of producing a wide beam such as the wide transmit beam 406. Either of these techniques enables the radar system 104 to illuminate a large volume of space. However, to obtain target angular precision and angular resolution, the receive antenna elements 402 and the digital beamformer 316 can be used to generate thousands of narrow and steered beams (e.g., 2000 beams, 4000 beams, or 6000 beams), such as the narrow receive beams 408. In this way, the radar system 104 can effectively monitor the external environment and accurately determine the angles of arrival of reflections within the external environment.

[0063] Returning to Figure 3The transceiver 306 includes circuitry and logic to transmit and receive radar signals via the antenna array 304. The components of the transceiver 306 can include amplifiers, mixers, switches, analog-to-digital converters, filters, and the like to condition the radar signals. The transceiver 306 can also include logic to perform in-phase / quadrature (I / Q) operations, such as modulation or demodulation. The transceiver 306 can be configured for continuous wave radar operation or pulsed radar operation. Various modulations can be used to produce the radar signals, including linear frequency modulation, triangular frequency modulation, stepped frequency modulation, or phase modulation.

[0064] The transceiver 306 can generate radar signals within a frequency range (e.g., a spectrum) such as between 1 gigahertz (GHz) and 400 GHz, between 4 GHz and 100 GHz, or between 57 GHz and 63 GHz. The spectrum can be divided into multiple frequency sub-spectra with similar bandwidths or different bandwidths. The bandwidths can be on the order of 500 megahertz (MHz), 1 GHz, 2 GHz, and the like. As an example, different frequency sub-spectra can include frequencies between approximately 57 GHz and 59 GHz, 59 GHz and 61 GHz, or 61 GHz and 63 GHz. Multiple frequency sub-spectra with the same bandwidth and that can be contiguous or non-contiguous can also be selected for coherence. The multiple frequency sub-spectra can be transmitted simultaneously or separated in time using a single radar signal or multiple radar signals. Contiguous frequency sub-spectra enable radar signals with wider bandwidths, while non-contiguous frequency sub-spectra can further emphasize amplitude and phase differences, which enable the angle estimator 318 to resolve angle ambiguities. The attenuation mitigator 314 or the angle estimator 318 can cause the transceiver 306 to utilize one or more frequency sub-spectra to improve the performance of the radar system 104, as described further with respect to FIGS. 3A-3C. Figure 5 and Figure 6 are further described.

[0065] The power manager 320 enables the radar system 104 to conserve power within the smartphone 102 or externally. For example, internally, the power manager 320 can cause the radar system 104 to collect data using a predetermined power mode or a particular duty cycle. Instead of operating in a low power mode or a high power mode, the power manager 320 dynamically switches between different power modes such that response latency and power consumption are managed together based on activity within the environment. Generally, the power manager 320 determines when and how to conserve power and incrementally adjusts power consumption to enable the radar system 104 to work within the power limitations of the smartphone 102. In certain instances, the power manager 320 can monitor the amount of power remaining available and adjust the operation of the radar system 104 accordingly. For example, if the amount of power remaining is low, the power manager 320 can continue to operate in a low power mode instead of switching to a higher power mode.

[0066] For example, the low power mode can use a low duty cycle on the order of a few hertz (e.g., approximately 1 Hz or less than 5 Hz), which reduces power consumption to a few milliwatts (mW) (e.g., between approximately 2 mW and 5 mW). On the other hand, the high power mode can use a high duty cycle on the order of tens of hertz (Hz) (e.g., approximately 20 Hz or greater than 10 Hz), which results in the radar system 104 consuming power on the order of a few milliwatts (e.g., between approximately 8 mW and 20 mW). While the low power mode can be used to monitor the external environment or detect an approaching user, the power manager 320 can switch to the high power mode if the radar system 104 determines that the user is beginning to perform a gesture. Different triggers can cause the power manager 320 to switch between the different power modes. Exemplary triggers include motion or lack of motion, the presence or absence of a user (e.g., the presence or absence of a user), the user moving into or out of a specified region (e.g., a region defined by a distance, azimuth, or elevation angle), the distance of a user from the smartphone 102 such as the perceived distance 706 described above, a change in velocity of motion associated with a user, or a change in reflected signal strength (e.g., due to a change in radar cross section). Generally, triggers that indicate a low likelihood of a user interacting with the smartphone 102 or a preference for using longer response delays to collect data can result in activating a lower power mode to conserve power.

[0067] The power manager 320 can also conserve power by turning off one or more components within the transceiver 306 (e.g., a voltage controlled oscillator, a multiplexer, an analog-to-digital converter, a phase-locked loop, or a crystal oscillator) during periods of inactivity. These periods of inactivity can occur on the order of microseconds (ps), milliseconds (ms), or seconds (s) if the radar system 104 is not actively transmitting or receiving radar signals. For example, periods of inactivity can occur when there is no user present or when there is a user present but the smartphone 102 is determined to be unattended (e.g., based on one or more attentiveness cues described in this specification). Additionally, the power manager 320 can control the use of different hardware components within the radar system 104 to conserve power. For example, if the processor 308 includes a low power processor and a high power processor (e.g., processors with different amounts of memory and computing capabilities), the power manager 320 can switch between using the low power processor for low level analysis (e.g., detecting motion, determining the location of a user, or monitoring the environment) and using the high power processor for cases where the radar-based application 106 requests high fidelity or accurate radar data (e.g., for gesture recognition or user orientation).

[0068] In addition to the internal power saving techniques described above, the power manager 320 can also save power by activating or deactivating other external components or sensors within the smartphone 102. These external components can include a speaker, a camera sensor, a global positioning system, a wireless communication transceiver, a display, a gyroscope, or an accelerometer. Because the radar system 104 can use a small amount of power to monitor the environment, the power manager 320 can appropriately turn on or off these external components based on where the user is located or what the user is doing. In this way, the smartphone 102 can seamlessly respond to the user and save power without using an automatic shut-off timer or requiring the user to physically touch or verbally control the smartphone 102.

[0069] Figure 5 Additional details of an example implementation 500 of the radar system 104 within the smartphone 102 are illustrated. In the example 500, the antenna array 304 is located underneath an external housing (such as a glass cover or case) of the smartphone 102. Depending on its material properties, the external housing can act as an attenuator 502 that attenuates or distorts the radar signals sent and received by the radar system 104. The attenuator 502 can include different types of glass or plastic, some of which can be found within the display screen, external housing, or other components of the smartphone 102 and have a dielectric constant (e.g., relative dielectric constant) between approximately four to ten. Thus, the attenuator 502 is opaque or semi-transparent to the radar signals 506 and can cause a portion of the transmitted or received radar signals 506 to be reflected (as shown by the reflected portion 504). For conventional radar, the attenuator 502 can reduce the effective range that can be monitored, prevent small targets from being detected, or reduce the overall accuracy.

[0070] Assuming the transmit power of the radar system 104 is limited and the external housing is not to be redesigned, one or more attenuation-related properties of the radar signals 506 (e.g., the frequency sub-spectrum 508 or the steering angle 510) or attenuation-related properties of the attenuator 502 (e.g., the distance 512 between the attenuator 502 and the radar system 104 or the thickness 514 of the attenuator 502) are adjusted to mitigate the effects of the attenuator 502. Some of these properties can be set during manufacturing or adjusted by the attenuation mitigator 314 during operation of the radar system 104. For example, the attenuation mitigator 314 can cause the transceiver 306 to transmit the radar signals 506 using a selected frequency sub-spectrum 508 or steering angle 510, cause the platform to move the radar system 104 closer to or farther away from the attenuator 502 to change the distance 512, or prompt the user to apply another attenuator to increase the thickness 514 of the attenuator 502.

[0071] Appropriate adjustments can be made by the attenuation mitigator 314 based on predetermined characteristics of the attenuator 502 (e.g., characteristics stored in the computer-readable medium 204 of the smartphone 102 or within the system medium 310) or by processing returns of the radar signal 506 to measure one or more characteristics of the radar signal 506. Even if certain attenuation-related characteristics are fixed or constrained, the attenuation mitigator 314 can take these limitations into account to balance each parameter and achieve the target radar performance. As a result, the attenuation mitigator 314 enables the radar system 104 to achieve enhanced precision and greater effective range to detect and track users located on the opposite side of the attenuator 502. These techniques provide an alternative to increasing the transmit power (which increases the transmit power of the radar system 104) or changing the material characteristics of the attenuator 502 (which can be both difficult and expensive once the device is in production).

[0072] Figure 6 An example scheme 600 implemented by the radar system 104 is illustrated. Portions of the scheme 600 can be performed by the processor 308, the computer processor 202, or other hardware circuitry. The scheme 600 can be customized to support different types of smartphones 102 and radar-based applications 106, and also enables the radar system 104 to achieve the target angular precision despite design limitations.

[0073] The transceiver 306 produces raw data 602 based on individual responses of the receive antenna elements 402 to received radar signals. The received radar signals can be associated with one or more frequency sub-spectra 604 selected by the angle estimator 318 to facilitate angular ambiguity resolution. For example, the frequency sub-spectra 604 can be selected to reduce the number of side lobes or reduce the amplitude of the side lobes (e.g., reduce the amplitude by 0.5 dB, 1 dB, or more). The number of frequency sub-spectra can be determined based on the target angular precision or computational limitations of the radar system 104.

[0074] The raw data 602 contains numerical information (e.g., in-phase and quadrature data) for a time period, different wave numbers, and a plurality of channels respectively associated with the receive antenna elements 402. A fast Fourier transform (FFT) 606 is performed on the raw data 602 to generate pre-processed data 608. The pre-processed data 608 includes numerical information across the time period for different ranges (e.g., a range grid) and for the plurality of channels. A Doppler filtering process 610 is performed on the pre-processed data 608 to generate range-Doppler data 612. The Doppler filtering process 610 can include another FFT that generates amplitude and phase information for a plurality of range grids, a plurality of Doppler frequencies, and for the plurality of channels. A digital beamformer 316 produces beamformed data 614 based on the range-Doppler data 612. The beamformed data 614 contains numerical information for a set of azimuth and / or elevation angles that represent fields of view for which different steering angles or beams are formed by the digital beamformer 316. Although not depicted, the digital beamformer 316 can alternatively generate the beamformed data 614 based on the pre-processed data 608, and the Doppler filtering process 610 can generate the range-Doppler data 612 based on the beamformed data 614. To reduce computational load, the digital beamformer 316 can process a portion of the range-Doppler data 612 or the pre-processed data 608 based on a range, time, or Doppler frequency interval of interest.

[0075] The digital beamformer 316 can be implemented using a single look beamformer 616, a multi-look interferometer 618, or a multi-look beamformer 620. Generally, the single look beamformer 616 can be used for deterministic objects (e.g., point source targets with a single phase center). For non-deterministic targets (e.g., targets with multiple phase centers), the multi-look interferometer 618 or the multi-look beamformer 620 is used to improve accuracy over the single look beamformer 616. A human is an example of a non-deterministic target and has multiple phase centers 622 that can vary based on different line of sight angles (as shown by 624-1 and 624-2) (e.g., due to changes in the user’s orientation relative to the smartphone 102, such as turning down or away, leaning forward, or moving closer or further away). Variations in constructive or destructive interference generated by the multiple phase centers 622 can present challenges for conventional radar to accurately determine angular position. However, the multi-look interferometer 618 or the multi-look beamformer 620 performs coherent averaging to increase the accuracy of the beamformed data 614. The multi-look interferometer 618 coherently averages two channels to generate phase information that can be used to accurately determine angular information. On the other hand, the multi-look beamformer 620 can use linear or non-linear beamformers (such as Fourier, Capon, multiple signal classification (MUSIC), or minimum variance distortionless response (MVDR)) to coherently average two or more channels. The increased accuracy provided via the multi-look beamformer 620 or the multi-look interferometer 618 enables the radar system 104 to recognize small gestures or distinguish between multiple parts of a user.

[0076] The angle estimator 318 analyzes the beamformed data 614 to estimate one or more angular positions. The angle estimator 318 can utilize signal processing techniques, pattern matching techniques, or machine learning. The angle estimator 318 also resolves angular ambiguities that can result from the design of the radar system 104 or the field of view that the radar system 104 monitors. An example angular ambiguity is shown within an amplitude plot 626 (e.g., an amplitude response).

[0077] The amplitude plot 626 depicts the amplitude difference that can occur for different angular positions of a target and for different steering angles 510. A first amplitude response 628-1 is shown (with a solid line) for a target located at a first angular position 630-1. Likewise, a second amplitude response 628-2 is shown (with a dashed line) for a target located at a second angular position 630-2. In this example, the difference is considered over angles between -180 degrees and 180 degrees.

[0078] As shown in the amplitude plot 626, for both angular positions 630-1 and 630-2, there is ambiguity. The first amplitude response 628-1 has a highest peak at the first angular position 630-1 and a smaller peak at the second angular position 630-2. Although the highest peak corresponds to the actual position of the target, the smaller peak would cause ambiguity for the first angular position 630-1 because it is within some threshold that the conventional radar can not confidently determine whether the target is at the first angular position 630-1 or the second angular position 630-2. In contrast, the second amplitude response 628-2 has a smaller peak at the second angular position 630-2 and a higher peak at the first angular position 630-1. In this case, the smaller peak corresponds to the position of the target.

[0079] Although conventional radars can be limited to using the highest peak amplitude to determine the angular position, the angle estimator 318 instead analyzes subtle differences in the shape of the amplitude responses 628-1 and 628-2. Characteristics of the shape can include, for example, roll-off, peak or null width, angular position of the peak or null, height or depth of the peak and null, sidelobe shape, symmetry in the amplitude response 628-1 or 628-2, or lack of symmetry within the amplitude response 628-1 or 628-2. Similar shape characteristics can be analyzed in the phase response, which can provide additional information to resolve angular ambiguity. Thus, the angle estimator 318 maps a unique angular signature or directional pattern to the angular position.

[0080] The angle estimator 318 can include an algorithm or suite of tools that can be selected according to the type of smartphone 102 (e.g., computing power or power limitations) or based on the target angular resolution of the radar-based application 106. In certain implementations, the angle estimator 318 can include a neural network 632, a convolutional neural network (CNN) 634, or a long short-term memory (LSTM) network 636. The neural network 632 can have various depths or numbers of hidden layers (e.g., three hidden layers, five hidden layers, or ten hidden layers) and can also include different numbers of connections (e.g., the neural network 632 can include a fully connected neural network or a partially connected neural network). In certain cases, the CNN 634 can be used to increase the computational speed of the angle estimator 318. The LSTM network 636 can be used to enable the angle estimator 318 to track the target. Using machine learning techniques, the angle estimator 318 employs a non-linear function to analyze the shape of the amplitude response 628-1 or 628-2 and generate angular probability data 638 that indicates the likelihood that the user or a portion of the user is within an angular grid. The angle estimator 318 can provide angular probability data 638 for some angular grid, such as two angular grids, to provide a probability that the target is on the left or right side of the smartphone 102, or angular probability data 638 for thousands of angular grids (e.g., to provide angular probability data 638 for continuous angular measurements).

[0081] Based on the angular probability data 638, the tracker module 640 produces angular position data 642, which identifies the angular position of the target. The tracker module 640 can determine the angular position of the target based on the angular grid with the highest probability in the angular probability data 638 or based on prediction information (e.g., previously measured angular position information). The tracker module 640 can also track one or more moving targets to enable the radar system 104 to confidently distinguish or identify targets. Other data can also be used to determine the angular position, including range, Doppler, velocity, or acceleration. In some cases, the tracker module 640 can include an alpha-beta tracker, a Kalman filter, a multiple hypothesis tracker (MHT), or the like.

[0082] The quantizer module 644 takes the angular position data 642 and quantizes the data to produce quantized angular position data 646. The quantization can be performed based on the target angular resolution for the radar-based application 106. In some cases, a small number of quantization levels can be used, such that the quantized angular position data 646 indicates whether the target is to the right or left of the smartphone 102 or identifies that the target is within a 90-degree quadrant. For some radar-based applications 106, such as user proximity detection, this can be sufficient. In other cases, a large number of quantization levels can be used, such that the quantized angular position data 646 indicates the angular position of the target to within a fraction of a degree, a degree, five degrees, or the like. This resolution can be used for higher resolution radar-based applications 106, such as gesture recognition. In some implementations, the digital beamformer 316, the angle estimator 318, the tracker module 640, and the quantizer module 644 are implemented together in a single machine learning module.

[0083] These and other capabilities and configurations, as well as Figures 1 to 6 The manner in which the described entities interact and operate is set forth below. The described entities can be further divided, combined, used with other sensors or components, and the like. In this manner, different implementations of the smartphone 102 with different configurations of the radar system 104 and non-radar sensors can be used to implement the described smartphones, systems, and methods implemented in electronic devices. Figure 1 Figures 2 to 6 The detailed illustrations of the example operating environment 100 and Figures 2 to 6 The detailed illustrations of the example operating environment 100 and

[0084] Example system

[0085] As described above, the techniques and systems described herein can also enable the smartphone 102 to provide functionality based on the presence of a user and to receive 3D gestures from the user to interact with the device. Figure 7FIG. 13 illustrates another example environment 1300 in which the techniques enabling a smartphone, system, and method implemented in an electronic device can be implemented. The example operating environment 1300 includes a smartphone 102 that includes a radar system 104 and a radar-based application 702 or is associated with the same. In some implementations, the smartphone 102 can include a display, such as the display 108. Although the example operating environment 1300 is illustrated in the context of the smartphone 102, other electronic devices (as described above with reference to Figure 1 and 2 ) can also be used to implement the features and techniques described with reference to Figures 7 to 20 In the example environment 1300, the radar system 104 provides a radar field 110, as described above with reference to Figures 1 to 6 The radar system 104 also enables the smartphone 102 to sense and analyze reflections from an object 704 in the radar field 110. In the example described with reference to Figures 7 to 20 The smartphone 102 is a smartphone 102, although the features and techniques described can be used with any of a variety of electronic devices (e.g., as described with reference to Figure 2

[0086] As shown, the object 704 is a person or user (person 704 or user 704) of the smartphone 102, although the object 704 can be any of a variety of objects (such as wood, plastic, metal, fabric, or organic material (e.g., the user 704)) that the radar system 104 can sense and analyze reflections from. Based on the analysis of the reflections, the radar system 104 can provide radar data, as described above with reference to Figure 7 and can pass the radar data to other entities, such as the radar-based application 702. Figures 1 to 6 The radar-based application 702 can be any of a variety of radar-based applications that can determine the presence of the user 704 within a perceived distance 706 of the smartphone 102 using the radar data. The perceived distance can be any appropriate distance, such as three feet, seven feet, ten feet, or fourteen feet (or one meter, two meters, three meters, or four meters) and is consistent with the range of the radar field 110, as shown in the detailed view 1300-1 of

[0087] In other cases, the perceived distance 706 can be less than the maximum range of the radar field 110. The perceived distance 706 can be predefined, user-selectable, or determined by a machine learning module that is included in or associated with the radar system 104 or the radar-based application 702. Figure 7

[0088] ​​Based on the presence of the user 704 within the perceived distance 706 of the smartphone 102, the radar-based application 702 can provide presence-based features or functions for the smartphone 102 (e.g., via the display 108). For example, as shown in the detailed view 700-1, the user 704 has moved toward the smartphone 102 (indicated by the arrow 706) and is within the perceived distance 706. In response, the smartphone 102 provides presence-based functions, such as an electronic assistant that includes a voice interface (shown as a text box ("Hello") 710). Additionally, the radar-based application 702 can determine, using radar data, that the user 704 is outside of the perceived distance 706 and modify or cease providing presence-based features or functions based on the change in the presence of the user 704 within the perceived distance 706. In this way, the presence of the user 704 within the perceived distance 706 can be used to provide input or instructions to interact with the smartphone 102. Figure 7 As shown in the detailed view 700-2, the user 704 has moved toward the smartphone 12 (indicated by the arrow 710) and is within the perceived distance 706. In response, the smartphone 102 provides presence-based functions, such as an electronic assistant that includes a voice interface (shown as a text box ("Hello") 712). Additionally, the radar-based application 702 can determine, using radar data, that the user 704 is outside of the perceived distance 706 and modify or cease providing presence-based features or functions based on the change in the presence of the user 704 within the perceived distance 706. In this way, the presence of the user 704 within the perceived distance 706 can be used to provide input or instructions to interact with the smartphone 102.

[0089] The radar-based application 702 can include or be in communication with a presence module 708 that can store both information related to the presence of the user 704 within the perceived distance 706 (based on radar data) and information related to features and functions based on the presence of the user 704 within the perceived distance 706 in internal or external memory. In some implementations, the radar-based application 702 can determine the presence or absence of the user 704 within the perceived distance 706 using information stored by the presence module 708. In other implementations, the presence module 708 itself can determine the presence or absence of the user 704 within the perceived distance 706. As shown, the presence module 708 is included in the radar-based application 702, but the presence module 708 can be a separate entity that is part of or separate from the radar-based application 702 or the smartphone 102. Figure 7 As shown, the presence module 708 is included in the radar-based application 702, but the presence module 708 can be a separate entity that is part of or separate from the radar-based application 702 or the smartphone 102. In some implementations of either of the examples 100 or 700, the presence module 708 and the directional module 114 (of the radar-based application 106 or 702) can be a single module that can perform the operations described with reference to both the presence module 708 and the directional module 114 or in combination with either of the radar-based applications 106 or 702. Figure 1 As shown, the presence module 708 is included in the radar-based application 702, but the presence module 708 can be a separate entity that is part of or separate from the radar-based application 702 or the smartphone 102. In some implementations of either of the examples 100 or 700, the presence module 708 and the directional module 114 (of the radar-based application 106 or 702) can be a single module that can perform the operations described with reference to both the presence module 708 and the directional module 114 or in combination with either of the radar-based applications 106 or 702.

[0090] The radar-based application 106 can also include or be in communication with a 3D gesture module (e.g., 3D gesture module 116) that can store both information related to determining 3D gestures (e.g., 3D gestures of user 704) based on radar data and information related to actions corresponding to the 3D gestures. Thus, the radar-based application 702 can detect 3D gestures of user 704 and determine actions corresponding to the 3D gestures. The 3D gestures can be any of a variety of gestures, including a scroll gesture made by a hand moving in a horizontal direction (e.g., from a left side of smartphone 102 to a right side of smartphone 102) over smartphone 102, a wave gesture made by a user’s arm rotating about an elbow, a push gesture made by a user’s hand moving in a vertical direction (e.g., from a bottom side of smartphone 102 to a top side of smartphone 102) over smartphone 102. Other types of 3D gestures or actions can also be made, such as a reach gesture made by a user’s hand moving toward smartphone 102, a doorknob turn gesture made by a user’s hand with fingers curled around an imaginary doorknob and turned in a clockwise or counterclockwise manner to mimic turning the imaginary doorknob, and a spindle twist gesture made by rubbing a thumb and at least one other finger together. Each of these example gesture types can be detected by radar system 104. When each of these gestures is detected, smartphone 102 can perform an action, such as displaying new content, moving a cursor, activating one or more sensors, opening an application, controlling a game, pinning content to a screen, silencing an alarm, controlling a user interface, or manipulating an AR element. In this way, radar system 104 provides touchless control of smartphone 102.

[0091] As described with reference to detailed view 700-2, when user 704 is within perceptual distance 706, the presence-based functionality can be to provide an electronic assistant or voice interface feature. In other implementations, the presence-based functionality can be another functionality, as described with reference to detailed view 700-3. Figures 8 to 12

[0092] Figure 8 ​Figures illustrate example 800, where the function based on presence is adjustment of the volume of a ringtone, alarm, or notification. In detail view 800-1, user 802 is relatively far from smartphone 102 and outside of the perception distance 706 (indicated by the dashed oval 706), and the ringtone volume is at the default level set by user 802 at 100 percent, as indicated by the ringtone icon 804. In contrast, in detail view 800-2, user 806 is relatively close to smartphone 102 and within the perception distance 706 (again indicated by the dashed oval 706), and the volume is reduced, as indicated by the smaller ringtone icon 808. The volume can be reduced by any appropriate amount (e.g., 25%, 50%, or 75%), and the reduction can be based on any of a variety of factors, such as the distance between user 804 and smartphone 102 or a pre-set or user-selected amount.

[0093] Figure 9 Figures illustrate example 900, where the function based on presence is adjustment of the mode of communication used by an electronic assistant included with smartphone 102. The electronic assistant can communicate in a variety of modes to provide text, audio, video, etc. (e.g., on display 108). In detail view 900-1, user 902 is relatively far from smartphone 102 and outside of the perception distance 706 (indicated by the dashed oval 706), and the electronic assistant interacts in a voice mode without opening display 108, as indicated by the ringtone icon 904 of smartphone 102 and the dimmed display 102. Next, detail view 900-2 shows user 902 relatively close to smartphone 102 and within the perception distance 706 (again indicated by the dashed oval 706), and the electronic assistant supplements the audio communication mode of detail view 900-1 (ringtone icon 904) with glanceable display information 906 (e.g., information that can be read or understood at a glance, such as large text, symbols, or animations) as shown on display 108. In this way, the described technology can reduce the power consumption of smartphone 102 by limiting the use of display 108 to a level appropriate for the presence of user 902.

[0094] In some implementations, the adjustment of the mode of communication can also include adjustment of the content presented on display 108. For example, in detail view 900-3, user 902 picks up smartphone 102. As previously described, radar-based application 702 is able to determine 3D gestures and corresponding actions (e.g., using 3D gesture module 116). Thus, when user 902 reaches for smartphone 102, radar-based application 702 can determine that the reaching is a 3D gesture, and the corresponding action for the electronic assistant is to enable text input for user 902 and provide more rich visual information 908 (e.g., more and smaller text, embedded links, and video), as shown on display 108.

[0095] Figure 10 Figure illustrates an example 1000 in which the function based on storage is to provide notification of the presence of an object within a perceived distance. As described above, the smartphone 102 (e.g., through the radar system 104) can detect whether a user is within a perceived distance of the smartphone 102 and whether the user is alone or with others. Using this information, the smartphone 102 (e.g., through the radar-based application 702) can provide notifications about the status of the user of the smartphone 102 (e.g., the "spatial relational context" of the user, such as whether the user is free (alone), with someone, away, etc.). The notifications can be provided to one or more contacts stored on or accessible by the smartphone 102, and can be provided in any of a variety of ways, such as through a text message or status indicator displayed when the other party views the user's contact information.

[0096] For example, in detailed view 1000-1, the user 1002, whose phone includes presence-based features, views a contact list 1004 (indicated with dashed brackets) that includes some contacts that also have phones (or other electronic devices) that include presence-based features. In detailed view 1000-1, the top four contacts provide notifications of the presence of the user associated with the contact (e.g., George is not available, Susie and Mary are with someone, and Ingrid is free), and the bottom two contacts do not provide notifications (e.g., because the contacts have turned off presence-based features, the contacts' phones do not support presence-based features, or for other reasons). In detailed view 1000-2, the user 1002 selects the contact icon for Susie, and the presence-based function provides the option 1006 to contact Susie. As described above, the notification for Susie indicates that there are other people with Susie, so the option 1006 can include send message (MESSAGE), call now (CALL), or call when alone (CALL WHEN ALONE).

[0097] In detail view 1000-3, assume that user 1002 selects option 1006 to call when Susie is no longer with other people. By using option 1006, both user 1002 and Susie (as shown by another user 1008) receive a phone call simultaneously when the time to contact is appropriate, as indicated by the vibrating indicators around the smart phone 102 and Susie's phone. In some embodiments, how many contacts receive the notification and which contacts receive the notification can be selected by the user or based on default settings. Additionally, option 1006 can be selected by user 1008 (e.g., Susie) so that user 1008 can determine what options the caller (e.g., user 1002) has when provided with the notification.

[0098] By broadcasting presence information or sending it to specific contacts, presence-based functionality that provides notification of the presence of a subject within a perceived distance can also enhance remote collaboration tools. For example, in a document sharing application, the notification can help a remote user determine whether another user is actually in front of a computer or just has the document sharing application open. In some cases, the radar-based application 702 can provide additional details in the notification. For example, by using additional sensors or features (e.g., geo-locating features or hands-free driving features), the notification can include information that allows the user to understand that the potential recipient of the call is in a car, is driving (or not), and is with other people. The radar-based application 702 can also suggest other methods of communication when the notification indicates that the potential recipient is with other people, such as by suggesting email as a way to avoid diverting the attention of the person with other people (e.g., who can be in a meeting).

[0099] Figure 11Figures illustrate example 1100, where the function based on presence is to provide a notification of a previously presented reminder. In detail view 1100-1, user 1102 sets a reminder through the electronic assistant using a voice command (shown as text box 1104). In detail view 1100-2, assume that user 1102 is doing chores at a location outside of the perception distance 706. While user 1102 is outside of the perception distance 706 for the duration of the reminder, the time for the reminder passes and the reminder would have been presented when user 1102 left. In some implementations, because the smartphone 102 knows that the user left, the reminder can not have been presented at all, or the reminder can have been presented at a lower volume or in a different mode (e.g., text instead of voice). In detail view 1100-3, assume that user 1102 returns 30 minutes after the reminder is believed to have been presented. When the smartphone 102 determines that user 1102 is within the perception distance 706, the reminder is presented (shown as text box 1106). In some implementations, the reminder can include additional information that can help user 1102. For example, a suggestion to hang the washed clothes through the clothes dryer for a few minutes to reduce wrinkles, a suggestion to change a restaurant reservation, etc.

[0100] Figure 12 Figures illustrate example 1200, where the function based on presence is to provide an enhanced media experience. For example, wallpaper, maps, and other digital content can include animation and depth. In detail view 1200-1, user 1202 is reading a book and reaches for the smartphone 102. The display 108 automatically turns on and presents a short wallpaper animation 1204. In detail view 1200-1, the wallpaper animation 1204 is a clock face that shows how long user 1202 has been reading, but other wallpaper animations 1204 can be presented (favorite movie clip, video recorded by the user, etc.). The wallpaper animation 1204 can be any appropriate length (e.g., 1 second, 2 seconds, or 3 seconds) and can be user selected or selected by the radar-based application 702. In some implementations, when user 1202 withdraws the extended hand, the wallpaper animation 1204 is displayed in reverse (e.g., the clock face is reversed to show the time when user 1202 started reading). The radar-based application 702 can present other content based on the presence of the user. For example, in detail view 1200-2, user 1206 is within the perception distance 706 and, in response, the radar-based application 702 presents a favorite image 1208.

[0101] Further, by using digital beamforming and other techniques, as described with reference to Figures 3 to 6 Figures, the radar system 104 can determine a user's point of view. The radar-based application 702 can then dynamically adjust the image presented on the display 108 to accommodate the user's point of view, creating the illusion of a three-dimensional space on the display 108. For example,Figure 2 In some embodiments, the user's smartphone is placed on a table, and a three- dimensional (3D) object (a map, a sculpture, a building, etc.) is rendered on the display (not shown). As the user moves around the table, the rendering of the 3D object changes to accommodate the user's point of view, enriching the three- dimensional illusion by giving the user the appearance of moving around the real object. In some embodiments, the user can rotate the device, rather than moving around a fixed device, and view the 3D object from different angles.

[0102] In other embodiments, the smartphone 102 can determine, using the radar-based application 702 or another entity, for example, that more than one object is present while the object 704 is within the perceived distance (e.g., determine the presence of another object within the perceived distance 706). In response, the radar-based application 702 can modify the presence-based functionality of the smartphone 102. Several examples of such embodiments are described below with reference to Figure 13 and Figure 14 .

[0103] Figure 13 FIGURE 13 illustrates an example 1300 in which the modification of the presence-based functionality based on the presence of another object within the perceived distance 706 is an adjustment to the notification mode used by the smartphone 102. In the detailed view 1300-1, a user 1302 and another user 1304 are sitting together at a table. Because the radar-based application 702 can determine that both the user 1302 and the other user 1304 are within the perceived distance 706, the radar-based application 702 modifies the notification mode of the smartphone 102. In some embodiments, the notifications can be completely muted (e.g., nothing is rendered on the display 108). In other embodiments, as shown in the detailed view 1300-1, the notification mode can be modified so that only a notification indicator 1306 is displayed, rather than a full notification. For example, an icon indicates that a notification (e.g., a notification of a calendar event, a task, or other user-defined notification) is available for viewing, rather than indicating the notification itself. In this way, the smartphone 102 is less disruptive in a business or social setting, and potentially private information that would be displayed in a full notification is not publicly displayed.

[0104] The smartphone 102 can return to the default notification mode, or return to the notification mode that was running before the other user 1304 entered the perceived distance 706, in a variety of ways. For example, as shown in the detailed view 1300-2, the user 1302 can reach for the smartphone 102 to return to the previous notification mode (as indicated by the arrow 1308). After returning to the previous notification mode, the content 1310 of the received notification can be presented by the smartphone 102. In other implementations, different 3D gestures can be used to return to the previous notification mode, or the smartphone can automatically return to the previous notification mode when it is determined that the other user 1304 is no longer within the perceived distance 706.

[0105] Figure 14 An example 1400 is illustrated in which the modification of presence-based functionality is an adjustment to the electronic assistant mode used by the smartphone 102 based on the presence of another object within the perceived distance 706. For example, the electronic assistant can provide reminders and notifications including potentially private information (e.g., content of personal reminders or work-related events) when the user is alone, but limit the content when others are present.

[0106] Consider the detailed view 1400-1 in which the user 1402 is alone and within the perceived distance 706. Because the user 1402 is alone, the electronic assistant can provide an audio reminder for the user 1402 to take medication using a voice interface (shown by the text box 1404 indicating that another audio reminder will be provided in 10 minutes). Assume that a few minutes later (less than 10 minutes later), another user 1406 drops by as shown in the other detailed view 1400-2. The other user 1406 sits within the perceived distance 706 and the radar-based application 702 modifies the electronic assistant mode so that audio is not provided (as shown by the mute icon 1408). In addition to audio, notifications can be muted until the other user 1406 leaves, or reminders can be provided in another mode that provides more privacy, such as a vibration mode, a text mode, or a notification indicator mode, as described with reference to Figure 3

[0107] In another example (not shown), the user 1402 can be watching or listening to content when the other user 1406 enters the perceived distance 706. Even if the user 1402 does not notice that the other user 1406 has come within the perceived distance 706, the smartphone 102 (through the radar-based application or another entity, such as a voice assistant) can recognize the presence of the other user and temporarily pause or hide the content (e.g., in the case of private or confidential content).

[0108] ​In other implementations of embodiments including any of the examples described above, the smartphone 102 (using the radar-based application 702 or another entity) can determine whether the user is alone in conjunction with other sensors. For example, the smartphone 102 can use a microphone to determine the ambient noise level around the user. Based on the presence and ambient noise level, the smartphone 102 can then adjust the volume of phone ringtones, notifications, and voice assistants to appropriate levels to attract the user's attention. Additionally, the radar-based application 702 can communicate with other sensors (e.g., a microphone) so that the modification of presence-based features is still performed when the voice of another user is detected (even if the other person is outside the perception distance 706). Further, by using the 3D gesture module 116 described above, the smartphone 102 can make the ringtones, notifications, and voice assistants quieter (or completely silent) when the user reaches for the smartphone 102, as the smartphone 102 can determine that the user reaching for the smartphone 102 is a 3D gesture indicating that the user will respond.

[0109] In other implementations, the smartphone 102 can determine (using radar data) 3D gestures of objects within the perception distance and perform actions corresponding to the 3D gestures. The determination of 3D gestures can be performed by various entities, such as the radar-based application 702 or the 3D gesture module 116, or a combination of these or other entities. The following describes several examples of such implementations. Figures 15 to 20

[0110] Figure 15 FIGURE 13 illustrates an example 1300 in which a 3D gesture is a hand wave over the display of the smartphone 102 that activates a do-not-disturb mode, which silences ringtones, notifications, and the voice assistant until the user exits the do-not-disturb mode. Sometimes, a user can want to limit what information can be revealed to another person, even when the other person is within the perception distance 706. The techniques described in example 1300 allow the user to share screen content with another person without exposing other content and details of reminders and notifications.

[0111] In detail view 1300-1, assume that the user 1302 wants to give the smartphone 102 to another user 1304 to share a photo. The user 1302 wants to protect personal information, such as other photos or content in reminders and notifications, from being inadvertently revealed to the other user 1304. The user 1302 performs a hand wave from right to left, as shown by arrow 1306, and the radar-based application 702 or another entity places the smartphone 102 in a do-not-disturb mode. With the confidence that personal information will not be revealed to the other user 1304, the user 1302 can then give the smartphone 102 to the other user 1304, as shown by another detail view 1300-2. ​

[0112] In another implementation (not shown), the radar-based application 702 can automatically place the smartphone 102 in pin mode by determining, using radar data, that the device is being handed to another person. As described above, the radar system 104 can enable the radar-based application 702 to determine that two users are present. By determining the relative distance between the first user and the device and the orientation of the device relative to the two users, the pin mode can be automatically activated without requiring an intentional gesture. As described above, these techniques provide a simple and discreet way for a user to protect personal information using 3D gestures while still allowing selected content to be viewed by others.

[0113] Figure 16 Figures illustrate an example 1600 in which the 3D gesture is a hand reaching toward the smartphone 102 to silence the ringer. As described above, the radar-based application 702 can adjust the ringer volume based on the distance between the user and the device. Sometimes, when a user is within the perceived distance 706, the user can want to silence the ringer before answering the call (or not answering the call when the caller is determined). In detailed view 1600-1, a user 1602 is sitting next to the smartphone 102. The smartphone 102 is ringing, as indicated by the ringer volume icon 1604. Assume that the user 1602 wants to silence the ringer before deciding whether to answer. As shown in detailed view 1600-2, the user 1602 reaches toward the smartphone 102, as indicated by arrow 1606. The radar-based application 702 determines the reach as a 3D gesture for the corresponding action of silencing the ringer, which is illustrated by the mute icon 1608. In this way, the techniques described in example 1600 allow a user to silence the ringer by reaching for the phone without having to answer the call or interact with the device controls.

[0114] Figure 17 Figures illustrate an example 1700 in which the 3D gesture is a hand reaching toward the smartphone 102 to exit full-screen mode and access controls on the user interface. As described above, the radar-based application 702 can use the perceived distance 706 and the attention cue to keep an application in a particular mode (e.g., full-screen mode with no visible controls) even when a user is not interacting with the device. In some cases, when a user is within the perceived distance 706, the user can want to interact with the controls of the application without picking up the device or tapping a menu.

[0115] In detail view 1700-1, user 1702 is sitting next to smartphone 102. In this example, assume that user 1702 is watching a video and wants to pause the video. As shown in detail view 1700-2, user 1702 reaches for smartphone 102, as indicated by arrow 1704. Radar-based application 702 determines the reach as a 3D gesture that corresponds to an action to exit full-screen mode and provide a user interface or other controls (shown by pause icon 1706). In some implementations, if user 1702 ends the 3D gesture (e.g., stops reaching and withdraws the hand), the full-screen mode is restored. As shown and described in example 1700, these techniques allow a user to exit full-screen mode and interact with a user interface by reaching for the phone, without having to tap or activate the user interface.

[0116] Figure 18 Example 1800 illustrates a 3D gesture that is a reach for smartphone 102 toward the camera to access camera controls after a photo is framed. As described above, radar-based application 702 can use a perception distance 706 (not shown) and attention cues to keep the application in a particular mode (e.g., full-screen mode with no visible controls). In some cases, many camera controls are unnecessary and only serve to make obstructions and clutter on the display. Thus, a user within the perception distance 706 can want a full-screen environment without controls for a period of time (such as when framing a photo), and then interact with full controls after framing is complete.

[0117] In detail view 1800-1, user 1802 is using the camera on smartphone 102 to frame a photo of a sunset. In this example, assume that user 1802 is trying to frame the photo to include particular landscape features and the sunset, and wants to frame the photo in full-screen mode without controls (or minimal controls, such as shutter button 1804). As shown in detail view 1800-1, user 1802 frames the photo using one hand, so that more of the display is visible. In detail view 1800-2, user 1802 reaches for smartphone 102, as indicated by arrow 1806. Radar-based application 702 determines the reach as a 3D gesture that corresponds to an action to provide a user interface or other controls (such as cropping window 1808). In some implementations, if user 1802 ends the 3D gesture (e.g., stops reaching and withdraws the hand), the full-screen mode is restored. In other implementations, radar-based application 702 can determine that user 1802 is reaching to tap shutter button 1804 (e.g., by determining where user 1802 is reaching), and then only present shutter button 1804, rather than all additional controls. As shown and described in example 1800, these techniques allow a user to enjoy full-screen mode to frame a photo and interact with a user interface by reaching for the phone, without having to tap or activate the user interface.

[0118] Figure 19 An example 1900 is illustrated in which a 3D gesture is reaching for a smartphone 102 to activate a back light or other light source so that the smartphone 102 is more easily located in a low light environment. As described above, the radar-based application 702 can use the perceived distance 706 (not shown) and the attention cue to keep the application in a particular mode (e.g., a power saving dark screen mode). In some cases, the user can even want the screen to be completely off, such as in a deliberately dark environment. Thus, a user within the perceived distance 706 can have turned off the screen of the smartphone 102 or put it in a dim mode while sleeping or watching a movie and then need a small amount of light to help locate the smartphone 102 in the dark environment.

[0119] In detail view 1900-1, a user 1902 is getting ready for bed. In this example, assume that the user 1902 forgot to set an alarm and wants to find the smartphone 102 and set the alarm. As shown in detail view 1900-1, the user 1902 reaches for the smartphone 102 (shown by arrow 1904). The radar-based application determines the reaching as a 3D gesture that corresponds to the action of providing dim light (e.g., a back light) (shown by shaded region 1906). In detail view 1900-2, assume that the user 1902 has dropped the smartphone 102 on the floor of a theater and cannot see it in the dark to retrieve it. The user 1902 reaches for the smartphone 102, as shown by another arrow 1908. The radar-based application 702 again determines the reaching as a 3D gesture that corresponds to the action of providing light (shown by another shaded region 1910).

[0120] Similarly, in detail view 1900-3, assume that the user 1902 has placed the smartphone 102 in a bag. The user 1902 reaches for the smartphone 102, as shown by another arrow 1912. The radar-based application 702 again determines the reaching as a 3D gesture that corresponds to the action of providing light (shown by another shaded region 1914). In some implementations, if the user ends the 3D gesture (e.g., stops reaching and withdraws the hand), the dark screen mode is restored. As shown and described in example 1900, these techniques allow the user to use a dark screen mode to save power or reduce distraction from a bright screen in a dark environment and to be able to locate the phone by reaching for the phone. In other implementations (not shown), the radar-based application 702 determines the reaching as a 3D gesture that corresponds to the action of turning on a vibration mode that gets stronger (e.g., vibrates with increased frequency, amplitude, or both) as the user 1902 gets closer to the smartphone 102.

[0121] Additionally, with reference to Figures 15 to 19 The described techniques, i.e., 3D gestures can be used to provide Figures 17 to 19other features not shown. For example, user interface elements in the browser (such as a navigation bar) can be provided when the user reaches for the smartphone 102, and then disappear when the user moves their hand away. In another example, a lock screen or home screen can automatically present when the user reaches for the smartphone 102 (e.g., the user does not have to press a start button or home button to display the lock screen). In other implementations, the smartphone 102 can detect the shape or orientation of the approaching hand, and change the interface accordingly. For example, the input surface can provide a touchpad interface when the user reaches with one hand (or reaches with one finger), or provide a full keyboard when the user reaches with two hands.

[0122] In other implementations, the smartphone 102 can use the radar-based application 702 or another entity to determine that more than one object will be present while the object 704 is within the perception distance (e.g., determine that another object is present within the perception distance 706 while the object is within the perception distance 706). In response to determining the presence of the object within the perception distance 706, the radar-based application 702 can determine 3D gestures by the object within the perception distance (e.g., using radar data), and perform actions corresponding to the determined gestures. The smartphone 102 can also distinguish the objects from one another, and keep track of which objects make which 3D gestures.

[0123] Figure 20 FIGURE 8 illustrates an example 2000 in which the object (not shown) within the perception distance 706 is a user playing a game using the smartphone (in this case, the smartphone 102), and 3D gestures are used to control the game. Consider the example shown in detail view 2000-1, in which user 2002 and user 2004 are playing a two-player game on the smartphone 102. In detail view 2000-1, user 2004 makes a 3D gesture (shown by the arrow 2006 from left to right). In response to the 3D gesture, the radar-based application 702 can enable the game to perform an action corresponding to the 3D gesture (e.g., return an object in the game environment across a centerline, as shown on the display 108). In this way, user 2002 and user 2004 can control gameplay of the game on the smartphone 102 without obstructing the view of the game.

[0124] In another example shown in detailed view 2000-2, user 2008 is playing a single-player game on smartphone 102. In detailed view 2000-2, user 2008 makes a 3D gesture (shown by arrow 2010 from right to left), and radar-based application 702 can enable the game to perform an action corresponding to the 3D gesture (e.g., move an object in the game environment from left to right). Thus, radar-based application 702 can interact with an application (such as a game) using 3D gestures without obstructing the user's view. In this way, because radar-based application 702 can distinguish between multiple users and their gestures, the game environment can be expanded, even on a single, small screen or device, to provide a better and more immersive multi-player gaming experience.

[0125] Example method

[0126] Figure 21 And Figure 22 An example method 2100 is depicted that enables a smartphone, system, and method implemented in an electronic device. Method 2100 can be performed with an electronic device that includes a radar system that provides a radar field (e.g., smartphone 102 described above). The electronic device can also include a display, such as display 108. The radar field can be used to determine an orientation of the electronic device relative to an object in the radar field, and provide an orientation-based function of the electronic device. The radar field can also be used to determine a change in the orientation of the electronic device, and modify the orientation-based function based on the changed orientation.

[0127] Method 2100 is shown as a set of blocks that specify operations performed, but the method is not necessarily limited to the order or combination of the operations shown. Further, any of the operations can be repeated, combined, re-organized, or linked to provide a variety of additional and / or alternative methods. In the following discussion, reference is made to the example operating environment 100 and Figure 1 entities and processes detailed in Figures 2 to 20 are merely exemplary. The techniques are not limited to being performed by one entity or multiple entities operating on one device.

[0128] In 2102, a radar field is provided. The radar field can be provided by any of a variety of electronic devices (e.g., smartphone 102 described above) that include a radar system (e.g., radar system 104) and a radar-based application (e.g., either or both of radar-based applications 106 or 702, which can include one or more of orientation module 114, 3D gesture module 116, or presence module 708, and any combination thereof). Further, the radar field can be any of a variety of types of radar fields, such as radar field 110 described above.

[0129] At 2104, reflections from objects in a radar field are sensed by a radar system. The objects can be any of a variety of objects, such as wood, plastic, metal, fabric, or organic material. For example, the objects can be a person or a body part of a person (e.g., a hand), such as the objects 112 or 704 described above.

[0130] At 2106, the reflections from the objects in the radar field are analyzed. The analysis can be performed by any of a variety of entities (e.g., the radar system 104 or any of the radar-based applications described herein), and can include a variety of operations or determinations, such as the operations or determinations described above with reference to Figures 3 to 6 the radar-based applications described herein.

[0131] At 2108, radar data, such as the radar data described above, is provided based on the analysis of the reflections. The radar data can be provided by any of a variety of entities, such as the radar system 104 or any of the radar-based applications described herein. In some implementations, the radar system can provide the radar data and pass the radar data to other entities (e.g., any of the radar-based applications and other modules described above with reference to Figures 1 to 20 the radar-based applications described herein. Figure 22 The description of the method 2100 continues in Figure 21 block 2110, as indicated by the letter "A" following block 2108 (which corresponds to the letter "A" preceding block 2110 of Figure 22 the method 2100).

[0132] At 2110, a determination is made of an orientation of the electronic device relative to the object. The determination is based on the radar data, and can be provided by any of a variety of entities, such as the radar system 104 or any of the radar-based applications or other modules described herein.

[0133] At 2112, a directionally-based function of the electronic device is provided in response to the determination of the orientation of the electronic device. The directionally-based function can be provided by any of a variety of entities, such as the radar system 104, any of the radar-based applications 106 or 702, or any of the other modules described herein.

[0134] At 2114, a change in the orientation of the electronic device relative to the object is determined. The determination is based on the radar data, and can be performed by any of a variety of entities, such as the radar system 104, any of the radar-based applications 106 or 702, or any of the other modules described herein.

[0135] In 2116, the orientation-based function is modified in response to determining the change in the orientation of the electronic device relative to the object. The modification of the orientation-based function can be performed by any of a variety of entities, such as the radar system 104, any of the radar-based applications 106 or 702, or any of the other modules described herein.

[0136] As described with reference to Figure 1 One example of an orientation-based function is a do not disturb (DND) mode (e.g., a mode that mutes reminders, notifications, or other device-initiated communications). In this example, the initial orientation to activate the DND mode can be a landscape orientation. It should be noted that the landscape orientation is relative to where the user is located, and not the previous location of the electronic device. Thus, the user does not have to first orient the electronic device and then place it in landscape mode. Rather, the user simply rotates the electronic device to a landscape orientation, and the DND mode is activated.

[0137] In this example, the change in the orientation of the electronic device relative to the object is a rotation of the electronic device. The rotation can be in a plane that is substantially parallel to the viewing surface of the display 108 (e.g., a flat rotation on the surface). Moreover, the user does not have to pick up the electronic device in order to change the orientation to be determined. Once it is determined that the change in the orientation of the electronic device has caused the electronic device to enter another orientation relative to the object (e.g., a portrait orientation), the electronic device exits the DND mode. Once the DND mode is deactivated, any messages or notifications that were muted are displayed so that the user can address them.

[0138] Consider the following additional examples, which are illustrated in Figures 23 to 25 The additional examples describe example implementations of the electronic device 102 that can implement additional details of the method 2100. In the examples described with reference to Figures 23 to 25 The described features and techniques are implemented in a smartphone 102, although the described features and techniques can be used with any of a variety of electronic devices (e.g., as described with reference to Figure 2 The described features and techniques are implemented in a smartphone 102, although the described features and techniques can be used with any of a variety of electronic devices (e.g., as described with reference to Figure 23 An example implementation 2300 is depicted in which the orientation-based function of the smartphone 102 is to present content (e.g., through the display 108) in a user-facing orientation. In this example, the smartphone 102 provides a radar field 110 (e.g., through the radar system 104). In the detailed view 2300-1, a user 2302 (an object in the radar field) is viewing content on the smartphone 102 in a portrait orientation. The smartphone 102 (using, for example, the radar-based application 106) determines the orientation of the smartphone 102 relative to the user 2302, and presents the content on the display in a user-facing orientation, as shown by the samurai image 2304.

[0139] In detail view 2300-2, user 2302 changes the orientation of smartphone 102 to another orientation relative to user 2302 (e.g., a landscape orientation), as indicated by arrow 2306. In this example, the change in orientation is a planar rotation on the table on which smartphone 102 is resting (e.g., the change in orientation of the smartphone relative to the user is a rotation of the smartphone in a plane that is generally parallel to the viewing surface of display 108). Smartphone 102 (using, for example, radar-based application 106) can then determine that the change in orientation has resulted in smartphone 102 being in another orientation relative to user 2302, and in response, maintain the user-facing orientation of the content while the smartphone is in the other orientation. Thus, samurai image 2304 remains in the user-facing orientation of detail view 2300-1 even though smartphone 102 has been rotated, and user 2302 has neither picked up the device nor manually changed the orientation.

[0140] Figure 24 An example implementation 2400 is depicted in which a direction-based function of smartphone 102 is to present content (e.g., via display 108) in an orientation facing a user who is looking at the display. In this way, a user can pass smartphone to another user, who will automatically get the display in the proper orientation. In this example, smartphone 102 provides radar field 110 (e.g., via radar system 104). In this example, assume that user 2402 and another user 2404 (e.g., objects in the radar field include a first user and a second user) are viewing content on smartphone 102 in a landscape orientation (e.g., an orientation facing the user). Smartphone 102 (using, for example, radar-based application 106) determines the orientation of smartphone 102 relative to user 2402, and presents the content on the display in an orientation facing the user, where the content is oriented toward user 2402 (this view is not shown in FIG. 24). Figure 24

[0141] ​Continuing the example, assume that the user 2402 decides to have another user 2404 take a closer look at the content, and moves the smartphone 102 toward the other user 2404, as indicated by arrow 2406 (e.g., by displacing the smartphone 102, the user 2402 changes the orientation of the smartphone 102 relative to the user 2402, which causes the smartphone 102 to be farther away from the first user and closer to the second user). The smartphone 102 (using, for example, the radar-based application 106) can then determine that the change in orientation has caused the smartphone 102 to be in another orientation relative to the user 2402, and in response, present the content on the display in an orientation facing the other user 2404. Thus, the content is automatically oriented to face the other user 2404, even though the smartphone 102 has been rotated, and the user 2402 has neither picked up the device nor manually changed the orientation.

[0142] Figure 25 An example implementation 2500 is depicted in which the radar-based application 106 or another entity can use the attention cue to determine a content display mode to present content on a display (e.g., the display 108) of the smartphone 102. The radar-based application 106 uses the attention cue to determine an orientation of the smartphone 102 relative to the user 112, and also uses a change in the attention cue to determine a change in the orientation of the smartphone 102 relative to the user 112. In this way, the attention cue can be used to determine whether the user 112 is paying attention to the smartphone 102.

[0143] The attention cue is a body posture or position of the user 112, such as leaning forward or backward, or turning left or right, which can be detected using radar data, as described with respect to Figure 1 For example, the radar data can be used to determine the attention cue by determining an angle of the user's torso or head relative to a plane that is generally parallel to a viewing surface of the smartphone 102 (e.g., a viewing surface of the display), such as a table on which the smartphone 102 is resting (e.g., the attention cue is whether the user 112 is leaning toward the smartphone 102) or an angular position of the user's torso or head (e.g., the attention cue is whether the user 112 is facing away from the smartphone 102) or a presence of the user 112 within a threshold distance of the smartphone 102.

[0144] In the example implementation 2500, the orientation-based function of the smartphone 102 is to present content on the display in an attention cue-based content display mode (e.g., a first content display mode). The content display mode can be any of a variety of modes that the smartphone 102 can use when displaying or presenting content, such as a media pause mode or a media play mode, a lock screen mode, a sleep or wake mode, a full screen or slideshow mode, or including a particular screen brightness level or volume level.

[0145] In detail view 2500-1, assume that user 112 is reading a book while listening to music via smartphone 102. The first content display mode in this example is a dimmed brightness level, as shown by shaded region 2502 on the display of smartphone 102. The attention cue in this example is the angular position of the head of user 112, as shown by shaded arrow 2504, which indicates that user 112 is looking at the book, rather than at the screen. In this way, smartphone 102 can automatically dim the screen when user 112 looks away, but continue to play music, thereby saving battery power, by basing the modification of the directional-based functionality on the attention cue.

[0146] Continuing the example in another detail view 2500-2, user 112 continues to listen to music, but has turned to smartphone 102, as shown by curved arrow 2506. The turning of the head of user 112 indicates that user 112 is now looking at smartphone 102, rather than at the book, as shown by another shaded arrow 2508. The turning of the head of user 112 is a change in the attention cue. In response to determining the change in the attention cue, smartphone 102 (via radar-based application 106 or another entity) can present the content in another content display mode (e.g., modify the directional-based functionality of smartphone 102 to present the content in a second content display mode). In example 2500, the second content display mode is to present a user interface on the display, shown by music icon 2510. User 112 can interact with the user interface (e.g., via touch, voice, or 3D gesture input) to control the music player to play music.

[0147] As described above, in some embodiments including the embodiments described above, smartphone 102 can determine, via radar-based application 106 or another entity, that a rotation (e.g., a rotation of smartphone 102 or the torso or head of user 112) is effective to cause a modification of the directional-based functionality if the rotation exceeds a threshold. The threshold can be any appropriate rotation, such as (25 degrees, 45 degrees, 90 degrees, or 180 degrees). Similarly, a displacement can be effective to cause a modification of the directional-based functionality if the displacement causes smartphone 102 to be at least a threshold distance away from the first user. The threshold distance can be any appropriate distance, such as 6 inches, 12 inches, or 18 inches. Further, an angle (e.g., an angle of forward or backward tilt of user 112) can be effective to cause a modification of the directional-based functionality if the angle exceeds a threshold angle. The threshold angle can be any appropriate angle, such as (15 degrees, 25 degrees, 35 degrees, or 45 degrees). The threshold rotation, distance, or angle can be predefined, user-selectable, or determined via a machine learning module included in or associated with radar system 104 or radar-based application 106.

[0148] Additionally, if a position, angle, rotation, and other orientation changes remain for at least a threshold amount of time, then these changes can be effective to cause modification of orientation-based functionality. The threshold amount of time can be any appropriate amount of time (e.g., 0.5 seconds, 1.5 seconds, or 2.5 seconds), and can be predefined, user-selectable, or determined by a machine learning module included in or associated with the radar system 104 or radar-based application 106.

[0149] It should be noted that these techniques for smartphones, systems, and methods implemented in electronic devices can be more secure than other techniques that can provide similar functionality. Not only can an unauthorized person typically not obtain a user's position, orientation, or 3D gestures (especially user-defined gestures, micro-gestures, and gesture-based or position-based gestures) (e.g., unlike a password), but a user's radar image (even if the user's radar image includes the user's face) cannot visually identify the user as a photograph or video can. Even so, in addition to the descriptions above, a user can be provided with controls that allow the user to make choices about whether and when any of the systems, programs, managers, modules, or features described in this document can enable collection of user information (e.g., information about a user's social network, social actions or activities, employment, a user's preferences, or a user's current location), as well as whether to send content or communications to a user from a server. In addition, certain data can be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user's identity can be treated so that no personally identifiable information can be determined for the user, or a user's geographic location can be generalized where location information is obtained (such as to a city, postal code, or state level), so that a particular location of a user cannot be determined. Thus, the user can have control over what information is collected about the user, how that information is used, and what information is provided to the user or used in connection with the user.

[0150] Example Computing System

[0151] Figure 26 Various components of an example computing system 2600, which can be implemented as the example computing system 2600 referred to previously, are shown in FIG. 26. The example computing system 2600 includes a bus 2601, a processor 2602 for executing instructions, e.g., instructions stored in a memory area 2603 or on a storage medium 2604 or a removable media device 2605, a display 2606, a user input device 2607, such as a keyboard, a mouse, a joystick, a game controller, a touch screen, a microphone, a camera, and / or a memory area 2608. Figures 1 to 25 Any type of client, server, and / or electronic device described can be implemented to implement a smartphone, system, and method implemented in an electronic device, or techniques to enable a smartphone, system, and method implemented in an electronic device can be implemented in the example computing system 2600.

[0152] Computing system 2600 includes communication device 2602 that enables wired and / or wireless communication of device data 2604 (e.g., radar data, 3D gesture data, authentication data, reference data, received data, data being received, data scheduled for broadcast, packet of data of data) with some, the internet, and / or any one or more external devices not physically attached to computing system 2600 but which can be in communication via I / O interface 2608. Device data 2604 or other device content can include configuration settings of the device, media content stored on the device, and / or information associated with a user of the device. Media content stored on computing system 2600 can include any type of radar, biometric, audio, video and / or image data. Computing system 2600 includes one or more data inputs 2606 via which any type of data, media content, and / or inputs can be received, such as speech data, interactions with a radar field, touch input, user-selectable inputs (explicit or implicit), messages, music, television media content, recorded video content, and any other type of audio, video, and / or image data received from any content and / or data source. Data inputs 2606 can include, for example, radar-based applications 106 and 702, orientation module 114, 3D gesture module 116, or presence module 708.

[0153] Computing system 2600 also includes communication interface 2608 that can be implemented as any one or more of a serial and / or parallel interface, a wireless interface, any type of network interface, any type of modem, and / or as any other type of communication interface. Communication interface 2608 provides a connection and / or communication links between computing system 2600 and a communication network by which other electronic, computing, and communication devices communicate data with computing system 2600.

[0154] Computing system 2600 includes one or more processors 2610 (e.g., any of microprocessors, controllers, or other controllers) which process various computer- executable instructions to control the operation of computing system 2600 and / or enable techniques for smart phones, systems, and methods implemented in electronic devices or enable techniques implemented in electronic devices for smart phones, systems, and methods. Alternatively or additionally, computing system 2600 can be implemented with any one or combination of hardware, firmware, or fixed logic circuitry that is implemented in connection with processing and control circuits which are generally identified at 2612. Although not shown, computing system 2600 can include a system bus or data transfer system that couples the various components within the device. A system bus can include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a serial bus, a parallel bus, and / or a processor or local bus using any of a variety of bus architectures.

[0155] Computing system 2600 also includes computer-readable media 2614, such as one or more memory devices that enable persistent and / or non-transitory data storage (i.e., in contrast to mere signal transmission), examples of which include random access memory (RAM), non-volatile memory (e.g., any one or more of a read-only memory (ROM), flash memory, EPROM, EEPROM, or the like), and a disk storage device. A disk storage device can be implemented as any type of magnetic or optical storage device, such as a hard disk drive, a recordable and / or rewritable compact disc (CD), any type of a digital versatile disc (DVD), and the like. Computing system 2600 can also include a mass storage media device (mass storage media) 2616.

[0156] Computer-readable media 2614 provide data storage mechanisms to store the device data 2604, as well as various device applications 2618 and any other types of information and / or data related to operational aspects of computing system 2600. For example, an operating system 2620 can be maintained as a computer application with computer-readable media 2614 and executed on processors 2610. The device applications 2618 can include a device manager, such as any form of a control application, software application, signal-processing and control modules, code that is native to a particular device, abstract modules, gesture-recognition modules, and other modules. The device applications 2618 also can include system components, engines, or managers for implementing the smart phone, systems, and methods implemented in electronic devices, such as the radar system 104, the radar-based application 106, the radar-based application 702, the directional module 114, the 3D gesture module 116, or the presence module 708. Computing system 2600 can also include or have access to one or more machine learning modules or systems.

[0157] Some examples are described below.

[0158] Example 1. A smartphone, the smartphone comprising: a display; a radar system, the radar system implemented at least in part in hardware, the radar system configured to: provide a radar field; sense reflections from objects in the radar field; analyze the reflections from the objects in the radar field; and provide radar data based on the analysis of the reflections; one or more computer processors; and one or more computer-readable media having stored thereon instructions that, in response to execution by the one or more computer processors, perform operations including: determining, based on a first subset of the radar data, an orientation of the smartphone relative to the objects; providing, in response to determining the orientation of the smartphone, an orientation-based function of the smartphone; determining, based on a second subset of the radar data, a change in the orientation of the smartphone relative to the objects; and modifying, in response to the change in the orientation, the orientation-based function of the smartphone.

[0159] Example 2. The smartphone of example 1, wherein: when the smartphone is determined to be in a first orientation relative to the objects, the orientation-based function of the smartphone is to enter a do-not-disturb (DND) mode, particularly a DND mode that mutes device-originated communications; the change in the orientation of the smartphone relative to the objects is a rotation of the smartphone in a plane that is substantially parallel to a viewing surface of the display, the rotation effective to cause the smartphone to be in a second orientation relative to the objects; and the modification to the orientation-based function is to exit the DND mode.

[0160] Example 3. The smartphone of example 1 or 2, wherein: the objects are a user; the orientation-based function of the smartphone is to present content on the display in a user-facing orientation; the change in the orientation of the smartphone relative to the user is a rotation of the smartphone in a plane that is substantially parallel to a viewing surface of the display, the rotation effective to cause the smartphone to be in another orientation relative to the user; and the modification to the orientation-based function is to maintain the user-facing orientation of the content while the smartphone is in the other orientation relative to the user.

[0161] Example 4. The smartphone of at least one of the preceding examples, wherein: the object comprises at least a first user and a second user; the orientation-based function of the smartphone is to present content on the display in an orientation facing the first user; the change in orientation of the smartphone relative to the first user is a displacement of the smartphone effective to cause the smartphone to be farther from the first user and closer to the second user; and the modification to the orientation-based function is to present content on the display in an orientation facing the second user.

[0162] Example 5. The smartphone of at least one of the preceding examples, wherein: the object is a user; determining an orientation of the smartphone relative to the user based on the radar data further comprises detecting an attention cue from the user; the orientation-based function of the smartphone is to present content on the display in a first content display mode, the first content display mode based on the attention cue; the change in orientation of the smartphone relative to the user is a change in the attention cue; and the modification to the orientation-based function of the smartphone is to present content on the display in a second content display mode, the second content display mode based on the change in the attention cue.

[0163] Example 6. The smartphone of Example 5, wherein: the first content display mode and the second content display mode comprise one or more of: a media pause mode, a media play mode, a lock screen mode, a sleep mode, a wake mode, a full screen mode, a slideshow mode, a screen brightness level, or a volume level.

[0164] Example 7. The smartphone of Example 5 or 6, wherein the attention cue is a body posture of the user, the body posture comprising one or more of: an angle of a torso of the user relative to a plane substantially parallel to a viewing surface of the display; an angle of a head of the user relative to a plane substantially parallel to a viewing surface of the display; an angular position of the torso of the user; an angular position of the head of the user; or a presence of the user within a threshold distance of the smartphone.

[0165] Example 8. A system comprising: an electronic device; a radar system implemented at least partially in hardware, the radar system configured to: provide a radar field; sense reflections from objects in the radar field; analyze the reflections from the objects in the radar field; and provide radar data based on the analysis of the reflections; one or more computer processors; and one or more computer-readable media having stored thereon instructions that, in response to execution by the one or more computer processors, perform operations including: determining, based on a first subset of the radar data, a presence of the objects within a perception distance of the electronic device; in response to determining the presence of the objects within the perception distance, providing a presence-based functionality of the electronic device; determining, based on a second subset of the radar data, that the objects are outside the perception distance of the electronic device; and in response to determining that the objects are outside the perception distance, ceasing to provide the presence-based functionality.

[0166] Example 9. The system of example 8, wherein the presence-based functionality is one or more of: an adjustment to a volume of a ringtone, an alarm, or a notification; an adjustment to a mode of communication used by an electronic assistant included in the electronic device; an adjustment to content presented on a user interface; providing a notification of the presence of the objects within the perception distance, the notification being provided to one or more contacts stored on or accessible by the electronic device; or providing a notification of a reminder previously presented when the objects are not within the perception distance.

[0167] Example 10. The system of example 8 or 9, wherein the electronic device includes a display, and the radar-based application is further configured to: determine, while the objects are within the perception distance, a presence of another object within the perception distance; and in response to determining the presence of the another object within the perception distance, modify the presence-based functionality of the electronic device.

[0168] Example 11. The system of at least one of examples 8-10, wherein the radar-based application is further configured to: determine, based on the radar data, a 3D gesture of the objects within the perception distance of the electronic device; and perform an action corresponding to the determined 3D gesture.

[0169] Example 12. The system of at least one of Examples 8-11, wherein the electronic device includes a display, and the radar-based application is further configured to: while the object is within the awareness distance, determine a presence of another object within the awareness distance; and responsive to the presence of the another object within the awareness distance and based on the radar data, determine: a gesture of the object within the awareness distance of the electronic device; and another gesture of the another object within the awareness distance of the electronic device; perform an action corresponding to the determined gesture; and perform an action corresponding to the determined another gesture.

[0170] Example 13. The system of Example 8, wherein the radar system further includes a digital beamformer and an angle estimator, and the radar system is configured to monitor angles between about -90 degrees and about 90 degrees in a field of view.

[0171] Example 14. A method implemented in an electronic device that includes a radar system and a radar-based application, the method comprising: providing, by the radar system, a radar field; sensing, by the radar system, reflections from an object in the radar field; analyzing the reflections from the object in the radar field; based on the analysis of the reflections, providing radar data; based on a first subset of the radar data, determining an orientation of the electronic device relative to the object; responsive to determining the orientation of the electronic device, providing an orientation-based functionality of the electronic device; based on a second subset of the radar data, determining a change in the orientation of the electronic device relative to the object; and responsive to the change in the orientation, modifying the orientation-based functionality of the electronic device.

[0172] Example 15. The method of Example 14, wherein the electronic device includes a display, the change in the orientation of the electronic device relative to the object is a rotation of the electronic device in a plane that is substantially parallel to a viewing surface of the display, and the method further comprises: responsive to determining the orientation of the electronic device relative to the object, causing the electronic device to enter a do-not-disturb (DND) mode, particularly a DND mode that mutes device-initiated communications; determining that the change in the orientation of the electronic device is effective to cause the electronic device to be in another orientation relative to the object; and responsive to determining that the electronic device is in the other orientation relative to the object, causing the electronic device to exit the DND mode.

[0173] Example 16. The method of example 14 or 15, wherein: the electronic device includes a display; the object is a user; the change in orientation of the electronic device relative to the user is a rotation of the electronic device in a plane that is substantially parallel to a viewing surface of the display; and the method further comprises: presenting content on the display in a user-facing orientation in response to determining the orientation of the electronic device relative to the user; determining that a change in orientation of the electronic device is effective to cause the electronic device to be in another orientation relative to the user; and in response to determining that the electronic device is in the other orientation, maintaining the user-facing orientation of the content while the electronic device is in the other orientation relative to the user.

[0174] Example 17. The method of at least one of examples 14 to 16, wherein: the electronic device includes a display; the object includes at least a first user and a second user; determining the orientation of the electronic device relative to the object includes determining the orientation of the electronic device relative to the first user; determining a change in orientation of the electronic device relative to the object includes determining a change in orientation of the electronic device relative to the first user; the change in orientation of the electronic device relative to the first user is a displacement of the electronic device that is effective to cause the electronic device to be farther from the first user and closer to the second user; and the method further comprises: presenting content on the display in a first user-facing orientation in response to determining the orientation of the first user; and presenting content on the display in a second user-facing orientation in response to determining the change in orientation of the electronic device relative to the first user.

[0175] Example 18. The method of at least one of examples 14 to 17, wherein: the electronic device includes a display; the object is a user; determining the orientation of the electronic device relative to the user based on the radar data further comprises detecting a gaze cue from the user; the change in orientation of the electronic device relative to the user is a change in the gaze cue; and the method further comprises: presenting content on the display in a first content display mode, the first content display mode based on the gaze cue; and in response to determining the change in the gaze cue, presenting content on the display in a second content display mode, the second content display mode based on the change in the gaze cue.

[0176] Example 19. The method of example 18, wherein: the first content display mode and the second content display mode include one or more of: a media pause mode, a media play mode, a screen brightness level, a lock screen mode, a sleep mode, a full screen mode, a slideshow mode, or a volume level.

[0177] Example 20. The method of example 18 or 19, wherein the attention cue is a body pose of the user, the body pose comprising one or more of: an angle of a torso of the user relative to a plane that is substantially parallel to a viewing surface of the display or an axis that is substantially perpendicular to the plane containing the electronic device; an angular position of the torso of the user; an angle of a head of the user relative to a plane that is substantially parallel to a viewing surface of the display or an axis that is substantially perpendicular to the plane containing the electronic device; an angular position of the head of the user; or a presence of the user within a threshold distance of the electronic device.

[0178] CONCLUSION

[0179] While implementations of techniques and apparatuses for smartphones, systems, and methods that enable implementation in electronic devices have been described in language specific to features and / or methods, it is to be understood that the subject of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as example implementations of smartphones, systems, and methods that enable implementation in electronic devices.

Claims

1. An electronic device, comprising: a radar system, the radar system implemented at least partially in hardware, the radar system configured to: provide a radar field; sense reflections from a user in the radar field; analyze the reflections from the user in the radar field; and provide radar data based on the analysis of the reflections; one or more computer processors; and one or more computer-readable media having stored thereon instructions that, in response to execution by the one or more computer processors, perform operations including: determining, based on a first subset of the radar data, a presence of the user within a perception distance of the electronic device; determining, based on a second subset of the radar data, whether the user is alone within the perception distance of the electronic device; in response to determining that the user is within the perception distance and whether the user is alone within the perception distance of the electronic device, determining a spatial relational context of the user; and providing the spatial relational context of the user to one or more other electronic devices.

2. The electronic device of claim 1, wherein: the spatial relational context of the user includes a status of the user with respect to the electronic device; and the status of the user with respect to the electronic device includes: available, the available status indicating that the user is alone within the perception distance of the electronic device; busy, the busy status indicating that the user is within the perception distance of the electronic device and the user is not alone; or away, the away status indicating that the user is not within the perception distance of the electronic device. providing the spatial relational context of the user to one or more other electronic devices further includes providing a status indicator: the status indicator configured to be displayed with contact information of the user on the one or more other electronic devices; and 3. The electronic device of claim 2, wherein, the status indicator indicating the status of the user with respect to the electronic device. the operations further include providing, to the one or more other electronic devices, a selectable control configured to be displayed with the contact information of the user, the selectable control including at least one of: a first selectable control configured to cause the one or more other electronic devices to send a message to the electronic device; 4. The electronic device of claim 3, wherein, a second selectable control configured to cause the one or more other electronic devices to place a telephone call to the electronic device; a third selectable control configured to cause the one or more other electronic devices to place a telephone call to the electronic device when the status of the user with respect to the electronic device changes from busy or away to available; or a fourth selectable control configured to cause the electronic device and the one or more other electronic devices to simultaneously receive a telephone call from each other when the status of the user with respect to the electronic device changes from busy or away to available. ​ ​ ​ 5. The electronic device of claim 4, wherein, providing the one or more other electronic devices with the spatial relationship context of the user further includes communicating the spatial relationship context to the one or more other electronic devices using a text message.

6. The electronic device of claim 2, wherein, The text message includes a selectable control for contacting the user, the selectable control including at least one of:

7. The electronic device of claim 6, wherein, a first selectable control configured to cause the one or more other electronic devices to send a message to the electronic device; a second selectable control configured to cause the one or more other electronic devices to place a telephone call to the electronic device; a third selectable control configured to cause the one or more other electronic devices to place a telephone call to the electronic device when a status of the user with respect to the electronic device changes from busy or away to available; or a fourth selectable control configured to cause the electronic device and the one or more other electronic devices to simultaneously receive a telephone call from each other when a status of the user with respect to the electronic device changes from busy or away to available. communicating the spatial relationship context to the one or more other electronic devices using a text message further includes determining, based on user input through the electronic device, that one or more of the first selectable control, the second selectable control, the third selectable control, or the fourth selectable control is to be included in the text message. The operations further include:

8. The electronic device of claim 7, wherein, determining, based on user input, one or more other electronic devices to provide the spatial relationship context of the user; or 9.The electronic device of claim 1, wherein, determining, based on a default setting of the electronic device, one or more other electronic devices to provide the spatial relationship context of the user.

10. The electronic device of claim 1, wherein: the second subset of the radar data and the first subset of the radar data are the same subset of the radar data; the second subset of the radar data does not include radar data of the first subset of the radar data; or the second subset of the radar data includes at least a portion of the first subset of the radar data.

11. An electronic device, comprising: one or more computer processors; and one or more computer-readable media having stored thereon instructions that, in response to execution by the one or more computer processors, perform operations including: receiving, from another electronic device that includes a radar system, a spatial relationship context of a user of the other electronic device, the spatial relationship context of the user being determined based on one or more subsets of radar data provided by the radar system and including a status of the user with respect to the other electronic device; and ​ ​ ​ ​ presenting, on a display of the electronic device and in association with contact information of the user, a status indicator, the status indicator indicating a status of the user with respect to the other electronic device.

12. The electronic device of claim 11, wherein, the status of the user with respect to the other electronic device includes: available, an available status indicating that the user is alone within a perceived distance of the other electronic device; busy, a busy status indicating that the user is within the perceived distance of the other electronic device and the user is not alone; or away, an away status indicating that the user is not within the perceived distance of the other electronic device.

13. The electronic device of claim 11, wherein, presenting the status indicator further includes displaying the status indicator on the display of the electronic device with the contact information of the user.

14. The electronic device of claim 13, wherein, displaying the status indicator on the display of the electronic device with the contact information of the user further includes displaying a selectable control for contacting the user, the selectable control being displayed with the contact information of the user and including at least one of: a first selectable control that, when selected, causes the electronic device to send a message to the other electronic device; a second selectable control that, when selected, causes the electronic device to place a telephone call to the other electronic device; a third selectable control that, when selected, causes the electronic device to place a telephone call to the other electronic device when the status of the user with respect to the other electronic device changes from busy or away to available; or a fourth selectable control that, when selected, causes the electronic device and the other electronic device to simultaneously receive a telephone call from each other when the status of the user with respect to the other electronic device changes from busy or away to available.

15. The electronic device of claim 11, wherein, the one or more subsets of radar data are subsets of radar data provided by a radar system of the other electronic device, the radar system being implemented at least in part in hardware and configured to provide a radar field, sense reflections from the user in the radar field, analyze the reflections from the user in the radar field, and provide the radar data based on the analysis of the reflections.

16. A method implemented in an electronic device, the electronic device including a radar system, the method comprising: providing, by the radar system, a radar field; sensing, by the radar system, reflections from a user in the radar field; analyzing the reflections from the user in the radar field; based on the analysis of the reflections, providing radar data; based on a first subset of the radar data, determining a presence of the user within a perceived distance of the electronic device; based on a second subset of the radar data, determining whether the user is alone within the perceived distance of the electronic device; in response to determining that the user is within the perceived distance and whether the user is alone within the perceived distance of the electronic device, determining a spatial relational context of the user; and providing the spatial relational context of the user to one or more other electronic devices. ​ 17. The method of claim 16, wherein, The spatial relationship context of the user includes a status of the user with respect to the electronic device, and the status of the user with respect to the electronic device includes: available, an available status indicating that the user is alone within a perceived distance of the electronic device; busy, a busy status indicating that the user is within the perceived distance of the electronic device and the user is not alone; or away, an away status indicating that the user is not within the perceived distance of the electronic device.

18. The method of claim 17, wherein, Providing the spatial relationship context of the user to one or more other electronic devices includes providing a status indicator configured to be presented with contact information of the user on respective displays of the one or more other electronic devices and to indicate the status of the user with respect to the electronic device, and the method further includes: providing, to the one or more other electronic devices, a selectable control configured to be displayed with the contact information of the user, the selectable control including at least one of: a first selectable control configured to cause the one or more other electronic devices to send a message to the electronic device; a second selectable control configured to cause the one or more other electronic devices to place a telephone call to the electronic device; a third selectable control configured to cause the one or more other electronic devices to place a telephone call to the electronic device when the status of the user with respect to the electronic device changes from busy or away to available; or a fourth selectable control configured to cause the electronic device and the one or more other electronic devices to simultaneously receive a telephone call from each other when the status of the user with respect to the electronic device changes from busy or away to available.

19. The method of claim 17, wherein: the electronic device includes a non-radar sensor; and the status of the user with respect to the electronic device further includes a moving status determined based at least in part on data from the non-radar sensor and indicating that the user is not alone and is traveling in a vehicle.

20. The method of claim 19, wherein, the non-radar sensor includes one or more of a geolocation sensor or a detection sensor capable of detecting when the electronic device is operating in a hands-free driving mode. the non-radar sensor includes one or more of a geolocation sensor or a detection sensor capable of detecting when the electronic device is operating in a hands-free driving mode.

Images