Smart phone, system and method implemented in electronic device

Abstract

The invention relates to a smartphone, a system and a method implemented in an electronic device. Techniques and systems are described herein that enable smartphones, systems, and methods implemented in an electronic device. The techniques and systems use radar fields to accurately determine the position and body orientation of a user relative to an electronic device, such as a smartphone. The radar field also enables the device to receive 3D gestures from the user to interact with the device. The techniques allow the device to provide functionality based on the presence and orientation of the user, and appropriately adjust the timing, content, and format of the device's interaction with the user.

Description

[0001] Description of the case

[0002] This application is a divisional application of Chinese invention patent application No. 201980017923.9, with an application date of August 5, 2019. Background Art

[0003] Apps on smartphones and other electronic devices provide an ever-growing array of productivity, entertainment, and communication features that have become almost essential. We listen to music, watch videos, and share presentations and documents. These apps remind us of appointments (and sometimes even schedule them), notify us when someone texts or calls, and keep track of projects at work, school, and home. Moreover, as these apps, and the devices on which they run, become more familiar with our schedules and preferences, they suggest alternate transportation routes, suggest restaurants, and otherwise communicate with us independently. Yet, for all the help our apps provide us, for all the computing power and artificial intelligence they have, they remain socially invisible and can become intrusive. That is, no matter how "smart" a smartphone may be, it has no idea what its user is doing (or even if the user is near the device) when it displays or plays reminders, alerts, or suggestions. So if we move to the other side of the device, or try to share the displayed content with someone across the table, the content may be upside down. If we rotate the device, it may not recognize what we want to do until we pick it up, orient it where we want it, and put it down again. In other cases, devices may interrupt us at inconvenient times, or display alerts or notifications in inappropriate locations or at embarrassing volumes or brightness, which may include personal and private information. As a result, utilizing the powerful and fun features of our apps may be inconvenient, embarrassing, and frustrating, and we may not realize the full potential of our electronic devices and apps due to their limited awareness. Summary of the invention

[0004] This document describes techniques and systems that enable smartphones, systems, and methods to be implemented in electronic devices. These techniques and systems use radar fields to accurately determine the location and body orientation of a user relative to an electronic device such as a smartphone. The radar fields also enable the device to receive three-dimensional (3D) gestures from the user to interact with the device. These techniques allow the device to provide functionality based on the user's presence and orientation, and allow the timing, content, and format of the device's interactions with the user to be appropriately adjusted.

[0005] Aspects described below include a system that includes a smartphone, a display, a radar system, one or more computer processors, and one or more computer-readable media. The radar system is implemented at least in part in hardware and provides a radar field. The radar system also senses reflections from objects in the radar field and analyzes reflections from objects in the radar field. The radar system further provides radar data based on the analysis of the reflections. The one or more computer processors include stored instructions that, when executed by the one or more computer processors, can perform operations. The operations include determining an orientation of the smartphone relative to an object based on a first subset of the radar data, and providing orientation-based functionality of the smartphone in response to determining the orientation of the smartphone. The operations also include determining a change in the orientation of the smartphone relative to the object based on a second subset of the radar data, and modifying the orientation-based functionality of the smartphone in response to the change in orientation.

[0006] The aspects described below also include a system comprising an electronic device, a radar system, one or more computer processors, and one or more computer-readable media. The radar system is implemented at least in part in hardware and provides a radar field. The radar system also senses reflections from objects in the radar field and analyzes reflections from objects in the radar field. The radar system further provides radar data based on the analysis of the reflections. One or more computer-readable media include stored instructions that, when executed by one or more computer processors, perform operations. The operations include determining the presence of an object within a perception range of the electronic device based on a first subset of the radar data, and providing a presence-based function of the electronic device in response to determining the presence of the object within the perception range. The operations also include determining that an object is outside the perception range of the electronic device based on a second subset of the radar data, and stopping providing a presence-based function in response to determining that the object is outside the perception range.

[0007] The aspects described below also include a method implemented in an electronic device including a radar system and a radar-based application. The method includes providing a radar field by the radar system, and sensing reflections from objects in the radar field by the radar system. The method also includes analyzing the reflections from the objects in the radar field, and providing radar data based on the analysis of the reflections. The method further includes determining the orientation of the electronic device relative to the object based on a first subset of the radar data; and providing orientation-based functionality of the electronic device in response to determining the orientation of the electronic device. The method also includes determining a change in the orientation of the electronic device relative to the object based on a second subset of the radar data, and modifying the orientation-based functionality of the electronic device in response to the change in orientation.

[0008] Aspects described below also include a system that includes an electronic device that includes or is associated with a device for providing a radar field that provides radar data based on sensing and analyzing reflections from objects in the radar field. The system also includes a device for determining an orientation of the electronic device relative to an object in the radar field and providing an orientation-based function of the electronic device in response to the determined orientation. The system also includes a device for determining a change in the orientation of the electronic device relative to an object in the radar field and modifying the orientation-based function of the electronic device in response to the determined change in orientation.

[0009] This summary is provided to introduce simplified concepts related to smartphones, systems, and methods implemented in electronic devices, which will be further described in the following detailed description and figures. This summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Details of one or more aspects of smartphones, systems, and methods implemented in electronic devices are described in this document with reference to the following figures. In all figures, the same numbers are used to reference similar features and components:

[0011] Figure 1 An example environment is illustrated in which techniques enabling smartphones, systems, and methods implemented in electronic devices may be implemented.

[0012] Figure 2 A smartphone, system and method including a radar system and capable of being implemented in an electronic device are shown. Figure 1 An example implementation of a smartphone.

[0013] Figure 3 Graphics Figure 1 and Figure 2 Example implementation of a radar system.

[0014] Figure 4 Graphics Figure 3 Example arrangement of receiving antenna elements of a radar system.

[0015] Figure 5 Graphics Figure 1 and Figure 2 Additional details of an example implementation of a radar system.

[0016] Figure 6 The diagram can be Figure 1 and Figure 2 Example scenario for radar system implementation.

[0017] Figure 7Another example environment is illustrated in which techniques enabling smartphones, systems, and methods implemented in electronic devices may be implemented.

[0018] Figure 8-12 The diagram can be used with Figure 7 An example of presence-based functionality used in conjunction with the techniques of a smartphone, system, and method described in for implementation in an electronic device.

[0019] Fig.13 and Fig.14 The diagram is as follows Figure 7-12 An example of modification based on existing functionality described in .

[0020] Figure 15-20 The diagram can be used with Figure 7-14 Examples of 3D gestures used in conjunction with the techniques described in the accompanying drawings for smartphones, systems, and methods implemented in electronic devices.

[0021] Fig.21 and 22 Example methods are described to enable smartphones, systems, and methods implemented in electronic devices.

[0022] Figure 23-25 Graphical implementation Fig.21 and Fig. 22 An exemplary embodiment of an electronic device with additional details of a method.

[0023] Fig.26 The diagram can be implemented as shown in the reference Figures 1 to 25 Any type of client, server, and / or electronic device described may be used to implement various components of an example computing system that implements a smartphone, systems, and methods in an electronic device or in which techniques that enable implementation of a smartphone, systems, and methods in an electronic device may be implemented. DETAILED DESCRIPTION

[0024] This document describes technologies and systems that enable smartphones, systems, and methods implemented in electronic devices. As described above, because smartphones and other electronic devices do not know whether a user is near the device, or how the user is positioned relative to the device (and the device's display), the device may not always present content in a way that is convenient for the user. In addition, and again because of lack of awareness, electronic devices may disturb users by displaying reminders or notifications (which may also include personal and private information) at inappropriate or inconvenient times and places, or at embarrassing volumes. It can be frustrating and challenging to orient a device correctly to display content (especially for multiple users) or to remember to put the device in "silent" mode. In addition, users sometimes forget to turn off silent mode, which can result in missing important reminders. As a result, users may not be able to realize the full potential of their smartphones and other devices due to the device's limited recognition of the surrounding environment.

[0025] The described techniques and systems employ a radar system to accurately determine the position and physical orientation of a user relative to an electronic device. The radar field also enables the device to accurately determine three-dimensional (3D) gestures from the user (e.g., including one or more movement gestures in any direction within the 3D space illuminated by the radar field 110), which can be used to interact with the device. Unless otherwise indicated by a specific context, increased accuracy refers to an increased degree of improvement, an increased conformity to the truth, or both an increased degree of improvement and an increased conformity to the truth. By appropriately orienting the display and adjusting the timing, content, and format of the device's interaction with the user, these techniques enable the device to provide functionality based on the user's presence and orientation. Therefore, the described techniques and systems can improve the quality and effectiveness of the user's experience, and thereby increase the user's efficiency, workflow, and entertainment.

[0026] Consider an electronic device that includes a Do Not Disturb (DND) feature that can be activated by orienting the electronic device in a particular way. For example, the DND feature can be activated by orienting the display of the device to face the surface on which the device is sitting. In this example, the electronic device is placed screen-down to enter a mode in which the device does not provide interruptions, such as calendar reminders, email or text notifications, etc. This type of orientation-based DND mode typically prevents the user from even seeing a clock or timer on the screen (e.g., reminding the user when to exit DND mode). In addition, to exit DND mode, the user must pick up the device, tilt it until the display orientation is correct, and then put the device back (which may change the display orientation again). When there are multiple viewers, some of the orientations may still be wrong. Interactions with orientation-based features (such as DND mode or display orientation) are always inconvenient or frustrating, which may reduce efficiency and the user's quality of experience with orientation-based features, and even reduce the likelihood that users will interact with these features.

[0027] Contrast these conventional techniques with the systems and techniques described in this document, which can improve efficiency and usability in multiple areas. For example, in the above example, the user puts the electronic device in DND mode, and it may be difficult to exit DND mode in a simple and convenient manner. In this case, the electronic device may include a radar system that can provide a radar field that extends into an area around the device (e.g., a five, eight, or thirteen-foot radius around the device). The radar sensor can use radar signals reflected from objects entering the radar field to detect the presence and location of the user and the orientation of the device relative to the user. The user can then enter DND mode by turning the device with the display facing up to a specific orientation (e.g., a horizontal orientation). In this case, the user can exit DND mode using a simple orientation change (such as rotating the device to a portrait orientation). This easy movement can put the device in the correct orientation relative to the user without the user having to pick up the device or tilt it because the radar field allows the device to know the desired orientation based on the user's position.

[0028] In this way, the described techniques and systems allow for simple and convenient interaction with orientation-based features. Users can enjoy the benefits and convenience of these features without the disruption and interruption caused by attempting to use orientation-based features without the described techniques. This can increase efficiency and reduce user frustration, such as having to adjust and readjust the orientation of the device to achieve a desired result, thereby increasing the quality of the user experience.

[0029] This is just one example of how the techniques and devices described herein can be used to allow users to enjoy features based on orientation or presence. Other examples and implementations thereof are described in this document. Now, this document turns to an example environment, after which example systems, devices, methods, and components are described.

[0030] Operating Environment

[0031] Figure 1 An example environment 100 is illustrated in which techniques enabling smartphones, systems, and methods implemented in electronic devices may be implemented. The example environment 100 includes a smartphone 102 that includes or is associated with a radar system 104 and a radar-based application 106. In some implementations, the smartphone 102 may include a display 108. Some implementations of the radar system 104 are particularly advantageous when applied in the context of a smartphone such as the smartphone 102 where there are concurrent issues such as low power requirements, requirements for processing efficiency, limitations on spacing and placement of antenna elements, and other issues, and certain embodiments of the radar system 104 are even more advantageous in the specific context of a smartphone where radar detection of fine gestures is desired. Although these embodiments are particularly advantageous in the context of a smartphone where fine radar detection of gestures is desired, it should be understood that the applicability of the features and advantages of the present invention is not necessarily limited thereto, and other embodiments involving other types of electronic devices may also be within the scope of the present teachings.

[0032] In the example environment 100, the radar system 104 provides a radar field 110 by transmitting one or more radar signals or waveforms, as described below with reference to Figures 3 to 6 Radar field 110 is a volume of space from which radar system 104 can detect reflections of radar signals and waveforms (e.g., radar signals and waveforms reflected from objects in the volume of space). Radar system 104 also enables smartphone 102 to sense and analyze reflections from objects 112 in radar field 110. Figure 1 In the described example, although the described features and techniques are implemented in the smartphone 102, the described features and techniques can be used with any of a variety of electronic devices (e.g., as described in reference Figure 2 described).

[0033] Object 112 may be any of a variety of objects from which radar system 104 may sense and analyze reflections, such as wood, plastic, metal, fabric, or a human body part (e.g., a hand of a user of smartphone 102). Figure 1As shown in FIG. 1 , object 112 is a person or user of smartphone 102 (eg, person 112 or user 112 ). Based on the analysis of the reflections, radar system 104 may provide radar data including various types of information associated with radar field 110 and reflections from object 112 , as described below with reference to FIG. Figure 3-6 As described (eg, radar system 104 may transmit radar data to other entities, such as radar-based applications 106 ).

[0034] It should be noted that radar data may be provided continuously or periodically over time based on sensed and analyzed reflections from objects 112 in radar field 110. The location of object 112 may change over time (e.g., object 112 may move within radar field 110), and therefore, radar data may change over time corresponding to the changed location, reflections, and analysis. Because radar data may change over time, radar system 104 may provide radar data including one or more subsets of radar data corresponding to different time periods. For example, radar system 104 may provide a first subset of radar data corresponding to a first time period, a second subset of radar data corresponding to a second time period, and so on.

[0035] Radar-based application 106 can be any of a variety of radar-based applications that can use radar data to determine the orientation of smartphone 102 relative to object 112 and provide orientation-based features or functions to smartphone 102 (e.g., via display 108). In addition, radar-based application 106 can use radar data to determine changes in the orientation of smartphone 102 relative to the object and modify the orientation-based features or functions based on the change in orientation. In this way, the orientation of object 112 within radar field 110 can be used to provide input or instructions to interact with smartphone 102.

[0036] Radar-based application 106 may include or communicate with orientation module 114, which may store in internal or external memory information related to determining the orientation of smartphone 102 relative to object 112 (based on radar data) and information related to features and functions corresponding to a particular orientation of smartphone 102 with respect to object 112. In some embodiments, radar-based application 106 may use the information stored by orientation module 114 to determine the orientation of smartphone 102 relative to object 112. In other embodiments, orientation module 114 itself may determine the orientation of smartphone 102 relative to object 112. Figure 1 As shown in FIG. 1 , orientation module 114 is included in radar-based application 106 , but orientation module 114 may be a separate entity as part of or separate from radar-based application 106 or smartphone 102 .

[0037] Radar-based application 106 may also include or communicate with 3D gesture module 116, which may store both information related to determining a 3D gesture based on radar data (e.g., a 3D gesture by object 112) and information related to an action corresponding to the 3D gesture. Thus, radar-based application 106 may detect a 3D gesture by object 112 and determine an action corresponding to the 3D gesture.

[0038] exist Figure 1 1, the orientation-based functionality of the smartphone 102 relative to the object 112 (in this case, the user 112) is a do not disturb (DND) mode (e.g., a mode in which alerts, notifications, or communications initiated by other devices are muted). Because the display 108 is still visible, non-intrusive content (e.g., a clock, calendar, or media player interface) can still be displayed. As shown in the detailed view 100-1, the smartphone 102 is placed in the DND mode by positioning the smartphone 102 in a first orientation relative to the user 112 (in this case, a landscape orientation). As shown in reference Figure 3-6 As described, radar system 104 may use radar field 110 to sense and analyze reflections from objects in radar field 110 in a manner that enables identification of device orientation with high resolution and accuracy. For example, because smartphone 102 knows its own orientation relative to the radar field, radar data as described above may be used (e.g., by radar-based application 106) to determine a first orientation of smartphone 102 relative to user 112 and provide a DND mode.

[0039] In detailed view 100-2, user 112 has rotated smartphone 102 to a second orientation (in this case, a portrait orientation) relative to user 112. The rotation of smartphone 102 is indicated by arrow 118. Similarly, radar-based application 106 (or another application) can use radar data to determine a change from a landscape orientation to a portrait orientation and, in response to the change in orientation, exit DND mode.

[0040] Note that it is not necessary for the user 112 to tilt or pick up the smartphone 102. For example, in some embodiments, the rotation can be a plane rotation on the surface, a rotation within a plane substantially parallel to the viewing surface of the display 108, or a rotation about an axis substantially perpendicular to the plane containing the smartphone 102, and still effectively put the smartphone 102 in a portrait (second) orientation relative to the user 112. In addition, upon exiting the DND mode, the radar-based application 106 can display any reminders, notifications, or other device-initiated communications that were muted when the smartphone 102 was in the DND mode. In some embodiments, if the rotation exceeds a threshold distance, the radar-based application 106 can determine that the rotation effectively causes a modification of the orientation-based functionality. The threshold distance can be any appropriate distance, such as 25 degrees, 45 degrees, 90 degrees, or 180 degrees, and can be predefined, user-selectable, or determined via a machine learning module included in or associated with the radar system 104 or the radar-based application 106.

[0041] Consider three additional examples (not illustrated) that illustrate how the described technology can implement orientation-based functionality. In the first example, the orientation-based functionality of the smartphone 102 is to present content on the display 108 in an orientation facing the user. In this way, the smartphone 102 can provide videos, photos, games, and other media automatically oriented toward the user 112 via the radar-based application 106 or another application without the user 112 having to pick up the smartphone 102. The user 112 can change the orientation of the smartphone 102, and the smartphone 102 can determine via the radar-based application 106 or another application that the orientation has changed to another orientation, and automatically maintain the user-facing orientation of the content when the smartphone 102 is in another orientation relative to the user 112.

[0042] For example, user 112 may change the orientation of smartphone 102 by moving to another location or by rotating smartphone 102. As described above, the user does not have to tilt or pick up the device. Rather, the change in orientation of smartphone 102 relative to user 112 may be an in-plane rotation on the surface, a rotation within a plane substantially parallel to the viewing surface of display 108, or a rotation of smartphone 112 about an axis substantially perpendicular to the plane containing smartphone 102, and still effectively result in smartphone 102 being in another orientation relative to user 112.

[0043] In a second example, there are two users 112. For example, a first user (Sam) is the owner of a smartphone 102, and a second user (Chris) is a colleague of Sam. In this example, assume that Sam has a presentation that Chris wants to review. The orientation-based functionality of the smartphone 102 will present the content on the display 108 in an orientation facing Sam (e.g., an orientation facing the first user, similar to the orientation-based functionality described in the previous example). Further assume that Chris and Sam are sitting on opposite sides of a table, and Sam agrees to let Chris review the presentation. Sam can let Chris review the presentation by moving the smartphone 102 toward Chris. As described above, radar data can be used (e.g., by a radar-based application 106) to determine a change in the orientation of the smartphone 102. In this case, the change is that the smartphone 102 is farther away from Sam and closer to Chris (e.g., the change in the orientation of the smartphone 102 relative to Sam is a displacement of the smartphone 102 that causes the smartphone 102 to be farther away from Sam and closer to Chris).

[0044] Upon determining a change in orientation, the smartphone 102, via the radar-based application 106 or another application, can automatically modify the orientation-based functionality to present content on the display at an orientation facing Chris (e.g., a second user-facing orientation). Thus, Sam can share a presentation with Chris without having to rotate or flip the smartphone 102. More specifically, when Sam moves the device toward Chris, the displayed content can be automatically redirected toward Chris. In this way, the smartphone 102 can provide videos, photos, games, and other media that are automatically oriented in the appropriate orientation without the user 112 having to manually adjust the orientation.

[0045] In a third example, the radar-based application 106 or another application can use the attention cue to determine the orientation of the smartphone 102 relative to the user 112. Figure 3-6 As described, the radar system 104 can use the radar field 110 to sense and analyze reflections from objects in the radar field 110 in a manner that can identify the number and posture of people in the radar field 110 with high resolution and accuracy. For example, by using angular resolution and digital beamforming, the radar system 104 can distinguish between leaning and standing. In this way, the smartphone 102 can detect attention cues from the user 112. An attention cue is a body posture or position of the user 112, such as leaning forward or backward, or an orientation of the torso or head of the user 112 (e.g., toward or away from the smartphone 102). Attention cues can be used to determine whether the user 112 is paying attention when using the smartphone 102 to read a document, listen to music, look at a photo, etc.

[0046] As described above, radar data may be used to determine an attention cue. For example, radar data may be used to determine an attention cue by determining the angle of the torso or head of user 112 relative to a plane substantially parallel to a viewing surface of display 108 (such as a table on which smartphone 102 is resting) (e.g., whether user 112 is leaning toward smartphone 102), or the angular position of the torso or head of user 112 (e.g., whether user 112 is turned away from smartphone 102), or the presence of user 112 within a threshold distance of smartphone 102.

[0047] In this example, the orientation-based functionality of the smartphone 102 is to present content on the display 108 in a content display mode (e.g., a first content display mode) based on an attention cue. For example, the content display mode may be a media pause mode or a media play mode, a lock screen mode, a sleep or wake mode, a full screen or slide show mode, or include a specific screen brightness level or volume level. The smartphone 102 using the radar-based application 106 or another application may determine a change in the attention cue (e.g., a change in orientation between the user 112 and the smartphone 102), such as a change from leaning forward to leaning backward or a turn of the user 112's head from looking at the display 108 to not looking at the display.

[0048] In response to determining a change in the attention cue, the radar-based application 106 (or another application) can present the content in another content display mode (e.g., modifying the orientation-based functionality of the smartphone 102 to present the content in the second content display mode). As with the first content display mode, the other content display modes can be a media pause mode or a media play mode, a lock screen mode, a sleep or wake mode, a full screen or slide show mode, or include a specific screen brightness level or volume level. Continuing with the above example, assume that the user 112 is watching a video in full screen mode and then leans forward toward the smartphone 102. In response, the radar-based application 106 can determine that the user 112 is about to interact with the video player and exit the full screen mode.

[0049] In this manner, the smartphone 102 can use the attention cues to help determine the context in which the user 112 is using the smartphone 102, thereby providing an improved user experience. For example, if the smartphone 102 determines that the user 112 is not paying attention to a video that is playing, such as by determining that the user 112 has turned away from the display 108 (or has left the room), the smartphone 102 can pause the video and dim the screen brightness so that the user does not miss any content and save battery power.

[0050] In some embodiments, including the above example embodiments, the radar-based application 106 may determine that a rotation (e.g., a rotation of the smartphone 102 or the torso or head of the user 112) is effective to cause a modification to the orientation-based functionality if the rotation exceeds a threshold. The threshold may be any suitable rotation, such as (25 degrees, 45 degrees, 90 degrees, or 180 degrees). Similarly, if the displacement is effective to cause the smartphone 102 to become farther away from Sam and closer to Chris by at least one threshold distance, the displacement may be effective to cause a modification to the orientation-based functionality. The threshold distance may be any suitable distance, such as six, twelve, or eighteen inches. In addition, if the angle (e.g., the angle at which the user 112 leans forward or backward) exceeds a threshold angle, the angle may be effective to cause a modification to the orientation-based functionality. The threshold angle may be any suitable angle, such as 15 degrees, 25 degrees, 35 degrees, or 45 degrees. The threshold rotation, distance, or angle may be predefined, user-selectable, or determined via a machine learning module included in or associated with the radar system 104 or the radar-based application 106.

[0051] Additionally, if the orientation change is maintained for at least a threshold time, the radar-based application 106 may determine that the orientation change is effective to cause a modification of the orientation-based functionality. The threshold time may be any suitable time (e.g., 0.5, 1.5, or 2.5 seconds) and may be predefined, user-selectable, or determined via a machine learning module included in or associated with the radar system 104 or the radar-based application 106.

[0052] In more detail, consider Figure 2 , which illustrates an example implementation 200 of a smartphone 102 (including a radar system 104 , a radar-based application 106 , and optionally a display 108 ) that may enable the smartphone, systems, and methods implemented in an electronic device. Figure 2 The smartphone 102 is illustrated as including other non-limiting example devices in which the smartphone, systems, and methods implemented in electronic devices may be implemented, including mobile phones 102-1, tablet computers 102-2, laptop computers 102-3, desktop computers 102-4, computing watches 102-5, computing glasses 102-6, gaming systems 102-7, home automation control systems 102-8, and vehicles 102-9. Other devices may also include televisions, entertainment systems, audio systems, drones, trackpads, drawing tablets, netbooks, e-readers, home security systems, and other home appliances. Note that electronic devices that may implement the described techniques may be wearable, non-wearable but mobile, or relatively non-mobile (e.g., desktops and appliances).

[0053] An exemplary overall lateral dimension of smartphone 102 may be, for example, approximately eight centimeters by approximately fifteen centimeters. An exemplary footprint of radar system 104 may be further limited, such as approximately four millimeters by six millimeters including the antenna. An exemplary power consumption of radar system 104 may be on the order of a few milliwatts (mW) to several mW (e.g., between approximately 2 mW and 20 mW). The requirement for such a limited footprint of radar system 104, which is needed to accommodate many other desirable features of smartphone 102 (e.g., camera, fingerprint sensor, display 108, etc.) in such a space-constrained package combined with power and processing constraints, may result in tradeoffs in accuracy and effectiveness of radar gesture detection, at least some of which may be overcome based on the teachings herein.

[0054] The smartphone 102 also includes one or more computer processors 202 and one or more computer-readable media 204, including memory media and storage media. Applications and / or operating systems (not shown) implemented as computer-readable instructions on the computer-readable media 204 can be executed by the computer processor 202 to provide some of the functions described herein. The smartphone 102 may also include a network interface 206. The smartphone 102 may use the network interface 206 to transmit data through a wired, wireless, or optical network. By way of example and not limitation, the network interface 206 may transmit data through a local area network (LAN), a wireless local area network (WLAN), a personal area network (PAN), a wide area network (WAN), an intranet, the Internet, a peer-to-peer network, a point-to-point network, or a mesh network.

[0055] Various implementations of the radar system 104 may include a system on a chip (SoC), one or more integrated circuits (ICs), a processor having embedded processor instructions or configured to access processor instructions stored in a memory, hardware having embedded firmware, a printed circuit board having various hardware components, or any combination thereof. The radar system 104 operates as a monostatic radar by sending and receiving its own radar signals. In some implementations, the radar system 104 may also cooperate with other radar systems 104 within the external environment to implement a bistatic radar, a multistatic radar, or a networked radar. As noted, the constraints or limitations of the smartphone 102 may affect the design of the radar system 104. For example, the smartphone 102 may have limited power available to operate the radar, limited computing power, size constraints, layout limitations, an external housing that attenuates or distorts the radar signal, and the like. The radar system 104 includes several features that enable advanced radar functionality and high performance despite these constraints, as described below with respect to Figure 3 further described. Note that in Figure 2, radar system 104 is illustrated as a part of smartphone 102. In other embodiments, radar system 104 may be separate from smartphone 102 or remote from smartphone 102.

[0056] These and other capabilities and configurations and Figure 1 The way entities act and interact will be described in more detail below. These entities can be further divided, merged, etc. Figure 1 Environment 100 and Figures 2 to 25 The detailed diagrams illustrate some of the many possible environments and devices in which the described techniques can be employed. Figures 3 to 6 Additional details and features of radar system 104 are described. Figures 3 to 6 , radar system 104 is described in the context of smartphone 102, but as noted above, the applicability of the features and advantages of the described systems and techniques is not necessarily limited thereto, and other embodiments involving other types of electronic devices may also be within the scope of the present teachings.

[0057] Figure 3 An example implementation 300 of a radar system 104 is illustrated, which may be used to enable implementation of a smartphone, system, and method in an electronic device. In example 300, the radar system 104 includes at least one of each of the following components: a communication interface 302, an antenna array 304, a transceiver 306, a processor 308, and a system medium 310 (e.g., one or more computer-readable storage media). The processor 308 may be implemented as a digital signal processor, a controller, an application processor, another processor (e.g., the computer processor 202 of the smartphone 102), or some combination thereof. The system medium 310, which may be included in or separate from the computer-readable medium 204 of the smartphone 102, includes one or more of the following modules: an attenuation mitigator 314, a digital beamformer 316, an angle estimator 318, or a power manager 320. These modules may compensate for or mitigate the effects of integrating the radar system 104 into the smartphone 102, thereby enabling the radar system 104 to recognize small or complex gestures, distinguish between different orientations of the user, continuously monitor the external environment, or achieve a target false alarm rate. With these features, the radar system 104 may be used in applications such as Figure 2 Radar system 104 is implemented in a variety of different devices as shown.

[0058] Using the communication interface 302, the radar system 104 can provide radar data to the radar-based application 106. The communication interface 302 can be a wireless interface or a wired interface, depending on whether the radar system 104 is implemented separately from the smartphone 102 or integrated therein. Depending on the application, the radar data can include raw or minimally processed data, in-phase and quadrature (I / Q) data, range-Doppler data, processed data including target location information (e.g., range, azimuth, elevation), cluttered map data, etc. In general, the radar data contains information that can be used by the radar-based application 106 to the smartphone, system, and method implemented in the electronic device.

[0059] The antenna array 304 includes at least one transmitting antenna unit (not shown) and at least two receiving antenna units (such as Figure 4 ). In some cases, antenna array 304 may include multiple transmit antenna elements to implement a multiple-input multiple-output (MIMO) radar capable of transmitting multiple different waveforms at a time (e.g., a different waveform for each transmit antenna element). The use of multiple waveforms may improve the measurement accuracy of radar system 104. For embodiments including three or more receive antenna elements, the receive antenna elements may be positioned in a one-dimensional shape (e.g., a line) or a two-dimensional shape. A one-dimensional shape enables radar system 104 to measure one angular dimension (e.g., azimuth or elevation), while a two-dimensional shape enables measurement of two angular dimensions (e.g., both azimuth and elevation). Reference Figure 4 An example two-dimensional arrangement of receive antenna elements is further described.

[0060] Figure 4 An example arrangement 400 of receiving antenna elements 402 is shown. For example, if the antenna array 304 includes at least four receiving antenna elements 402, the receiving antenna elements 402 may be arranged in a rectangular arrangement 404-1, such as Figure 4 Alternatively, if the antenna array 304 includes at least three receiving antenna elements 402, a triangular arrangement 404-2 or an L-shaped arrangement 404-3 may be used.

[0061] Due to the size or layout constraints of the smartphone 102, the element spacing between the receiving antenna elements 402 or the number of receiving antenna elements 402 may not be ideal for the angles that the radar system 104 is to monitor. In particular, the element spacing may cause the presence of angular ambiguity, which makes it challenging for conventional radars to estimate the angular position of a target. Therefore, conventional radars may limit the field of view (e.g., the angle to be monitored) to avoid ambiguous areas with angular ambiguity, thereby reducing false detections. For example, a conventional radar may limit the field of view to angles between approximately -45 degrees and 45 degrees to avoid angular ambiguity that occurs using a wavelength of 5 millimeters (mm) and an element spacing of 3.5 mm (e.g., an element spacing of 70% of the wavelength). Therefore, conventional radars may not be able to detect targets beyond the 45-degree field of view limit. In contrast, the radar system 104 includes a digital beamformer 316 and an angle estimator 318 that resolve the angular ambiguity and enable the radar system 104 to monitor angles beyond the 45-degree limit, such as angles between approximately -90 degrees and 90 degrees, or up to approximately -180 degrees and 180 degrees. These angular ranges may be applied in one or more directions (eg, azimuth and / or elevation). Thus, radar system 104 may achieve low false alarm rates for a variety of different antenna array designs, including element spacings less than, greater than, or equal to half the center wavelength of the radar signal.

[0062] Using antenna array 304, radar system 104 can form beams that are steered or unsteered, wide or narrow, or shaped (e.g., as a hemisphere, cube, sector, cone, or cylinder). As an example, one or more transmit antenna elements (not shown) may have an unsteered omnidirectional radiation pattern, or may be capable of producing a wide beam such as wide transmit beam 406. Any of these techniques enables radar system 104 to illuminate a large volume of space. However, in order to obtain target angular accuracy and angular resolution, receive antenna elements 402 and digital beamformer 316 may be used to generate thousands of narrow and steered beams (e.g., 2000 beams, 4000 beams, or 6000 beams), such as narrow receive beam 408. In this way, radar system 104 can effectively monitor the external environment and accurately determine the angle of arrival of reflections within the external environment.

[0063] return Figure 3, transceiver 306 includes circuits and logic for sending and receiving radar signals via antenna array 304. Components of transceiver 306 may include amplifiers, mixers, switches, analog-to-digital converters, filters, etc. to condition radar signals. Transceiver 306 may also include logic to perform in-phase / quadrature (I / Q) operations such as modulation or demodulation. Transceiver 306 may be configured for continuous wave radar operation or pulse radar operation. Various modulations may be used to generate radar signals, including linear frequency modulation, triangular frequency modulation, stepped frequency modulation, or phase modulation.

[0064] The transceiver 306 may generate a radar signal in a frequency range (e.g., spectrum) such as between 1 gigahertz (GHz) and 400 GHz, between 4 GHz and 100 GHz, or between 57 GHz and 63 GHz. The spectrum may be divided into a plurality of sub-spectra having similar bandwidths or different bandwidths. The bandwidth may be on the order of 500 megahertz (MHz), 1 GHz, 2 GHz, etc. As an example, different frequency sub-spectra may include frequencies between approximately 57 GHz and 59 GHz, between 59 GHz and 61 GHz, or between 61 GHz and 63 GHz. Multiple frequency sub-spectra having the same bandwidth and which may be continuous or non-continuous may also be selected for coherence. Multiple frequency sub-spectra may be transmitted simultaneously or separated in time using a single radar signal or multiple radar signals. Continuous frequency sub-spectra enable the radar signal to have a wider bandwidth, while non-continuous frequency sub-spectra may further emphasize amplitude and phase differences, which enables the angle estimator 318 to resolve angle ambiguity. The attenuation mitigator 314 or the angle estimator 318 may enable the transceiver 306 to utilize one or more frequency sub-spectra to improve the performance of the radar system 104, such as with respect to Figure 5 and Figure 6 Further described.

[0065] The power manager 320 enables the radar system 104 to save power either inside or outside the smartphone 102. For example, internally, the power manager 320 may enable the radar system 104 to collect data using a predetermined power mode or a specific duty cycle. Instead of operating in a low power mode or a high power mode, the power manager 320 dynamically switches between different power modes so that response latency and power consumption are managed together based on the activity within the environment. In general, the power manager 320 determines when and how to save power, and incrementally adjusts power consumption to enable the radar system 104 to operate within the power constraints of the smartphone 102. In some cases, the power manager 320 may monitor the amount of remaining available power and adjust the operation of the radar system 104 accordingly. For example, if the amount of remaining power is low, the power manager 320 may continue to operate in a low power mode instead of switching to a higher power mode.

[0066] For example, the low power mode may use a low duty cycle on the order of a few hertz (e.g., approximately 1 Hz or less than 5 Hz), which reduces power consumption to a few milliwatts (mW) (e.g., between approximately 2 mW and 5 mW). On the other hand, the high power mode may use a high duty cycle on the order of tens of hertz (Hz) (e.g., approximately 20 Hz or greater than 10 Hz), which causes radar system 104 to consume power on the order of several milliwatts (e.g., between approximately 8 mW and 20 mW). Although the low power mode may be used to monitor the external environment or detect an approaching user, if radar system 104 determines that the user is beginning to perform a gesture, power manager 320 may switch to the high power mode. Different triggers may cause power manager 320 to switch between different power modes. Exemplary triggers include motion or lack of motion, the appearance or disappearance of a user (e.g., presence or absence of a user), movement of a user into or out of a specified area (e.g., an area defined by distance, azimuth, or elevation), the distance of a user from the smartphone 102 such as the perceived distance 706 described above, a change in speed of motion associated with the user, or a change in reflected signal strength (e.g., due to a change in radar cross section). In general, triggers that indicate a low likelihood of user interaction with the smartphone 102 or a preference for using longer response delays to collect data may result in activation of a lower power mode to save power.

[0067] The power manager 320 may also save power by shutting down one or more components within the transceiver 306 (e.g., a voltage controlled oscillator, a multiplexer, an analog-to-digital converter, a phase-locked loop, or a crystal oscillator) during periods of inactivity. These periods of inactivity may occur if the radar system 104 is not actively transmitting or receiving radar signals, which may be on the order of microseconds (µs), milliseconds (ms), or seconds (s). For example, periods of inactivity may occur when a user is not present or when a user is present but is determined not to be paying attention to the smartphone 102 (e.g., based on one or more attention cues described herein). In addition, the power manager 320 may control the use of different hardware components within the radar system 104 to save power. For example, if the processor 308 includes a low-power processor and a high-power processor (e.g., a processor with different amounts of memory and computing power), the power manager 320 may switch between using the low-power processor for low-level analysis (e.g., detecting motion, determining a user's location, or monitoring the environment) and using the high-power processor for situations where the radar-based application 106 requests high-fidelity or accurate radar data (e.g., for gesture recognition or user orientation).

[0068] In addition to the internal power saving techniques described above, the power manager 320 can also save power by activating or deactivating other external components or sensors within the smartphone 102. These external components can include speakers, camera sensors, global positioning systems, wireless communication transceivers, displays, gyroscopes, or accelerometers. Because the radar system 104 can use a small amount of power to monitor the environment, the power manager 320 can appropriately turn these external components on or off based on where the user is or what the user is doing. In this way, the smartphone 102 can seamlessly respond to the user and save power without using an automatic shutoff timer or the user physically touching or verbally controlling the smartphone 102.

[0069] Figure 5 Additional details of an example implementation 500 of a radar system 104 within a smartphone 102 are illustrated. In example 500, an antenna array 304 is located below an external housing (such as a glass cover or housing) of the smartphone 102. Depending on its material properties, the external housing can act as an attenuator 502 that attenuates or distorts radar signals sent and received by the radar system 104. The attenuator 502 can include different types of glass or plastic, some of which can be found within a display screen, external housing, or other components of the smartphone 102 and have a dielectric constant (e.g., relative permittivity) between approximately four and ten. Thus, the attenuator 502 is opaque or translucent to the radar signal 506 and can cause a portion of the transmitted or received radar signal 506 to be reflected (as shown by reflected portion 504). For conventional radars, the attenuator 502 can reduce the effective range that can be monitored, prevent detection of small targets, or reduce overall accuracy.

[0070] Assuming that the transmit power of the radar system 104 is limited and redesigning the external housing is undesirable, one or more attenuation-related characteristics of the radar signal 506 (e.g., the frequency sub-spectrum 508 or the steering angle 510) or the attenuation-related characteristics of the attenuator 502 (e.g., the distance 512 between the attenuator 502 and the radar system 104 or the thickness 514 of the attenuator 502) are adjusted to mitigate the effects of the attenuator 502. Some of these characteristics may be set during manufacturing or adjusted by the attenuation mitigator 314 during operation of the radar system 104. For example, the attenuation mitigator 314 may cause the transceiver 306 to transmit the radar signal 506 using the selected frequency sub-spectrum 508 or steering angle 510, cause the platform to move the radar system 104 closer to or further away from the attenuator 502 to change the distance 512, or prompt the user to apply another attenuator to increase the thickness 514 of the attenuator 502.

[0071] Appropriate adjustments may be made by the attenuation mitigator 314 based on predetermined characteristics of the attenuator 502 (e.g., characteristics stored in the computer-readable medium 204 of the smartphone 102 or within the system medium 310) or by processing the return of the radar signal 506 to measure one or more characteristics of the radar signal 506. Even if certain attenuation-related characteristics are fixed or constrained, the attenuation mitigator 314 may still take these constraints into account to balance each parameter and achieve the target radar performance. As a result, the attenuation mitigator 314 enables the radar system 104 to achieve enhanced accuracy and a greater effective range to detect and track users located on opposite sides of the attenuator 502. These techniques provide an alternative to increasing the transmit power (which increases the transmit power of the radar system 104) or changing the material properties of the attenuator 502 (which may be difficult and expensive once the device is in production).

[0072] Figure 6 An example scheme 600 implemented by the radar system 104 is illustrated. Portions of the scheme 600 may be performed by the processor 308, the computer processor 202, or other hardware circuits. The scheme 600 may be customized to support different types of smartphones 102 and radar-based applications 106, and also enable the radar system 104 to achieve target angle accuracy despite design limitations.

[0073] The transceiver 306 generates raw data 602 based on the individual responses of the receive antenna units 402 to the received radar signals. The received radar signals may be associated with one or more frequency sub-spectra 604 selected by the angle estimator 318 to facilitate angular ambiguity resolution. For example, the frequency sub-spectra 604 may be selected to reduce the number of side lobes or reduce the amplitude of the side lobes (e.g., reduce the amplitude by 0.5 dB, 1 dB, or more). The number of frequency sub-spectra may be determined based on the target angle accuracy or the computational limitations of the radar system 104.

[0074] The raw data 602 includes digital information (e.g., in-phase and quadrature data) for time periods, different wave numbers, and multiple channels associated with the receiving antenna unit 402, respectively. A fast Fourier transform (FFT) 606 is performed on the raw data 602 to generate pre-processed data 608. The pre-processed data 608 includes digital information across time periods for different distances (e.g., range grids) and for multiple channels. A Doppler filtering process 610 is performed on the pre-processed data 608 to generate range-Doppler data 612. The Doppler filtering process 610 may include another FFT that generates amplitude and phase information for multiple range grids, multiple Doppler frequencies, and for multiple channels. The digital beamformer 316 generates beamforming data 614 based on the range-Doppler data 612. The beamforming data 614 includes digital information for a set of azimuth and / or elevation angles, which represents the field of view for which different steering angles or beams are formed by the digital beamformer 316. Although not depicted, the digital beamformer 316 may alternatively generate beamformed data 614 based on the preprocessed data 608, and the Doppler filtering process 610 may generate range-Doppler data 612 based on the beamformed data 614. To reduce the amount of computation, the digital beamformer 316 may process a portion of the range-Doppler data 612 or the preprocessed data 608 based on a range, time, or Doppler frequency interval of interest.

[0075] The digital beamformer 316 may be implemented using a single-look beamformer 616, a multi-look interferometer 618, or a multi-look beamformer 620. Typically, a single-look beamformer 616 may be used for deterministic objects (e.g., point source targets with a single phase center). For non-deterministic targets (e.g., targets with multiple phase centers), a multi-look interferometer 618 or a multi-look beamformer 620 is used to improve accuracy relative to the single-look beamformer 616. A person is an example of a non-deterministic target and has multiple phase centers 622 that may vary based on different line-of-sight angles (as shown at 624-1 and 624-2) (e.g., due to changes in the orientation of the user relative to the smartphone 102, such as turning down or away, leaning forward, or moving closer or farther). The changes in constructive or destructive interference generated by the multiple phase centers 622 may make it challenging for conventional radars to accurately determine angular position. However, the multi-look interferometer 618 or multi-look beamformer 620 performs coherent averaging to increase the accuracy of the beamformed data 614. The multi-look interferometer 618 coherently averages the two channels to generate phase information that can be used to accurately determine the angle information. On the other hand, the multi-look beamformer 620 can coherently average two or more channels using a linear or nonlinear beamformer such as Fourier, Capon, Multiple Signal Classification (MUSIC), or Minimum Variance Distortionless Response (MVDR). The increased accuracy provided via the multi-look beamformer 620 or multi-look interferometer 618 enables the radar system 104 to recognize small gestures or distinguish between multiple parts of a user.

[0076] Angle estimator 318 analyzes beamforming data 614 to estimate one or more angular positions. Angle estimator 318 may utilize signal processing techniques, pattern matching techniques, or machine learning. Angle estimator 318 also resolves angle ambiguities that may be caused by the design of radar system 104 or the field of view monitored by radar system 104. Example angle ambiguities are shown in amplitude graph 626 (e.g., amplitude response).

[0077] The magnitude graph 626 depicts the magnitude differences that may occur for different angular positions of the target and for different steering angles 510. A first magnitude response 628-1 (shown with a solid line) is shown for a target at a first angular position 630-1. Similarly, a second magnitude response 628-2 (shown with a dashed line) is shown for a target at a second angular position 630-2. In this example, differences are considered at angles between -180 degrees and 180 degrees.

[0078] As shown in the amplitude graph 626, there is an ambiguous region for two angular positions 630-1 and 630-2. The first amplitude response 628-1 has the highest peak at the first angular position 630-1 and a smaller peak at the second angular position 630-2. Although the highest peak corresponds to the actual position of the target, the smaller peak causes the first angular position 630-1 to be ambiguous because it is within a certain threshold and the conventional radar may not be able to confidently determine whether the target is at the first angular position 630-1 or the second angular position 630-2. In contrast, the second amplitude response 628-2 has a smaller peak at the second angular position 630-2 and a larger peak at the first angular position 630-1. In this case, the smaller peak corresponds to the position of the target.

[0079] While conventional radars may be limited to using the highest peak amplitude to determine angular position, angle estimator 318 instead analyzes subtle differences in the shapes of amplitude responses 628-1 and 628-2. Characteristics of the shapes may include, for example, roll-off, peak or null width, angular position of peaks or nulls, height or depth of peaks and nulls, sidelobe shapes, symmetry in amplitude response 628-1 or 628-2, or lack of symmetry within amplitude response 628-1 or 628-2. Similar shape characteristics may be analyzed in the phase response, which may provide additional information to resolve angle ambiguity. Thus, angle estimator 318 maps a unique angle signature or pattern to an angular position.

[0080] The angle estimator 318 may include an algorithm or toolkit that may be selected based on the type of smartphone 102 (e.g., computing power or power limitations) or the target angle resolution of the radar-based application 106. In some embodiments, the angle estimator 318 may include a neural network 632, a convolutional neural network (CNN) 634, or a long short-term memory (LSTM) network 636. The neural network 632 may have various depths or numbers of hidden layers (e.g., three hidden layers, five hidden layers, or ten hidden layers), and may also include different numbers of connections (e.g., the neural network 632 may include a fully connected neural network or a partially connected neural network). In some cases, the CNN 634 may be used to increase the computational speed of the angle estimator 318. The LSTM network 636 may be used to enable the angle estimator 318 to track the target. Using machine learning techniques, the angle estimator 318 employs a nonlinear function to analyze the shape of the amplitude response 628-1 or 628-2 and generates angle probability data 638, which indicates the likelihood that the user or a portion of the user is within the angle grid. Angle estimator 318 may provide angle probability data 638 for a few angle grids (e.g., two angle grids) to provide the probability that the target is located to the left or right of smartphone 102, or provide angle probability data 638 for thousands of angle grids (e.g., to provide angle probability data 638 for continuous angle measurements).

[0081] Based on the angle probability data 638, the tracker module 640 generates angular position data 642, which identifies the angular position of the target. The tracker module 640 can determine the angular position of the target based on the angle grid with the highest probability in the angle probability data 638 or based on prediction information (e.g., previously measured angular position information). The tracker module 640 can also track one or more moving targets so that the radar system 104 can distinguish or identify the targets with confidence. Other data can also be used to determine the angular position, including range, Doppler, velocity, or acceleration. In some cases, the tracker module 640 may include an alpha-beta tracker, a Kalman filter, a multiple hypothesis tracker (MHT), and the like.

[0082] The quantizer module 644 takes the angular position data 642 and quantizes the data to produce quantized angular position data 646. The quantization may be performed based on the target angular resolution for the radar-based application 106. In some cases, fewer quantization levels may be used so that the quantized angular position data 646 indicates whether the target is on the right or left side of the smartphone 102, or identifies that the target is located within a 90-degree quadrant. For some radar-based applications 106, such as user proximity detection, this may be sufficient. In other cases, a large number of quantization levels may be used so that the quantized angular position data 646 indicates the angular position of the target within an accuracy of a fraction of a degree, one degree, five degrees, etc. This resolution may be used for higher resolution radar-based applications 106, such as gesture recognition. In some embodiments, the digital beamformer 316, the angle estimator 318, the tracker module 640, and the quantizer module 644 are implemented together in a single machine learning module.

[0083] These and other capabilities and configurations and Figures 1 to 6 The entity actions and interactions are described below. The described entities can be further divided, combined, used with other sensors or components, etc. In this way, different embodiments of smartphones 102 with different configurations of radar system 104 and non-radar sensors can be used to implement smartphones, systems, and methods implemented in electronic devices. Figure 1 An example operating environment 100 and Figures 2 to 6 The detailed illustrations illustrate only some of the many possible environments and devices in which the described techniques can be employed.

[0084] Example System

[0085] As described above, the techniques and systems described herein may also enable smartphone 102 to provide functionality based on the user's presence and receive 3D gestures from the user to interact with the device. Figure 7Another example environment 700 is illustrated in which techniques enabling smartphones, systems, and methods implemented in electronic devices may be implemented. The example operating environment 700 includes a smartphone 102 that includes or is associated with a radar system 104 and a radar-based application 702. In some implementations, the smartphone 102 may include a display, such as the display 108. Although the example operating environment 700 is illustrated in the context of a smartphone 102, other electronic devices (such as those described above with reference to FIG. 1 ) may also be used. Figure 1 and 2 described) can also be used to implement reference Figures 7 to 20 In the example environment 700, the radar system 104 provides the radar field 110, as described above with reference to Figures 1 to 6 Radar system 104 also enables smartphone 102 to sense and analyze reflections from objects 704 in radar field 110. Figures 7 to 20 In the described example, smartphone 102 is smartphone 102, although the described features and techniques may be used with any of a variety of electronic devices (e.g., as described in reference Figure 2 described).

[0086] like Figure 7 As shown, object 704 is a person or user of smartphone 102 (person 704 or user 704), but object 704 can be any of a variety of objects (such as wood, plastic, metal, fabric, or organic materials (e.g., user 704)) from which radar system 104 can sense and analyze reflections. Based on the analysis of the reflections, radar system 104 can provide radar data, as described above with reference to Figures 1 to 6 As described, the radar data may be communicated to other entities, such as radar-based applications 702 .

[0087] Radar-based application 702 may be any of a variety of radar-based applications that can use radar data to determine the presence of user 704 within a sensing distance 706 of smartphone 102. The sensing distance may be any suitable distance, such as three feet, seven feet, ten feet, or fourteen feet (or one, two, three, or four meters), and consistent with the range of radar zone 110, such as Figure 7 700-1. In other cases, the sensing distance 706 may be less than the maximum range of the radar region 110. The sensing distance 706 may be predefined, user selectable, or determined by a machine learning module included in or associated with the radar system 104 or the radar-based application 702.

[0088] Based on the presence of user 704 within sensing distance 706 of smartphone 102, radar-based application 702 may provide presence-based features or functionality to smartphone 102 (e.g., via display 108). For example, Figure 7 As shown in detail view 700-2 of FIG. 1 , user 704 has moved toward smartphone 102 (indicated by arrow 710) and is within sensing distance 706. In response, smartphone 102 provides presence-based functionality, such as an electronic assistant including a voice interface (shown as text box (“Hello”) 712). Additionally, radar-based application 702 may use radar data to determine that user 704 is outside sensing distance 706 and, based on a change in the presence of user 704 within sensing distance 706, modify or cease providing presence-based features or functionality. In this manner, the presence of user 704 within sensing distance 706 may be used to provide input or instructions to interact with smartphone 102.

[0089] The radar-based application 702 may include or communicate with a presence module 708, which may store both information related to determining the presence of the user 704 within the perceived distance 706 (based on radar data) and information related to features and functions based on the presence of the user 704 within the perceived distance 706 in internal or external memory. In some embodiments, the radar-based application 702 may use the information stored by the presence module 708 to determine the presence or absence of the user 704 within the perceived distance 706. In other embodiments, the presence module 708 itself may determine the presence or absence of the user 704 within the perceived distance 706. Figure 7 As shown, the presence module 708 is included in the radar-based application 702, but the presence module 708 can be a separate entity that is part of or separate from the radar-based application 702 or the smartphone 102. In some embodiments of any of the examples 100 or 700, the presence module 708 and ( Figure 1 The) orientation module 114 can be a single module that can perform the operations described with reference to both the presence module 708 and the orientation module 114 or in combination with any of the radar-based applications 106 or 702.

[0090] The radar-based application 106 may also include or communicate with a 3D gesture module (e.g., 3D gesture module 116), which may store both information related to determining a 3D gesture (e.g., a 3D gesture of the user 704) based on radar data and information related to an action corresponding to the 3D gesture. Thus, the radar-based application 702 may detect the 3D gesture of the user 704 and determine an action corresponding to the 3D gesture. The 3D gesture may be any of a variety of gestures, including a scroll gesture made by moving a hand on the smartphone 102 in a horizontal direction (e.g., from the left side of the smartphone 102 to the right side of the smartphone 102), a waving gesture made by rotating the user's arm around the elbow, and a push gesture made by moving the user's hand on the smartphone 102 in a vertical direction (e.g., from the bottom side of the smartphone 102 to the top side of the smartphone 102). Other types of 3D gestures or actions may also be made, such as a reaching gesture made by moving the user's hand toward the smartphone 102, a doorknob turning gesture made by bending the fingers of the user's hand to grasp an imaginary doorknob and rotating it in a clockwise or counterclockwise manner to simulate turning an imaginary doorknob, and a spindle twisting gesture made by rubbing the thumb and at least one other finger together. Each of these example gesture types can be detected by the radar system 104. When each of these gestures is detected, the smartphone 102 can perform an action, such as displaying new content, moving a cursor, activating one or more sensors, opening an application, controlling a game, pinning content to the screen, muting an alarm, controlling a user interface, or manipulating an AR element. In this way, the radar system 104 provides touchless control of the smartphone 102.

[0091] As described with reference to detailed view 700-2, when user 704 is within perception distance 706, the presence-based functionality may be for providing an electronic assistant or voice interface feature. In other embodiments, the presence-based functionality may be another functionality, such as with reference to Figures 8 to 12 as described.

[0092] Figure 8An example 800 is illustrated in which the presence-based function is an adjustment of the volume of a ringtone, alarm, or notification. In detailed view 800-1, user 802 is relatively far from smartphone 102 and outside of perceived distance 706 (indicated by dashed ellipse 706), and the ringtone volume is one hundred percent of the default level set by user 802, as shown by ringtone icon 804. In contrast, in detailed view 800-2, user 806 is relatively close to smartphone 102 and within perceived distance 706 (again indicated by dashed ellipse 706), and the volume is reduced, as shown by smaller ringtone icon 808. The volume can be reduced by any suitable amount (e.g., 25%, 50%, or 75%), and the reduction can be based on any of a variety of factors, such as the distance between user 804 and smartphone 102 or a pre-set or user-selected amount.

[0093] Fig. 9 An example 900 is illustrated in which the presence-based functionality is an adjustment of the communication mode used by an electronic assistant included with a smartphone 102. The electronic assistant can communicate in various modes to provide text, audio, video, etc. (e.g., on the display 108). In the detailed view 900-1, the user 902 is relatively far away from the smartphone 102 and outside the perceived distance 706 (represented by the dashed oval 706), and the electronic assistant interacts in a voice mode without turning on the display 108 (as shown by the ringtone icon 904 of the smartphone 102 and the dimmed display 102). Next, the detailed view 900-2 shows the user 902 relatively close to the smartphone 102 and within the perceived distance 706 (again represented by the dashed oval 706), and the electronic assistant supplements the audio communication mode of the detailed view 900-1 (ringtone icon 904) with glanceable display information 906 (e.g., information that can be read or understood at a glance, such as large text, symbols, or animations) as shown on the display 108. In this manner, the described techniques may reduce power consumption of smartphone 102 by limiting usage of display 108 to levels appropriate for the presence of user 902 .

[0094] In some embodiments, the adjustment of the communication mode may also include an adjustment of the content presented on the display 108. For example, in the detailed view 900-3, the user 902 picks up the smartphone 102. As previously described, the radar-based application 702 is capable of determining 3D gestures and corresponding actions (e.g., using the 3D gesture module 116). Therefore, when the user 902 reaches out to the smartphone 102, the radar-based application 702 can determine that the reaching out is a 3D gesture, and the corresponding action of the electronic assistant is to enable the user 902 to enter text and provide richer visual information 908 (e.g., more and smaller text, embedded links, and videos), as shown on the display 108.

[0095] Fig.10 An example 1000 is illustrated in which the storage-based functionality is to provide notification of the presence of an object within a sensing distance. As described above, the smartphone 102 (e.g., via the radar system 104) can detect whether a user is within the sensing distance of the smartphone 102 and whether the user is alone or with others. Using this information, the smartphone 102 (e.g., via the radar-based application 702) can provide notifications about the status of the user of the smartphone 102 (e.g., the user's "spatial relationship context", such as whether the user is available (alone), with people, away, etc.). The notification can be provided to one or more contacts stored on or accessible by the smartphone 102, and can be provided in any of a variety of ways, such as through a text message or status indicator displayed when another party views the user's contact information.

[0096] For example, in detailed view 1000-1, user 1002, whose phone includes a presence-based feature, views a contact list 1004 (indicated by dashed brackets) that includes some contacts with phones (or other electronic devices) that also include a presence-based feature. In detailed view 1000-1, the top four contacts are those that provide notifications of the presence of users associated with the contacts (e.g., George is away, Susie and Mary are with people, and Ingrid is available), and the bottom two contacts do not provide notifications (e.g., because the contact has turned off the presence-based feature, the contact's phone does not support the presence-based feature, or for other reasons). In detailed view 1000-2, user 1002 selects Susie's contact icon, and the presence-based feature provides an option 1006 to contact Susie. As described above, Suzy's notification indicates that someone else is with Suzy, so options 1006 may include sending a message (MESSAGE), calling now (CALL), or calling when Suzy is no longer with the other person (CALL WHEN ALONE).

[0097] In detailed view 1000-3, assume that user 1002 selects option 1006 to call when Suzy is no longer with the other person. By using option 1006, when the time is right to contact, both user 1002 and Suzy (as shown by another user 1008) receive phone calls simultaneously, as indicated by vibration indicators around smartphone 102 and Suzy's phone. In some embodiments, how many contacts receive notifications and which contacts receive notifications can be selected by the user or based on default settings. In addition, option 1006 can be selected by user 1008 (e.g., Suzy) so that user 1008 can determine what options the caller (e.g., user 1002) has when being provided with notifications.

[0098] Presence-based functionality that provides notifications of the presence of objects within a perceived distance can also enhance remote collaboration tools by broadcasting presence information or sending it to specific contacts. For example, in a document sharing application, a notification can help a remote user determine whether another user is actually in front of the computer or just has the document sharing application open. In some cases, radar-based applications 702 can provide additional details in notifications. For example, by using additional sensors or features (e.g., geolocation features or hands-free driving features), notifications can include information that allows the user to understand that the potential recipient of the call is in a car, driving (or not driving), and with other people. Radar-based applications 702 can also suggest other communication methods when the notification indicates that the potential recipient is with other people, such as by suggesting email as a way to avoid distracting the attention of people who are with other people (e.g., people who may be in a meeting).

[0099] Fig.11An example 1100 is illustrated in which the presence-based function is to provide a notification of a previously presented reminder. In detailed view 1100-1, user 1102 sets a reminder using a voice command (shown as text box 1104) through an electronic assistant. In detailed view 1100-2, assume that user 1102 is doing housework at a location outside of perceived distance 706. During the time that user 1102 is outside of perceived distance 706, the time for the reminder passes, and the reminder is presented when user 1102 leaves. In some embodiments, because smartphone 102 knows that the user is away, the reminder may not be presented at all, or the reminder may be presented at a lower volume or in a different mode (e.g., text instead of voice). In detailed view 1100-3, assume that user 1102 returns 30 minutes after the reminder is thought to have been presented. When smartphone 102 determines that user 1102 is within perceived distance 706, a reminder (shown as text box 1106) is presented. In some embodiments, the reminder may include additional information that can help user 1102. For example, suggestions to run your laundry through the dryer for a few minutes to reduce wrinkles, or suggestions to change a restaurant reservation.

[0100] Fig.12 An example 1200 is illustrated in which the presence-based functionality is to provide an enhanced media experience. For example, wallpapers, maps, and other digital content can include animation and depth. In detailed view 1200-1, user 1202 is reading a book and then reaches for smartphone 102. Display 108 automatically turns on and presents a short wallpaper animation 1204. In detailed view 1200-1, wallpaper animation 1204 is a clock face that shows how long user 1202 has been reading, but other wallpaper animations 1204 (favorite movie clips, user-recorded videos, etc.) can also be presented. Wallpaper animation 1204 can be any appropriate length (e.g., 1 second, 2 seconds, or 3 seconds) and can be user-selected or selected by radar-based application 702. In some embodiments, when user 1202 withdraws his extended hand, wallpaper animation 1204 is displayed in reverse (e.g., the clock face is reversed to show the time when user 1202 began reading). Radar-based application 702 can present other content based on the user's presence. For example, in detailed view 1200 - 2 , user 1206 is within perceived distance 706 , and in response, radar-based application 702 presents favorite image 1208 .

[0101] Further, by using digital beamforming and other techniques, such as reference Figures 3 to 6 As described, radar system 104 can determine the user's viewing angle. Radar-based application 702 can then dynamically adjust the image presented on display 108 to accommodate the user's viewpoint, thereby creating the illusion of three-dimensional space on display 108. For example ( Figure 2 ), assume that a user's smartphone is placed on a table and a three-dimensional (3D) object (map, sculpture, building, etc.) is presented on the display. As the user moves around the table, the presentation of the 3D object changes to accommodate the user's viewpoint, enriching the three-dimensional illusion by giving the user the appearance of moving around the real object. In some embodiments, the user can rotate the device, rather than moving around a fixed device, and view the 3D object from different angles.

[0102] In other embodiments, smartphone 102 may use, for example, radar-based application 702 or another entity to determine that more than one object is present while object 704 is within the sensing distance (e.g., determining the presence of another object within sensing distance 706). In response, radar-based application 702 may modify the presence-based functionality of smartphone 102. Several examples of such embodiments are described below with reference to Fig.13 and Fig.14 Give a description.

[0103] Fig.13 An example 1300 is illustrated in which the modification to the presence-based functionality is an adjustment to the notification mode used by the smartphone 102 based on the presence of another object within the perception distance 706. In the detailed view 1300-1, the user 1302 and the other user 1304 are sitting together at a table. Because the radar-based application 702 can determine that the user 1302 and the other user 1304 are both within the perception distance 706, the radar-based application 702 modifies the notification mode of the smartphone 102. In some embodiments, the notification can be completely silent (e.g., nothing is presented on the display 108). In other embodiments, as shown in the detailed view 1300-1, the notification mode can be modified so that only the notification indicator 1306 is displayed, rather than the full notification. For example, the icon indicates that the notification (e.g., a notification of a calendar event, a task, or other user-defined notification) is available for viewing, rather than indicating the notification itself. In this way, the smartphone 102 is less intrusive in business or social situations, and potentially private information that may be displayed in the full notification is not publicly displayed.

[0104] Smartphone 102 may return to a default notification mode, or to a notification mode that was in operation before another user 1304 entered sensing distance 706, in a variety of ways. For example, as shown in detailed view 1300-2, user 1302 may reach toward smartphone 102 to return to a previous notification mode (as shown by arrow 1308). After returning to the previous notification mode, content 1310 of the received notification may be presented by smartphone 102. In other implementations, a different 3D gesture may be used to return to a previous notification mode, or the smartphone may automatically return to a previous notification mode when it is determined that other user 1304 is no longer within sensing distance 706.

[0105] Fig.14 An example 1400 is illustrated in which a modification to presence-based functionality is an adjustment to an electronic assistant mode used by smartphone 102 based on the presence of another object within perception distance 706. For example, the electronic assistant may provide reminders and notifications that include potentially private information (e.g., content of personal reminders or work-related events) when the user is alone, but limit content when others are present.

[0106] Consider detailed view 1400-1, in which user 1402 is alone and within sensing distance 706. Because user 1402 is alone, the electronic assistant can use the voice interface to provide user 1402 with an audio reminder to take medication (illustrated by text box 1404, which indicates that another audio reminder will be provided in 10 minutes). Assume that a few minutes later (less than 10 minutes later), another user 1406 drops by, as shown in another detailed view 1400-2. The other user 1406 sits down within sensing distance 706, and the radar-based application 702 modifies the electronic assistant mode so that no audio is provided (as shown by mute icon 1408). In addition to audio, notifications can be muted until the other user 1406 leaves, or reminders can be provided in another mode that provides more privacy (such as, vibration mode, text mode, or notification indicator mode), as shown in reference to FIG. Figure 3 Described.

[0107] In another example (not shown), user 1402 may be viewing or listening to content when another user 1406 enters sensing distance 706. Even if user 1402 does not notice that another user 1406 has come within sensing distance 706, smartphone 102 (via a radar-based application or another entity, such as a voice assistant) may recognize the presence of the other user and temporarily pause or hide the content (e.g., in the case of private or confidential content).

[0108] In other embodiments including any of the above examples, the smartphone 102 (using the radar-based application 702 or another entity) can determine whether the user is alone in conjunction with other sensors. For example, the smartphone 102 can use a microphone to determine the ambient noise level around the user. Based on the presence and ambient noise level, the smartphone 102 can then adjust the volume of the phone ringtone, notification, and voice assistant to an appropriate level to attract the user's attention. In addition, the radar-based application 702 can communicate with other sensors (e.g., a microphone) so that when the voice of another user is detected (even if the other person is outside the perception distance 706), the modification of the presence-based features is still performed. Further, by using the above-mentioned 3D gesture module 116, when the user reaches out toward the smartphone 102, the smartphone 102 can make the ringtone, notification, and voice assistant quieter (or completely silent) because the smartphone 102 can determine that the user's reaching out is a 3D gesture indicating that the user will respond.

[0109] In other embodiments, the smartphone 102 may determine (using radar data) a 3D gesture of an object within a sensed distance and perform an action corresponding to the 3D gesture. The determination of the 3D gesture may be performed by various entities (such as the radar-based application 702 or the 3D gesture module 116) or a combination of these or other entities. Figures 15 to 20 Several examples of such implementations are described.

[0110] Fig.15 Example 1500 is illustrated where the 3D gesture is a hand wave on the display of smartphone 102 that activates pinning mode, which locks the content on the screen and blocks incoming notifications until the user exits pinning mode. Sometimes, a user may want to limit what information can be revealed to another person even when the other person is within sensing distance 706. The techniques described in example 1500 allow a user to share screen content with another person without exposing other content and details of alerts and notifications.

[0111] In detailed view 1500-1, assume that user 1502 wants to hand smartphone 102 to another user 1504 to share photos. User 1502 wants to protect personal information (such as other photos or content in reminders and notifications) from being accidentally disclosed to other user 1504. User 1502 performs a right-to-left hand swipe, as indicated by arrow 1506, and radar-based application 702 or another entity places smartphone 102 in pinning mode. With increased confidence that personal information will not be disclosed to other user 1504, user 1502 can then hand smartphone 102 to other user 1504, as shown in another detailed view 1500-2.

[0112] In another embodiment (not shown), the radar-based application 702 can automatically place the smartphone 102 in pinning mode by using radar data to determine that the device is being handed to another person. As described above, the radar system 104 can enable the radar-based application 702 to determine that two users are present. By determining the relative distance between the first user and the device and the orientation of the device relative to the two users, the pinning mode can be automatically activated without the need for an intentional gesture. As described above, these techniques provide a way for users to protect personal information using simple and discreet 3D gestures while still allowing others to view selected content.

[0113] Fig.16 An example 1600 is illustrated in which the 3D gesture is reaching toward the smartphone 102 to silence the ringtone. As described above, the radar-based application 702 can adjust the ringtone volume based on the distance between the user and the device. Sometimes, when the user is within the perception distance 706, the user may want to silence the ringtone before answering the call (or not answer the call when determining who the caller is). In the detailed view 1600-1, the user 1602 is sitting next to the smartphone 102. The smartphone 102 is ringing, as shown by the ringtone volume icon 1604. Assume that the user 1602 wants to silence the ringtone before deciding whether to answer. As shown in the detailed view 1600-2, the user 1602 reaches toward the smartphone 102, as shown by the arrow 1606. The radar-based application 702 determines the reaching as a 3D gesture for the corresponding action of silencing the ringtone (shown by the mute icon 1608). In this way, the technology described in example 1600 allows a user to silence a ringing sound by reaching for the phone without having to answer the call or interact with the device controls.

[0114] Fig.17 An example 1700 is illustrated in which the 3D gesture is reaching toward the smartphone 102 to exit full screen mode and access controls on the user interface. As described above, the radar-based application 702 can use the sensing distance 706 and the attention prompt to keep the application in a specific mode (e.g., full screen mode without visible controls) even if the user is not interacting with the device. In some cases, when the user is within the sensing distance 706, the user may want to interact with the controls of the application without picking up the device or clicking on a menu.

[0115] In detailed view 1700-1, user 1702 is sitting next to smartphone 102. In this example, assume that user 1702 is watching a video and wants to pause the video. As shown in detailed view 1700-2, user 1702 reaches out toward smartphone 102, as shown by arrow 1704. Radar-based application 702 determines the reach as a 3D gesture that exits full-screen mode and provides a corresponding action of a user interface or other control (shown by pause icon 1706). In some embodiments, if user 1702 ends the 3D gesture (e.g., stops reaching out and retracts his hand), the full-screen mode resumes. As shown and described in example 1700, these technologies allow the user to exit full-screen mode and interact with the user interface by reaching for the phone without having to click or activate the user interface.

[0116] Fig.18 An example 1800 is illustrated in which the 3D gesture is reaching toward the smartphone 102 to access the camera controls after the photo is framed. As described above, the radar-based application 702 can use the perception distance 706 (not shown) and attention prompts to keep the application in a specific mode (e.g., a full-screen mode with no visible controls). In some cases, many camera controls are unnecessary and only create obstructions and clutter on the display. Therefore, a user within the perception distance 706 may want a full-screen environment without controls for a period of time, such as when framing a photo, and then interact with the full controls after the framing is completed.

[0117] In detail view 1800-1, user 1802 is framing a photo of a sunset using the camera on smartphone 102. In this example, assume that user 1802 is trying to frame a photo to include a particular landscape feature and the sunset, and wants to frame the photo in full screen mode with no controls (or minimal controls, such as shutter button 1804). As shown in detail view 1800-1, user 1802 uses one hand to frame the photo so that more of the display is visible. In detail view 1800-2, user 1802 reaches toward smartphone 102, as shown by arrow 1806. Radar-based application 702 determines the reach as a 3D gesture that provides a corresponding action of a user interface or other control (such as crop window 1808). In some embodiments, if user 1802 ends the 3D gesture (e.g., stops reaching and retracts the hand), the full screen mode resumes. In other implementations, the radar-based application 702 can determine that the user 1802 is only reaching to click the shutter button 1804 (e.g., by determining where the user 1802 is reaching), and then present only the shutter button 1804, without all of the additional controls. As shown and described in example 1800, these techniques allow the user to enjoy full screen mode to frame a photo and interact with the user interface by reaching for the phone without having to click or activate the user interface.

[0118] Fig.19 An example 1900 is illustrated in which the 3D gesture is reaching toward the smartphone 102 to activate a backlight or other light source, making it easier to locate the smartphone 102 in low-light environments. As described above, the radar-based application 702 can use the perception distance 706 (not shown) and attention prompts to keep the application in a specific mode (e.g., a power-saving dark screen mode). In some cases, the user may even want the screen to be completely turned off, such as in an intentionally dark environment. Therefore, a user within the perception distance 706 may turn off the screen of the smartphone 102 or put it in a dim mode when sleeping or watching a movie, and then need a small amount of light to help locate the smartphone 102 in a dark environment.

[0119] In detailed view 1900-1, user 1902 is getting ready to go to bed. In this example, assume that user 1902 forgot to set the alarm and wants to find smartphone 102 and set the alarm. As shown in detailed view 1900-1, user 1902 reaches toward smartphone 102 (shown by arrow 1904). The radar-based application determines the reach as a 3D gesture that provides a corresponding action of dim light (e.g., backlight) (shown by shaded area 1906). In detailed view 1900-2, assume that user 1902 has dropped smartphone 102 on the floor of a theater and cannot see it in the dark to retrieve it. User 1902 reaches toward smartphone 102, as shown by another arrow 1908. The radar-based application 702 again determines the reach as a 3D gesture that provides a corresponding action of light (shown by another shaded area 1910).

[0120] Similarly, in detailed view 1900-3, assume that user 1902 has placed smartphone 102 in a bag. User 1902 reaches toward smartphone 102, as indicated by another arrow 1912. Radar-based application 702 again determines the reach as a 3D gesture that provides a corresponding action of light (indicated by another shaded area 1914). In some embodiments, if the user ends the 3D gesture (e.g., stops reaching and retracts the hand), the dark screen mode is restored. As shown and described in example 1900, these techniques allow the user to use the dark screen mode by reaching toward the phone to save power or reduce distraction caused by the bright screen in a dark environment, and to locate the phone. In other embodiments (not shown), radar-based application 702 determines the reach as a 3D gesture that turns on a vibration mode, and the vibration mode becomes stronger (e.g., vibrates with increasing frequency, amplitude, or both) as user 1902 gets closer to smartphone 102.

[0121] In addition, refer to Figures 15 to 19 The described technique, namely 3D gestures can be used to provide Figures 17 to 19Other features not shown in the figure. For example, a user interface element in a browser (such as a navigation bar) can be provided when the user reaches for the smartphone 102, and then disappear when the user moves his hand away. In another example, when the user reaches toward the smartphone 102, a lock screen or a home screen can be automatically presented (e.g., the user does not have to press a start button or a home button to display the lock screen). In other embodiments, the smartphone 102 can detect the shape or orientation of the approaching hand and change the interface accordingly. For example, the input surface can provide a touchpad interface when the user extends one hand (or extends one finger), or provide a full keyboard when the user extends two hands.

[0122] In other embodiments, the smartphone 102 may use the radar-based application 702 or another entity to determine that more than one object is present while the object 704 is within the sensing distance (e.g., determining the presence of another object within the sensing distance 706 while the object is within the sensing distance 706). In response to determining the presence of the object within the sensing distance 706, the radar-based application 702 may determine a 3D gesture (e.g., using radar data) with the object within the sensing distance and perform an action corresponding to the determined gesture. The smartphone 102 may also distinguish objects from each other and keep track of which objects make which 3D gestures.

[0123] Fig. 20 An example 2000 is illustrated in which an object (not shown) within the perceived distance 706 is a user playing a game using a smartphone (in this case, smartphone 102), and 3D gestures are used to control the game. Consider the example shown in detail view 2000-1, in which user 2002 and user 2004 are playing a two-player game on smartphone 102. In detail view 2000-1, user 2004 makes a 3D gesture (shown by arrow 2006 from left to right). In response to the 3D gesture, radar-based application 702 can enable the game to perform an action corresponding to the 3D gesture (e.g., returning from crossing the centerline to an object in the game environment, as shown on display 108). In this way, user 2002 and user 2004 can control the gameplay on smartphone 102 without obstructing the view of the game.

[0124] In another example shown in detail view 2000-2, user 2008 is playing a single-player game on smartphone 102. In detail view 2000-2, user 2008 makes a 3D gesture (shown by arrow 2010 from right to left), and radar-based application 702 can enable the game to perform an action corresponding to the 3D gesture (e.g., moving an object in the game environment from left to right). Thus, radar-based application 702 can interact with an application (such as a game) using 3D gestures without obstructing the user's field of view. In this way, because radar-based application 702 can distinguish between multiple users and their gestures, the game environment can be expanded to provide a better and more immersive multiplayer gaming experience even on a single small screen or device.

[0125] Example Method

[0126] Fig.21 and Fig. 22 An example method 2100 is depicted that enables smartphones, systems, and methods implemented in electronic devices. The method 2100 can be performed with an electronic device (e.g., the smartphone 102 described above) that includes a radar system that provides a radar field. The electronic device may also include a display, such as display 108. The radar field may be used to determine the orientation of the electronic device relative to an object in the radar field and provide orientation-based functionality of the electronic device. The radar field may also be used to determine a change in the orientation of the electronic device and modify the orientation-based functionality based on the changed orientation.

[0127] The method 2100 is shown as a set of boxes that specify the operations performed, but the method is not necessarily limited to the order or combination of operations shown for performing the various boxes. Further, any of the one or more operations can be repeated, combined, reorganized, or linked to provide a variety of additional and / or alternative methods. In the following discussion, reference can be made to Figure 1 An example operating environment 100 and Figures 2 to 20 The entities and processes detailed in the present disclosure are merely exemplary. The techniques are not limited to being performed by one entity or multiple entities operating on one device.

[0128] At 2102, a radar field is provided. The radar field may be provided by any of a variety of electronic devices (e.g., the smartphone 102 described above) that include a radar system (e.g., the radar system 104) and a radar-based application (e.g., any one or both of the radar-based applications 106 or 702, which may include one or more of the orientation module 114, the 3D gesture module 116, or the presence module 708, and any combination thereof). Further, the radar field may be any of a variety of types of radar fields, such as the radar field 110 described above.

[0129] In 2104, reflections from objects in the radar field are sensed by the radar system. The object may be any of a variety of objects, such as wood, plastic, metal, fabric, or organic material. For example, the object may be a person or a body part of a person (e.g., a hand), such as object 112 or 704 described above.

[0130] At 2106, reflections from objects in the radar field are analyzed. The analysis may be performed by any of a variety of entities (eg, radar system 104 or any of the radar-based applications described herein) and may include various operations or determinations, such as reference Figures 3 to 6 Describes the action or determination.

[0131] At 2108, based on the analysis of the reflections, radar data, such as the radar data described above, is provided. The radar data may be provided by any of a variety of entities, such as the radar system 104 or any of the radar-based applications described herein. In some embodiments, the radar system may provide the radar data and communicate the radar data to other entities (e.g., reference Figures 1 to 20 Method 2100 is described in any of the radar-based applications and other modules described herein. Fig. 22 Continue, such as Fig.21 The letter "A" after box 2108 (which is the same as Fig. 22 The letter “A” before box 2110 corresponds to the letter “A” before box 2110).

[0132] At 2110, an orientation of the electronic device relative to the object is determined. The determination is based on radar data and may be provided by any of a variety of entities, such as radar system 104 or any of the radar-based applications or other modules described herein.

[0133] In response to determining the orientation of the electronic device, providing orientation-based functionality of the electronic device at 2112. The orientation-based functionality may be provided by any of a variety of entities, such as radar system 104, any of radar-based applications 106 or 702, or any of the other modules described herein.

[0134] At 2114, a change in orientation of the electronic device relative to the object is determined. The determination is based on radar data and may be performed by any of a variety of entities, such as radar system 104, any of radar-based applications 106 or 702, or any of the other modules described herein.

[0135] In response to determining a change in the orientation of the electronic device relative to the object, the orientation-based functionality is modified at 2116. The modification of the orientation-based functionality may be performed by any of a variety of entities, such as radar system 104, any of radar-based applications 106 or 702, or any of the other modules described herein.

[0136] As reference Figure 1 As described, one example of an orientation-based functionality is a Do Not Disturb (DND) mode (e.g., a mode that mutes alerts, notifications, or other device-initiated communications). In this example, the initial orientation for activating the DND mode may be a landscape orientation. It should be noted that the landscape orientation is relative to where the user is, not the previous position of the electronic device. Thus, the user does not have to first orient the electronic device and then place it in landscape mode. Instead, the user simply rotates the electronic device to a landscape orientation, and the DND mode is activated.

[0137] In this example, the change in the orientation of the electronic device relative to the object is a rotation of the electronic device. The rotation can be in a plane substantially parallel to the viewing surface of the display 108 (e.g., a flat rotation on the surface). In addition, the user does not have to pick up the electronic device in order to change the orientation to be determined. Once it is determined that the change in the orientation of the electronic device has brought the electronic device into another orientation relative to the object (e.g., a portrait orientation), the electronic device exits the DND mode. Once the DND mode is deactivated, any muted messages or notifications are displayed so that the user can address them.

[0138] consider Figure 23 to Figure 25 The following additional examples are shown, which describe example implementations of an electronic device 102 that can implement additional details of the method 2100. Figure 23 to Figure 25 In the described example, the described features and techniques are implemented in a smartphone 102, although the described features and techniques may be implemented in any of a variety of electronic devices (e.g., as described in reference to Figure 2 described). Fig.23 An example implementation 2300 is depicted in which an orientation-based functionality of a smartphone 102 is to present content (e.g., via a display 108) in an orientation facing a user. In this example, the smartphone 102 provides a radar field 110 (e.g., via a radar system 104). In a detailed view 2300-1, a user 2302 (an object in the radar field) is viewing content on the smartphone 102 in a portrait orientation. The smartphone 102 (using, e.g., a radar-based application 106) determines the orientation of the smartphone 102 relative to the user 2302 and presents the content on the display in the orientation that the user is facing, as shown by a warrior image 2304.

[0139] In detailed view 2300-2, user 2302 changes the orientation of smartphone 102 to another orientation relative to user 2302 (e.g., a landscape orientation), as indicated by arrow 2306. In this example, the change in orientation is a flat rotation on the table on which smartphone 102 is located (e.g., the change in orientation of the smartphone relative to the user is a rotation of the smartphone in a plane that is substantially parallel to the viewing surface of display 108). Smartphone 102 (using, for example, radar-based application 106) can then determine that the change in orientation has caused smartphone 102 to be in another orientation relative to user 2302, and in response, maintain the user-facing orientation of the content while the smartphone is in the other orientation. As a result, warrior image 2304 remains in the user-facing orientation of detailed view 23001, even though smartphone 102 has been rotated and user 2302 has neither picked up the device nor manually changed the orientation.

[0140] Fig.24 An example implementation 2400 is depicted in which an orientation-based functionality of a smartphone 102 is to present content (e.g., via a display 108) in an orientation that is oriented toward a user who is viewing the display. In this manner, a user can hand the smartphone to another user, who will automatically be presented with a properly oriented display. In this example, the smartphone 102 provides a radar field 110 (e.g., via a radar system 104). In this example, assume that a user 2402 and another user 2404 (e.g., objects in the radar field include a first user and a second user) are viewing content on the smartphone 102 in a landscape orientation (e.g., an orientation that is oriented toward the user). The smartphone 102 (using, for example, a radar-based application 106) determines the orientation of the smartphone 102 relative to the user 2402 and presents content on the display in an orientation that is oriented toward the user, wherein the content is oriented toward the user 2402 (such a view is not shown in FIG. 1 ). Fig.24 not shown).

[0141] Continuing with the example, assume that user 2402 decides to let another user 2404 take a closer look at the content, and moves smartphone 102 toward another user 2404, as indicated by arrow 2406 (e.g., by shifting smartphone 102, user 2402 changes the orientation of smartphone 102 relative to user 2402, which results in smartphone 102 being further away from the first user and closer to the second user). Smartphone 102 (using, for example, radar-based application 106) may then determine that the change in orientation has resulted in smartphone 102 being in another orientation relative to user 2402, and in response, presents the content on the display in an orientation facing another user 2404. Thus, the content is automatically oriented to face another user 2404, even though smartphone 102 has been rotated and user 2402 has neither picked up the device nor manually changed the orientation.

[0142] Fig.25 An example implementation 2500 is depicted in which a radar-based application 106 or another entity may use an attention cue to determine a content display mode to present content on a display (e.g., display 108) of a smartphone 102. The radar-based application 106 uses the attention cue to determine an orientation of the smartphone 102 relative to the user 112, and also uses changes in the attention cue to determine changes in the orientation of the smartphone 102 relative to the user 112. In this manner, the attention cue may be used to determine whether the user 112 is paying attention to the smartphone 102.

[0143] Note that the cue is the body posture or position of the user 112, such as leaning forward or backward, or turning left or right, which can be detected using radar data, such as with respect to Figure 1 For example, radar data can be used to determine an attention cue by determining an angle of a user's torso or head relative to a plane (such as a table on which smartphone 102 is resting) that is substantially parallel to a viewing surface (e.g., a viewing surface of a display) of smartphone 102 (e.g., an attention cue is if user 112 is leaning toward smartphone 102), or an angular position of user 112's torso or head (e.g., an attention cue is if user 112 is walking away from smartphone 102), or the presence of user 112 within a threshold distance of smartphone 102.

[0144] In example implementation 2500, the orientation-based functionality of the smartphone 102 is to present content on a display in a content display mode (e.g., a first content display mode) based on an attention cue. The content display mode may be any of a variety of modes that the smartphone 102 may use when displaying or presenting content, such as a media pause mode or a media play mode, a lock screen mode, a sleep or awake mode, a full screen or slideshow mode, or including a specific screen brightness level or volume level.

[0145] In detailed view 2500-1, assume that user 112 is reading a book while listening to music through smartphone 102. The first content display mode in this example is a dimmed brightness level, as shown by shaded area 2502 on the display of smartphone 102. The attention cue in this example is the angular position of the head of user 112, as shown by shaded arrow 2504, which indicates that user 112 is reading a book instead of looking at the screen. In this way, smartphone 102 can save battery power by automatically dimming the screen when user 112 looks away based on the attention cue, but continuing to play music.

[0146] Continuing with the example in another detailed view 2500-2, the user 112 continues to listen to the music, but has turned to the smartphone 102, as indicated by the curved arrow 2506. The turning of the user 112's head indicates that the user 112 is now looking at the smartphone 102, rather than the book, as indicated by another shaded arrow 2508. The turning of the user 112's head is a change in the attention cue. In response to determining a change in the attention cue, the smartphone 102 (via the radar-based application 106 or another entity) can present the content in another content display mode (e.g., modifying the orientation-based functionality of the smartphone 102 to present the content in a second content display mode). In example 2500, the second content display mode is presenting a user interface on the display, illustrated by the music icon 2510. The user 112 can interact with the user interface (e.g., via touch, voice, or 3D gesture input) to control the music player for playing the music.

[0147] As described above, in some embodiments, including the above-described example embodiments, the smartphone 102, using the radar-based application 106 or another entity, can determine that if a rotation (e.g., a rotation of the smartphone 102 or the torso or head of the user 112) exceeds a threshold, then the rotation is effective to cause a modification of the orientation-based functionality. The threshold can be any suitable rotation, such as (25 degrees, 45 degrees, 90 degrees, or 180 degrees). Similarly, if the displacement results in at least a threshold distance between the smartphone 102 and the first user, then the displacement may be effective to cause a modification of the orientation-based functionality. The threshold distance can be any suitable distance, such as 6 inches, 12 inches, or 18 inches. Further, if an angle (e.g., an angle at which the user 112 is tilted forward or backward) exceeds a threshold angle, then the angle may be effective to cause a modification of the orientation-based functionality. The threshold angle can be any suitable angle, such as (15 degrees, 25 degrees, 35 degrees, or 45 degrees). The threshold rotation, distance, or angle may be predefined, user selectable, or determined by a machine learning module included in or associated with radar system 104 or radar-based application 106 .

[0148] Additionally, position, angle, rotation, and other orientation changes may be effective in causing modification of orientation-based functionality if such changes persist for at least a threshold time. The threshold time may be any suitable time (e.g., 0.5 seconds, 1.5 seconds, or 2.5 seconds) and may be predefined, user selectable, or determined by a machine learning module included in or associated with radar system 104 or radar-based application 106.

[0149] It should be noted that these technologies for smartphones, systems and methods implemented in electronic devices may be more secure than other technologies that can provide similar functions. Not only is it generally impossible for unauthorized persons to obtain the user's location, orientation or 3D gestures (especially user-defined gestures, micro-gestures, and gestures based on posture or location) (for example, unlike passwords), but the user's radar image (even if the user's radar image includes the user's face) cannot visually identify the user like a photo or video. Even so, in addition to the above description, the user can also be provided with controls that allow the user to make the following choices: whether and when any of the systems, programs, managers, modules or features described in this document can enable the collection of user information (for example, information about the user's social network, social actions or activities, occupation, user's preferences or the user's current location), and whether to send content or communications to the user from the server. In addition, certain data can be processed in one or more ways before it is stored or used so that personally identifiable information is removed. For example, the user's identity can be processed so that personally identifiable information cannot be determined for the user, or the user's geographic location can be generalized (such as to the city, zip code or state level) when positioning information is obtained so that the user's specific location cannot be determined. Thus, the user can control what information is collected about the user, how that information is used, and what information is provided to or about the user.

[0150] Example computing system

[0151] Fig.26 Illustrate various components of an example computing system 2600, which may be implemented as described with reference to the previous Figures 1 to 25 Any type of client, server, and / or electronic device described may be used to implement the smartphone, systems, and methods implemented on the electronic device, or techniques enabling the smartphone, systems, and methods implemented on the electronic device may be implemented in the example computing system 2600 .

[0152] The computing system 2600 includes a communication device 2602 that enables wired and / or wireless communication of device data 2604 (e.g., radar data, 3D gesture data, authentication data, reference data, received data, data being received, data scheduled for broadcast, data packets of data). The device data 2604 or other device content may include configuration settings of the device, media content stored on the device, and / or information associated with a user of the device (e.g., the position, orientation, or identity of a person within a radar field). The media content stored on the computing system 2600 may include any type of radar, biometric, audio, video, and / or image data. The computing system 2600 includes one or more data inputs 2606, via which any type of data, media content, and / or input may be received, such as human speech, interaction with a radar field, touch input, user-selectable input (explicit or implicit), messages, music, television media content, recorded video content, and any other type of audio, video, and / or image data received from any content and / or data source. The data input 2606 may include, for example, radar-based applications 106 and 702 , the orientation module 114 , the 3D gesture module 116 , or the presence module 708 .

[0153] The computing system 2600 also includes a communication interface 2608, which can be implemented as any one or more of a serial and / or parallel interface, a wireless interface, any type of network interface, a modem, and any other type of communication interface. The communication interface 2608 provides a connection and / or communication link between the computing system 2600 and a communication network through which other electronic, computing, and communication devices communicate data with the computing system 2600.

[0154] The computing system 2600 includes one or more processors 2610 (e.g., any of a microprocessor, controller, or other controller) that can process various computer-executable instructions to control the operation of the computing system 2600 and enable or implement the techniques for the smartphone, system, and method implemented in the electronic device. Alternatively or additionally, the computing system 2600 can be implemented using any one or combination of hardware, firmware, or fixed logic circuitry implemented in conjunction with processing and control circuits, which are generally identified at 2612. Although not shown, the computing system 2600 may include a system bus or data transfer system that couples various components within the device. The system bus may include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus utilizing any one of a variety of bus architectures.

[0155] The computing system 2600 also includes computer-readable media 2614, such as one or more memory devices that enable persistent and / or non-transitory data storage (i.e., as opposed to mere signal transmission), examples of which include random access memory (RAM), non-volatile memory (e.g., any one or more of read-only memory (ROM), flash memory, EPROM, EEPROM, etc.), and disk storage devices. The disk storage device can be implemented as any type of magnetic or optical storage device, such as a hard drive, a recordable and / or rewritable compact disk (CD), any type of digital versatile disk (DVD), etc. The computing system 2600 may also include a mass storage medium device (storage medium) 2616.

[0156] The computer-readable medium 2614 provides a data storage mechanism to store device data 2604 as well as various device applications 2618 and any other type of information and / or data related to the operational aspects of the computing system 2600. For example, the operating system 2620 can be maintained as a computer application with the computer-readable medium 2614 and executed on the processor 2610. The device applications 2618 can include device managers (such as any form of control application), software applications, signal processing and control modules, native code for specific devices, abstract modules, gesture recognition modules, and other modules. The device applications 2618 can also include system components, engines or managers for implementing smartphones, systems and methods implemented in electronic devices, such as the radar system 104, radar-based applications 106, radar-based applications 702, orientation modules 114, 3D gesture modules 116, or presence modules 708. The computing system 2600 can also include or have access to one or more machine learning modules or systems.

[0157] Some examples are described below.

[0158] Example 1. A smartphone comprising: a display; a radar system, the radar system being at least partially implemented in hardware, the radar system being configured to: provide a radar field; sense reflections from objects in the radar field; analyze reflections from the objects in the radar field; and provide radar data based on the analysis of the reflections; one or more computer processors; and one or more computer-readable media having instructions stored thereon, the instructions performing operations in response to being executed by the one or more computer processors, the operations comprising: determining an orientation of the smartphone relative to the object based on a first subset of the radar data; providing orientation-based functionality of the smartphone in response to determining the orientation of the smartphone; determining a change in the orientation of the smartphone relative to the object based on a second subset of the radar data; and modifying the orientation-based functionality of the smartphone in response to the change in the orientation.

[0159] Example 2. A smartphone according to Example 1, wherein: when the smartphone is determined to be in a first orientation relative to the object, the orientation-based function of the smartphone is to enter a do not disturb (DND) mode, in particular a DND mode that mutes device-initiated communications; the change in the orientation of the smartphone relative to the object is a rotation of the smartphone within a plane substantially parallel to a viewing surface of the display, the rotation being effective to cause the smartphone to be in a second orientation relative to the object; and the modification to the orientation-based function is to exit the DND mode.

[0160] Example 3. A smartphone according to Example 1 or 2, wherein: the object is a user; the orientation-based functionality of the smartphone is to present content on the display in an orientation facing the user; the change in the orientation of the smartphone relative to the user is a rotation of the smartphone within a plane substantially parallel to a viewing surface of the display, the rotation being effective to cause the smartphone to be in another orientation relative to the user; and the modification to the orientation-based functionality is to maintain the user-facing orientation of the content while the smartphone is in the other orientation relative to the user.

[0161] Example 4. A smartphone according to at least one of the preceding examples, wherein: the object includes at least a first user and a second user; the orientation-based functionality of the smartphone is to present content on the display in an orientation facing the first user; the change in the orientation of the smartphone relative to the first user is a displacement of the smartphone, the displacement being effective in causing the smartphone to be farther away from the first user and closer to the second user; and the modification to the orientation-based functionality is to present content on the display in an orientation facing the second user.

[0162] Example 5. A smartphone according to at least one of the preceding examples, wherein: the object is a user; determining the orientation of the smartphone relative to the user based on the radar data further includes detecting an attention prompt from the user; the orientation-based functionality of the smartphone is to present content on the display in a first content display mode, the first content display mode being based on the attention prompt; a change in the orientation of the smartphone relative to the user is a change in the attention prompt; and a modification to the orientation-based functionality of the smartphone is to present content on the display in a second content display mode, the second content display mode being based on the change in the attention prompt.

[0163] Example 6. A smartphone according to Example 5, wherein: the first content display mode and the second content display mode include one or more of the following items: media pause mode, media play mode, lock screen mode, sleep mode, wake-up mode, full screen mode, slide mode, screen brightness level or volume level.

[0164] Example 7. A smartphone according to Example 5 or 6, wherein the attention cue is a body posture of the user, the body posture comprising one or more of the following: an angle of the user's torso relative to a plane substantially parallel to a viewing surface of the display; an angle of the user's head relative to a plane substantially parallel to a viewing surface of the display; an angular position of the user's torso; an angular position of the user's head; or the presence of the user within a threshold distance of the smartphone.

[0165] Example 8. A system comprising: an electronic device; a radar system, the radar system being implemented at least in part in hardware, the radar system being configured to: provide a radar field; sense reflections from objects in the radar field; analyze reflections from the objects in the radar field; and provide radar data based on the analysis of the reflections; one or more computer processors; and one or more computer-readable media, the one or more computer-readable media having instructions stored thereon, the instructions performing operations in response to being executed by the one or more computer processors, the operations comprising: determining the presence of the object within a perception range of the electronic device based on a first subset of the radar data; providing presence-based functionality of the electronic device in response to determining the presence of the object within the perception range; determining the object outside the perception range of the electronic device based on a second subset of the radar data; and ceasing to provide the presence-based functionality in response to determining that the object is outside the perception range.

[0166] Example 9. A system according to Example 8, wherein the presence-based functionality is one or more of: adjusting the volume of a ringtone, alarm, or notification; adjusting a communication mode used by an electronic assistant included in the electronic device; adjusting content presented on a user interface; providing a notification of the presence of the object within the perception distance, the notification being provided to one or more contacts stored on or accessible by the electronic device; or providing a notification of a previously presented reminder when the object is not within the perception distance.

[0167] Example 10. A system according to Example 8 or 9, wherein the electronic device includes a display and the radar-based application is further configured to: determine the presence of another object within the perception distance while the object is within the perception distance; and modify the presence-based functionality of the electronic device in response to determining the presence of the other object within the perception distance.

[0168] Example 11. A system according to at least one of Examples 8 to 10, wherein the radar-based application is further configured to: determine a 3D gesture of the object within the perception distance of the electronic device based on the radar data; and perform an action corresponding to the determined 3D gesture.

[0169] Example 12. A system according to at least one of Examples 8 to 11, wherein the electronic device includes a display and the radar-based application is further configured to: determine the presence of another object within the perception distance while the object is within the perception distance; and in response to the presence of the other object within the perception distance and based on the radar data, determine: a gesture of the object within the perception distance of the electronic device; and another gesture of the other object within the perception distance of the electronic device; perform an action corresponding to the determined gesture; and perform an action corresponding to the determined another gesture.

[0170] Example 13. The system of Example 8, wherein the radar system further comprises a digital beamformer and an angle estimator, and the radar system is configured to monitor angles between approximately -90 degrees and approximately 90 degrees in a field of view.

[0171] Example 14. A method, implemented in an electronic device, the electronic device including a radar system and a radar-based application, the method comprising: providing a radar field through the radar system; sensing reflections from objects in the radar field through the radar system; analyzing reflections from the objects in the radar field; providing radar data based on the analysis of the reflections; determining an orientation of the electronic device relative to the object based on a first subset of the radar data; providing orientation-based functionality of the electronic device in response to determining the orientation of the electronic device; determining a change in the orientation of the electronic device relative to the object based on a second subset of the radar data; and modifying the orientation-based functionality of the electronic device in response to the change in the orientation.

[0172] Example 15. A method according to Example 14, wherein the electronic device includes a display, the change in the orientation of the electronic device relative to the object is a rotation of the electronic device within a plane substantially parallel to a viewing surface of the display, and the method further includes: in response to determining the orientation of the electronic device relative to the object, causing the electronic device to enter a do not disturb (DND) mode, in particular a DND mode that mutes device-initiated communications; determining that the change in the orientation of the electronic device is effective to cause the electronic device to be in another orientation relative to the object; and in response to determining that the electronic device is in another orientation relative to the object, causing the electronic device to exit the DND mode.

[0173] Example 16. A method according to Example 14 or 15, wherein: the electronic device includes a display; the object is a user; the change in the orientation of the electronic device relative to the user is a rotation of the electronic device within a plane substantially parallel to a viewing surface of the display; and the method further includes: in response to determining the orientation of the electronic device relative to the user, presenting content on the display in an orientation facing the user; determining that the change in the orientation of the electronic device is effective to cause the electronic device to be in another orientation relative to the user; and in response to determining that the electronic device is in the other orientation, maintaining the user-facing orientation of the content when the electronic device is in the other orientation relative to the user.

[0174] Example 17. A method according to at least one of Examples 14 to 16, wherein: the electronic device includes a display; the object includes at least a first user and a second user; determining the orientation of the electronic device relative to the object includes determining the orientation of the electronic device relative to the first user; determining a change in the orientation of the electronic device relative to the object includes determining a change in the orientation of the electronic device relative to the first user; the change in the orientation of the electronic device relative to the first user is a displacement of the electronic device, and the displacement is effective in causing the electronic device to be farther away from the first user and closer to the second user; and the method further includes: in response to determining the orientation of the first user, presenting content on the display with a first user-facing orientation; in response to determining a change in the orientation of the electronic device relative to the first user, presenting content on the display with a second user-facing orientation.

[0175] Example 18. A method according to at least one of Examples 14 to 17, wherein: the electronic device includes a display; the object is a user; determining the orientation of the electronic device relative to the user based on the radar data further includes detecting an attention prompt from the user; a change in the orientation of the electronic device relative to the user is a change in the attention prompt; and the method further includes: presenting content on the display in a first content display mode, the first content display mode is based on the attention prompt; and in response to determining a change in the attention prompt, presenting content on the display in a second content display mode, the second content display mode is based on the change in the attention prompt.

[0176] Example 19. A method according to Example 18, wherein: the first content display mode and the second content display mode include one or more of the following items: media pause mode, media play mode, screen brightness level, lock screen mode, sleep mode, full screen mode, slide show mode or volume level.

[0177] Example 20. A method according to Example 18 or 19, wherein the attention cue is a body posture of the user, and the body posture includes one or more of the following items: an angle of the user's torso relative to a plane substantially parallel to a viewing surface of the display or an axis substantially perpendicular to the plane containing the electronic device; an angular position of the user's torso; an angle of the user's head relative to a plane substantially parallel to the viewing surface of the display or an axis substantially perpendicular to the plane containing the electronic device; an angular position of the user's head; or the presence of the user within a threshold distance of the electronic device.

[0178] in conclusion

[0179] Although embodiments of techniques and apparatus for enabling smartphones, systems, and methods implemented in electronic devices have been described in language specific to features and / or methods, it should be understood that the subject matter of the appended claims is not necessarily limited to the specific features or methods described. Rather, specific features and methods are disclosed as example embodiments of enabling smartphones, systems, and methods implemented in electronic devices.

Claims

1. An electronic device, comprising: A radar system, the radar system being implemented at least partially in hardware, the radar system being configured to: Provide radar fields; sensing reflections from users in the radar field; analyzing reflections from the user in the radar field; and providing radar data based on analyzing the reflections; one or more computer processors; as well as One or more computer-readable media having stored thereon instructions that in response to being executed by the one or more computer processors perform operations comprising: determining, based on the first subset of the radar data, a presence of the user within a sensing range of the electronic device; determining, based on a second subset of the radar data, whether the user is alone within a sensing range of the electronic device; In response to determining that the user is within the perception distance and whether the user is alone within the perception distance of the electronic device, determining a spatial relationship context of the user; and The spatial relationship context of the user is provided to one or more other electronic devices.

2. The electronic device according to claim 1, wherein: The user's spatial relationship context includes the user's status with respect to the electronic device; as well as The user's status regarding the electronic device includes: Available, the available state indicates that the user is alone within the sensing distance of the electronic device; Busy, where the busy state indicates that the user is within the sensing distance of the electronic device and the user is not alone; or Leave, the leave state indicates that the user is not within the sensing distance of the electronic device.

3. The electronic device according to claim 2, wherein: Providing the spatial relationship context of the user to one or more other electronic devices further includes providing a status indicator: The status indicator is configured to be displayed on the one or more other electronic devices together with the contact information of the user; and The status indicator indicates the user's status with respect to the electronic device.

4. The electronic device according to claim 3, wherein: The operations further include providing selectable controls to the one or more other electronic devices, the selectable controls being configured to be displayed together with the contact information of the user, the selectable controls comprising at least one of the following: a first selectable control configured to cause the one or more other electronic devices to send a message to the electronic device; a second selectable control configured to cause the one or more other electronic devices to place a telephone call to the electronic device; a third selectable control configured to cause the one or more other electronic devices to place a telephone call to the electronic device when the user's status with respect to the electronic device changes from busy or away to available; or A fourth selectable control is configured to cause the electronic device and the one or more other electronic devices to simultaneously receive phone calls from each other when the user's status with respect to the electronic device changes from busy or away to available.

5. The electronic device according to claim 4, wherein: Providing selectable controls configured to be displayed with the user's contact information to the one or more other electronic devices further includes providing one or more of the first selectable control, the second selectable control, the third selectable control, or the fourth selectable control based on user input through the electronic device.

6. The electronic device according to claim 2, wherein: Providing the spatial relationship context of the user to one or more other electronic devices further includes communicating the spatial relationship context to the one or more other electronic devices using a text message.

7. The electronic device according to claim 6, wherein: The text message includes a selectable control for contacting the user, the selectable control including at least one of: a first selectable control configured to cause the one or more other electronic devices to send a message to the electronic device; a second selectable control configured to cause the one or more other electronic devices to place a telephone call to the electronic device; a third selectable control configured to cause the one or more other electronic devices to place a telephone call to the electronic device when the user's status with respect to the electronic device changes from busy or away to available; or A fourth selectable control is configured to cause the electronic device and the one or more other electronic devices to simultaneously receive phone calls from each other when the user's status with respect to the electronic device changes from busy or away to available.

8. The electronic device according to claim 7, wherein: Using a text message to communicate the spatial relationship context to the one or more other electronic devices further includes determining, based on user input through the electronic device, that one or more of the first selectable control, the second selectable control, the third selectable control, or the fourth selectable control will be included in the text message.

9. The electronic device according to claim 1, wherein: The operations further include: Determining one or more other electronic devices that provide spatial relationship context for the user based on user input; or One or more other electronic devices that provide a spatial relationship context for the user are determined based on a default setting of the electronic device.

10. The electronic device according to claim 1, wherein: the second subset of the radar data and the first subset of the radar data being a same subset of the radar data; The second subset of radar data does not include radar data of the first subset of radar data; or The second subset of the radar data includes at least a portion of the first subset of the radar data.

11. An electronic device, comprising: one or more computer processors; as well as One or more computer-readable media having stored thereon instructions that in response to being executed by the one or more computer processors perform operations comprising: receiving, from another electronic device that includes a radar system, a spatial relationship context of a user of the other electronic device, the spatial relationship context of the user determined based on one or more subsets of radar data provided by the radar system and including a status of the user with respect to the other electronic device; as well as A status indicator is presented on a display of the electronic device and in association with the contact information of the user, the status indicator indicating a status of the user with respect to the other electronic device.

12. The electronic device according to claim 11, wherein: The user's status with respect to the other electronic device includes: Available, the available state indicates that the user is alone within the sensing distance of the other electronic device; Busy, the busy state indicating that the user is within the sensing distance of the other electronic device and the user is not alone; or Leave, the state of leaving indicates that the user is not within the sensing distance of the other electronic device.

13. The electronic device according to claim 11, wherein: Presenting the status indicator further includes displaying the status indicator on a display of the electronic device along with contact information of the user.

14. The electronic device according to claim 13, wherein: Displaying the status indicator on the display of the electronic device along with the user's contact information further includes displaying a selectable control for contacting the user, the selectable control being displayed along with the user's contact information and including at least one of: a first selectable control that, when selected, causes the electronic device to send a message to the other electronic device; a second selectable control that, when selected, causes the electronic device to place a telephone call to the other electronic device; a third selectable control that, when selected, causes the electronic device to place a telephone call to the other electronic device when the user's status with respect to the other electronic device changes from busy or away to available; or A fourth selectable control, which when selected causes the electronic device and the other electronic device to simultaneously receive a phone call from each other when the user's status with respect to the other electronic device changes from busy or away to available.

15. The electronic device according to claim 11, wherein: The one or more subsets of the radar data are subsets of radar data provided by a radar system of the other electronic device, the radar system being at least partially implemented in hardware and configured to provide a radar field, sense reflections from the user in the radar field, analyze reflections from the user in the radar field, and provide the radar data based on the analysis of the reflections.

16. A method implemented in an electronic device, the electronic device comprising a radar system, the method comprising: providing a radar field by means of said radar system; sensing, by the radar system, reflections from users in the radar field; analyzing reflections from the user in the radar field; providing radar data based on analyzing the reflections; determining, based on the first subset of the radar data, a presence of the user within a sensing range of the electronic device; determining, based on a second subset of the radar data, whether the user is alone within a sensing range of the electronic device; In response to determining that the user is within the perception distance and whether the user is alone within the perception distance of the electronic device, determining a spatial relationship context of the user; as well as The spatial relationship context of the user is provided to one or more other electronic devices.

17. The method according to claim 16, wherein: The spatial relationship context of the user includes the state of the user with respect to the electronic device, and the state of the user with respect to the electronic device includes: Available, the available state indicates that the user is alone within the sensing distance of the electronic device; Busy, where the busy state indicates that the user is within the sensing distance of the electronic device and the user is not alone; or Leave, the leave state indicates that the user is not within the sensing distance of the electronic device.

18. The method according to claim 17, wherein: Providing the spatial relationship context of the user to one or more other electronic devices includes providing a status indicator, the status indicator being configured to be presented on a corresponding display of the one or more other electronic devices together with the contact information of the user and indicating the status of the user with respect to the electronic devices, and the method further comprising: Providing a selectable control to the one or more other electronic devices, wherein the selectable control is configured to be displayed together with the contact information of the user, and the selectable control includes at least one of the following: a first selectable control configured to cause the one or more other electronic devices to send a message to the electronic device; a second selectable control configured to cause the one or more other electronic devices to place a telephone call to the electronic device; A third selectable control is configured to cause the one or more other electronic devices to place a telephone call to the electronic device when the user's status with respect to the electronic device changes from busy or away to available; or A fourth selectable control is configured to cause the electronic device and the one or more other electronic devices to simultaneously receive phone calls from each other when the user's status with respect to the electronic device changes from busy or away to available.

19. The method of claim 17, wherein: The electronic device includes a non-radar sensor; and The user's status with respect to the electronic device also includes a mobile status determined at least in part based on data from the non-radar sensor and indicating that the user is not alone and is traveling in a vehicle.

20. The method according to claim 19, wherein: The non-radar sensors include one or more of a geo-location sensor or a detection sensor capable of detecting when the electronic device is operating in a hands-free driving mode.

Images