User discrimination for radar-based gesture detector

Abstract

Techniques and apparatus for user discrimination of a radar-based gesture detector are described in this document. The techniques enable a computing device to distinguish users using a radar system that may collect and analyze radar characteristics of a user to distinguish the user from other users. The radar characteristics may include radar reflection characteristics of the user, such as topology, time, gesture, and / or contextual information. Users may be distinguished without determining individually identifiable information, and a computing device may record radar characteristics to distinguish each user at a later time and provide a custom experience. When an unregistered person is detected, the radar system may assign to the unregistered person an unregistered user identification containing detected radar characteristics to distinguish the person from other users at future time.

Description

Background Art

[0001] Computing devices utilizing virtual assistant (VA) technology are becoming increasingly popular, such as as VA technology enters smart home systems. Many current VA-equipped devices include an artificial intelligence (AI) assistant that detects commands issued by a user and then instructs the device to perform a task associated with the command. For example, a device may detect (e.g., using a microphone) a verbal command for "turn on the bedroom lights" and send a control signal to turn on the lights.

[0002] Many VA-equipped devices (e.g., integrated into smart home systems) have more than one user, and each of these users may wish to set their own preferences on the device, protect their information from other users, and execute commands in their own way. This means that in addition to distinguishing commands, VA-equipped devices may also need to distinguish between users to provide a personalized experience. However, some devices cannot distinguish between users, thereby reducing the functionality of the VA-equipped device. Summary of the invention

[0003] This document describes techniques and apparatus for user discrimination of radar-based gesture detectors. These techniques enable a computing device to distinguish users using a radar system that can collect and analyze radar characteristics of a user to distinguish the user from other users. The radar characteristics can include radar reflection characteristics of the user, such as topology, time, gesture, and / or contextual information. Users can be distinguished without determining personally identifiable information, which includes, for example, legal name, biometric information, personal address, facial details, images, speech-to-text analysis, etc. The computing device can record these radar characteristics to distinguish each user at a later time and provide a customized experience. When a new user (e.g., an unregistered person) is detected, the radar system can assign an unregistered user identifier containing the detected radar characteristics to the unregistered person to distinguish the person from other users.

[0004] Various aspects described below include a method, system, apparatus, and component for user identification of a radar-based gesture detector. The method may include transmitting a radar transmit signal from a radar system associated with a computing device, and receiving a radar receive signal at the radar system or another radar system associated with the computing device. The computing device may determine, based on the radar receive signal, a radar characteristic of an object from which the radar receive signal was reflected. Based on the radar characteristic of the object, the computing device may also determine that the object is an unregistered person who is not a registered user associated with the computing device. An unregistered user identifier may be assigned to the unregistered person, and the unregistered person may be associated with the radar characteristic of the object. The unregistered user identifier and the associated radar characteristic may be stored to enable the presence of the unregistered person to be determined at a future time. The presence may be determined by correlating future received radar receive signals having similar or identical radar characteristics with the associated radar characteristic of the unregistered person. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Devices and techniques for user recognition of radar-based gesture detectors are described with reference to the following figures.

[0006] The same numbers are used throughout the drawings to reference the same features and components: Figure 1 Example sequences are shown in which these techniques may be used; Figure 2 An example implementation of a radar system as part of a computing device is shown; Figure 3 An example environment is shown in which multiple computing devices are connected via a communication network to form a computing system; Figure 4 An example environment is shown in which a radar system is used by a computing device to detect the presence of a user; Figure 5 An example implementation of an antenna, analog circuitry, and system processor for a radar system is shown; Figure 6 shows an example implementation in which a user detection module can distinguish users; Figure 7 An example implementation of a machine learning (ML) model for distinguishing users of a computing device is shown; Figure 8 An example implementation of a computing device using additional sensors to improve the fidelity of user identification is shown; Fig. 9 shows an example sequence in which privacy settings are modified based on user presence; and Figure 10-1 , Figure 10-2 and Figure 10-3An example method for user recognition for a radar-based gesture detector is shown. DETAILED DESCRIPTION Overview

[0007] Computing devices that utilize virtual assistant (VA) technology are becoming increasingly popular, such as as VA technology enters smart home systems. Devices equipped with VA can make it easier for users to control, for example, lights in a room, music being played, calendar reminders, or phone conversations. Specifically, these devices can detect commands issued by a user to instruct the device to perform a task. A microphone or camera can be used to detect commands, and then the commands can be extracted (e.g., interpreted for the user's intent) using voice recognition technology or by identifying visual cues, respectively. For example, a user can provide a voice command of "turn the bedroom lights off", and the device can interpret the voice (e.g., using voice-to-text technology) to determine that the bedroom lights should be turned off.

[0008] Many VA-equipped devices (e.g., integrated into smart home systems) have more than one user, and each of these users may wish to set their own preferences on the device, protect their information from other users, and execute commands in their own way. Therefore, users need customized experiences to help speed up task progress, enable better anticipation of each user's needs, and improve users' sense of security around the device. For example, users may wish to provide a command to "check my calendar today" without having to identify themselves every time they approach the device.

[0009] Some VA-equipped devices automatically detect and identify users, but use sensors and / or recognition technology that is invasive. Users may be concerned about the privacy of their personally identifiable information around these devices. For example, a camera may collect images or videos that the user considers private, thereby preventing the user from feeling comfortable near the device. Some devices implement facial recognition technology that can be used to identify and track users, thereby reducing the user's ability to relax in their home when a VA-equipped device is nearby. For VA-equipped devices that utilize microphones, users may wonder if the device is listening to their private conversations and performing speech-to-text of conversations other than speech intended as commands.

[0010] Relatedly, users may desire controls to adaptively change the privacy of their personal information based on detection of individuals in a room. For example, when a user is alone in a room, they may wish to receive audio reminders about calendar events (e.g., medical appointments). When the user is in the room with guests, they may instead wish for the device to refrain from automatically announcing reminders so that their calendar information remains private. However, many devices do not provide users with adaptive privacy controls, which may reduce the privacy of a user's personal information and / or limit the functionality of the device.

[0011] To address these challenges, the present disclosure describes a method for user discrimination for a radar-based gesture detector. The radar-based gesture detector can primarily utilize a radar system to detect gesture commands performed by a user. The technology described herein improves the performance of VA-equipped devices by utilizing the radar-based gesture detector to: (1) detect and distinguish users without using personally identifiable information, thereby enabling a customized experience, (2) prompt users to start or continue gesture training based on their corresponding training history, and (3) adaptively adjust privacy controls to meet the user's privacy expectations. In this way, users can enjoy the functionality of a radar-based gesture detector in their homes. Example Environment

[0012] Figure 1 An example sequence 100 is shown in which these techniques may be used. These techniques may be performed by a computing device 102 configured as a radar-based gesture detector (e.g., a VA-equipped device). The computing device 102 may be used to automate tasks (e.g., operate room lights, control music playing in a room, remind a user of an appointment) through gesture commands. For example, a swipe of a user's hand may indicate a command to change the song being played, while a push-pull gesture may indicate a command to view the status of a timer in the kitchen. The computing device 102 may allow for multiple users 104, each of which may enjoy a customized experience with the computing device 102. In addition, the first computing device 102-1 may be connected (e.g., wirelessly) to another computing device 102-X (where X represents an integer value of 2, 3, 4, etc.) to create an interconnected network of radar-based gesture detectors. The detector network may be arranged to detect gesture commands in various rooms of a smart home, for example.

[0013] Specifically, computing device 102 may: (1) detect the presence of user 104 within proximity 106 without personally identifying user 104, (2) distinguish user 104 from other users to enable a customized experience, and then (3) perform an operation upon receiving a gesture command from user 104. Additionally or in lieu of (3), computing device 102 may prompt user 104 to begin or continue gesture training based on the user's training history. In one example, computing device 102 detects a registered user by correlating a radar reception signal of a detected object with one or more stored radar characteristics of the registered user. Computing device 102 may then perform an operation upon receiving a command and / or may prompt the user to continue gesture training based on its training history.

[0014] Computing device 102 may use radar system 108 to transmit one or more radar transmission signals (e.g., modulated electromagnetic (EM) waves in the radio frequency (RF) range) to detect the presence of users in nearby area 106. When an object (e.g., user 104) is detected within nearby area 106, the radar transmission signal may be reflected from user 104 and modified (e.g., in amplitude, time, phase, or frequency) based on topography and movement of user 104. The modified radar transmission signal (e.g., radar reception signal) may be received by radar system 108 and contain information for distinguishing user 104 from other users. For example, radar system 108 may use radar reception signal to determine the speed, size, shape, surface smoothness, or material of user 104. Radar system 108 may also determine the distance between user 104 and computing device 102 and / or the orientation of user 104 relative to computing device 102.

[0015] Although the vicinity 106 of the example sequence 100 is depicted as a hemisphere, in general, the vicinity 106 is not limited to the topography shown. The topography of the vicinity 106 may also be affected by nearby obstacles (e.g., walls, large objects). In addition, the radar system 108 may detect and detect users outside the vicinity 106 depicted in the example sequence 100. The boundary of the vicinity 106 may also correspond to an accuracy threshold, wherein users detected within the boundary are more likely to be accurately distinguished than users detected outside the boundary.

[0016] In example sequence 100-1, computing device 102 transmits a first radar transmit signal to nearby area 106, and then receives a first radar receive signal (e.g., a reflected radar transmit signal) associated with the presence of an object (e.g., first user 104-1). The first radar receive signal may include one or more radar characteristics (e.g., radar cross section (RCS) data, motion signature, gesture execution, etc.) that can be used to distinguish first user 104-1 from other users 104. Specifically, radar system 108 may compare the first radar receive signal with stored radar characteristics to determine whether first user 104-1 is a registered user. In this example, the first radar receive signal is associated with one or more stored radar characteristics of registered users.

[0017] To improve some VA-equipped devices, the computing device 102 in the example sequence 100-1 may forgo “personally identifying” the first user 104-1 (e.g., identifying private or personally identifiable information of the first user) when determining that the detected object is a registered user. For example, the computing device 102 may determine that the first user 104-1 is a registered user without the need for personally identifiable information, where the personally identifiable information may include legally identifiable information (e.g., legal name), personal address, biometric information, financial information, browsing history, etc. In addition, the computing device 102 may forgo identifying the first user 104-1’s personal device (e.g., a mobile phone, a device equipped with an electronic tag), collecting facial recognition information, or performing speech-to-text on potentially private conversations when determining that the first user 104-1 is a registered user. Instead of personally identifying the first user 104-1, the computing device 102 may use radar characteristics that do not contain confidential information to “distinguish” the first user 104-1 from another user (e.g., the second user 104-2), such as with respect to Figure 4 described.

[0018] Controls may be provided to the user 104 that allow the user 104 to choose whether and when the technology described herein may enable the collection of user information (e.g., information about the user's social network, social behavior, social activities, occupation, photos taken by the user, audio recordings recorded by the user, the user's preferences, the user's current location, etc.) and whether to send content or communications from the server to the user 104. In addition, before certain data is stored or used, it may be processed in one or more ways so that personally identifiable information is removed. For example, the user's identity may be processed so that personally identifiable information of the user 104 cannot be determined, or the user's geographic location in which location information is obtained may be generalized (e.g., to a city, zip code, or state level) so that the specific location of the user 104 cannot be determined. Thus, the user 104 can control what information is collected about the user 104, how the information is used, and what information is provided to the user 104.

[0019] After determining that first user 104-1 is a registered user, computing device 102 may prompt first user 104-1 to start or continue gesture training. For example, first user 104-1 may be halfway through training and have completed training on a first gesture. Computing device 102 may have stored information about the manner in which first user 104-1 performed the first gesture during training in a first training history. Based on the first training history (at example sequence 100-1), computing device 102 may prompt first user 104-1 to continue training on a second gesture rather than repeating training on the first gesture. As a result, determining that first user 104-1 is a registered user may allow computing device 102 to improve the efficiency of gesture training. Identification of first user 104-1 may also allow computing device 102 to activate settings (e.g., privacy settings, preferences) of registered users to provide a customized experience for first user 104-1.

[0020] At a later time, as shown in example sequence 100-2, second user 104-2 is on a couch with first user 104-1 in nearby area 106. Computing device 102 may use radar system 108 to send a second radar transmit signal to detect the presence of another object (e.g., second user 104-2). Radar system 108 may then compare the second radar receive signal with the stored radar characteristics to determine whether second user 104-2 is a registered user. In this example, the second radar receive signal is not found to be associated with one or more stored radar characteristics of a registered user (first user 104-1 or some other registered user). Therefore, second user 104-2 is distinguished as an unregistered person.

[0021] In the present disclosure, a registered user is a person who is associated with at least one stored radar characteristic and who, by virtue of a previous registration, has greater rights (e.g., rights to use and access devices and applications) to a computing device 102 or a device, system, or application associated with the computing device 102 than a person who has not previously registered. The previous registration may be based on i) a previously received intent to register with the computing device 102 or a device, system, or application associated with the computing device 102, or ii) a previous authorization for repeated use of the computing device 102 or a device, system, or application associated with the computing device 102. In some cases, one or more stored radar characteristics of a registered user or an unregistered person are stored locally, and in other cases are stored remotely, for example, because the person may need their radar characteristics to be stored only locally to address privacy issues. For example, a previous visitor (e.g., a guest who has previously been detected by a computing device 102 at home) may have stored radar characteristics, but no rights associated with a registered person. In another example, a new visitor (e.g., a visitor who encounters the computing device 102 for the first time) may not have stored radar characteristics and no rights associated with a registered person.

[0022] After determining that the second user 104-2 is an unregistered person, the computing device 102 assigns an unregistered user identification (e.g., a false identity, a pseudo identity) to the unregistered person, and the unregistered person can be associated with one or more radar characteristics of the second radar reception signal. The unregistered user identification can be stored to enable identification of the second user 104-2 at a future time. Specifically, the unregistered user identification can be used to correlate future received radar reception signals with one or more associated radar characteristics of the unregistered person.

[0023] Similar to the technique performed in example sequence 100-1, computing device 102 may forgo requiring personally identifiable information of second user 104-2 to determine that another object is an unregistered person. After second user 104-2 has been distinguished from first user 104-1 in example sequence 100-2, computing device 102 may determine that it is necessary to adjust (e.g., modify, restrict) the privacy settings of first user 104-1 to ensure that the first user's information remains private. For example, when second user 104-2 is present, first user 104-1 may want the device to avoid broadcasting calendar reminders (e.g., doctor's appointments). In addition, computing device 102 may prompt second user 104-2 to start gesture training, which may be recorded in a second training history of second user 104-2 (e.g., associated with an unregistered user identification). Although mentioned in the context of second user 104-2, the technique does not necessarily require personally identifiable information of first user 104-1. Thus, when a person with stored radar characteristics registers and thus becomes a registered user, the registration may give up personally identifiable information, such as the person's name. In this case, permissions to computing device 102 or devices, systems, or applications associated with computing device 102 may be obtained through the registration. However, in many cases, these permissions will be lower than those potentially granted to a registered user who has provided personally identifiable information, such as permissions to access financial applications.

[0024] For example, consider a situation where a registered user has not provided personally identifiable information, such as the person's name or other unique identifier (e.g., a social security number in the United States). The registered user may still have greater privileges than an unregistered user. One such situation is when, as part of registering with computing device 102, the person provides a password or other code that is associated with computing device 102 rather than the person. For example, computing device 102 may include a code or other indicator available to the purchaser of computing device 102, such as within the box in which the computing device is located when the computing device 102 is purchased. In essence, computing device 102 may have a code or information available to the person to indicate that the person has additional privileges compared to a stranger. While this may not be sufficient to allow access to highly sensitive or personally specific applications and accounts, such as financial accounts, it may allow access to control devices and applications associated with a home or car in which computing device 102 is placed or associated with the computing device. Thus, upon submitting a password sold with computing device 102 or another device, such as a serial number for an oven or stereo, the privileges associated with the device may be granted to the registered user upon submitting one of these identifiers.

[0025] By further example, assume that a person purchases a VA-equipped device to manage their home. The VA-equipped device includes a password. When the VA-equipped device obtains or is equivalent to the person's radar characteristics, the person submits the password. This then allows the now-registered user to control the home through VA-equipped devices (such as stereos, ovens, door locks, thermostats, and home security systems). In this example, these permissions are enabled without the need for the person's name or other unique identifier. In this way, the person can maintain their privacy and still have permissions to control devices and applications through their VA-equipped device (e.g., computing device 102).

[0026] Returning to the previous example, suppose that at a later time ( Figure 1 104-2) is present in the vicinity 106. If there is a user 104 (e.g., the first user 104-1, the second user 104-2) at this time, the third radar transmission signal can be reflected from the user 104, and the computing device 102 can receive a third radar reception signal including radar characteristics. The radar system 108 can compare these radar characteristics with, for example, the stored radar characteristics of the first user 104-1 (a registered user) and the second user 104-2 (an unregistered person associated with an unregistered user identifier) ​​to determine whether the first user 104-1 or the second user 104-2 is present. Based on this determination, the computing device 102 can customize settings and training prompts accordingly, such as those that are different based on the first user 104-1 being registered (e.g., having an account with the computing device 102) and the second user 104-2 not being registered (e.g., being a guest or friend of a registered user).

[0027] In the example, radar system 108 uses the third radar reception signal to determine that first user 104-1 (registered user) is again present within nearby area 106 based on the stored radar characteristics of the first user 104-1. Computing device 102 may then prompt first user 104-1 to complete their gesture training and / or activate their user settings based on the first training history. Alternatively, if radar system 108 determines that second user 104-2 (unregistered person) is again present within nearby area 106, computing device 102 may prompt second user 104-2 to continue their gesture training and / or activate predetermined user settings based on the second training history. Figure 2 Computing device 102 and radar system 108 are further described. Example computing device

[0028] Figure 2An example implementation 200 of a radar system 108 as part of a computing device 102 is shown. The computing device 102 is shown with various non-limiting example devices 202, including a home automation and control system 202-1, a desktop computer 202-2, a tablet computer 202-3, a laptop computer 202-4, a television 202-5, a computing watch 202-6, computing glasses 202-7, a gaming system 202-8, and a microwave oven 202-9. Other devices may also be used, including home service devices, smart speakers, smart thermostats, security cameras, baby monitors, Wi-Fi ® A router, a drone, a trackpad, a drawing tablet, a netbook, an e-reader, a home automation and control system, a wall display, a virtual reality headset, a vehicle, and another home appliance. Note that computing device 102 can be wearable, non-wearable but mobile, or relatively non-mobile (e.g., desktop computers and appliances).

[0029] The computing device 102 may include one or more processors 204 and one or more computer readable media (CRM) 206, which may include memory media and storage media. Applications and / or operating systems (not shown) embodied as computer readable instructions on the CRM 206 may be executed by the processor 204 to provide some of the functionality described herein. The CRM 206 may also include a radar-based application 208 that uses data generated by the radar system 108 to perform functions, such as gesture-based control. For example, the radar system 108 may detect a gesture performed by the user 104 that indicates a command to turn off a light in a room. The radar-based application 208 may use the command data to send a control signal (e.g., a trigger) to turn off the light in the room.

[0030] The computing device 102 may also include a network interface 210 for transmitting data through a wired, wireless, or optical network. For an interconnected system of multiple computing devices 102-X, each computing device 102 may communicate with another computing device 102 through the network interface 210. For example, the network interface 210 may transmit data through a local area network (LAN), a wireless local area network (WLAN), a personal area network (PAN), a wired area network (WAN), an intranet, the Internet, a peer-to-peer network, a point-to-point network, a mesh network, etc. Multiple computing devices 102-X may use a network interface 210 such as the one described above. Figure 3 The described communication networks are in communication with each other. Computing device 102 may also include a display (not shown).

[0031] Radar system 108 may be used as a stand-alone radar system or used with or embedded in many different computing devices or peripherals, such as embedded in a control panel that controls home appliances and systems, embedded in an automobile to control internal functions (e.g., volume, cruise control, or even the steering of the car), or as an accessory to a laptop computer to control computing applications on the laptop computer.

[0032] Radar system 108 may include communication interface 212 to transmit radar data (e.g., radar characteristics) to a remote device, but this communication interface may not be used when radar system 108 is integrated within computing device 102. Generally speaking, the radar data provided by communication interface 212 may be in a format usable by radar-based application 208.

[0033] The radar system 108 may also include at least one antenna 214 for transmitting and / or receiving radar signals. In some cases, the radar system 108 may include multiple antennas 214 implemented as antenna elements of an antenna array. The antenna array may include at least one transmitting antenna element and at least one receiving antenna element. In some cases, the antenna array may include multiple transmitting antenna elements to implement a multiple input multiple output (MIMO) radar capable of transmitting multiple different waveforms (e.g., each transmitting antenna element transmits a different waveform) at a given time. For implementations including three or more receiving antenna elements, the receiving antenna elements may be positioned in a one-dimensional shape (e.g., a line) or a two-dimensional shape (e.g., a triangle, a rectangle, or an L-shape). A one-dimensional shape may enable the radar system 108 to measure an angular dimension (e.g., an azimuth or an elevation), while a two-dimensional shape may enable the measurement of two angular dimensions (e.g., both an azimuth and an elevation). Each antenna 214 may be configured as a transducer or a transceiver alternatively. In addition, any one or more antennas 214 may be circularly polarized, horizontally polarized, or vertically polarized.

[0034] Using antenna arrays, the radar system 108 can form a beam that is steered or non-steered, wide or narrow, or shaped (e.g., as a hemisphere, cube, sector, cone, or cylinder). One or more transmitting antenna elements may have a non-steered omnidirectional radiation pattern, or may be capable of producing a wide steerable beam. Each of these technologies can enable the radar system 108 to illuminate a large area of ​​space. In order to achieve target angular accuracy and angular resolution, the receiving antenna elements can be used to generate thousands of narrow steered beams (e.g., 2000 beams, 4000 beams, or 6000 beams) through digital beamforming. In this way, the radar system 108 can efficiently monitor the external environment and detect gestures from one or more users 104.

[0035] The radar system 108 may also include at least one analog circuit 216 including circuitry and logic for transmitting and receiving radar signals using at least one antenna 214. The components of the analog circuit 216 may include amplifiers, mixers, switches, analog-to-digital converters, filters, etc. for conditioning radar signals. The analog circuit 216 may also include logic for performing in-phase / quadrature (I / Q) operations such as modulation or demodulation. A variety of modulations may be used to generate radar signals, including linear frequency modulation, triangular frequency modulation, step frequency modulation, or phase modulation. The analog circuit 216 may be configured to support continuous wave or pulse radar operation.

[0036] The analog circuit 216 can generate a radar signal in a spectrum (e.g., a frequency range) including frequencies between 1 and 400 gigahertz (GHz), between 1 and 24 GHz, between 2 and 6 GHz, between 4 and 100 GHz, or between 57 and 63 GHz. In some cases, the spectrum can be divided into multiple sub-spectra with similar or different bandwidths. Example bandwidths may be approximately 500 megahertz (MHz), 1 gigahertz (GHz), 2 gigahertz, etc. Different sub-spectra may include, for example, frequencies between approximately 57 and 59 GHz, 59 and 61 GHz, or 61 and 63 GHz. Although the example sub-spectra described above are continuous, other sub-spectra may be discontinuous. In order to achieve consistency, the analog circuit 216 can use multiple sub-spectra (continuous or discontinuous) with the same bandwidth to generate multiple radar signals, which can be transmitted simultaneously or separately in time. In some cases, multiple continuous sub-spectra may be used to transmit a single radar signal, thereby enabling the radar signal to have a wider bandwidth.

[0037] The radar system 108 may also include one or more system processors 218 and system media 220 (e.g., one or more computer-readable storage media). The system processor 218 may be implemented within the analog circuit 216 as, for example, a digital signal processor or a low-power processor. The system processor 218 may execute computer-readable instructions stored within the system media 220. Example digital operations performed by the system processor 218 may include fast Fourier transforms (FFTs), filtering, modulation or demodulation, digital signal generation, digital beamforming, etc.

[0038] System medium 220 may optionally include user detection module 222 and gesture detection module 224, which may be implemented using hardware, software, firmware, or a combination thereof. User detection module 222 and gesture detection module 224 may enable radar system 108 to process radar receive signals (e.g., electrical signals received at analog circuit 216) to detect the presence of user 104 and to detect gesture commands performed by user 104, respectively.

[0039] The user detection module 222 and the gesture detection module 224 may include one or more artificial neural networks (referred to herein as neural networks) to improve user identification and gesture recognition, respectively. A neural network may include a set of connected nodes (e.g., neurons or perceptrons) organized into one or more layers. As an example, the user detection module 222 and the gesture detection module 224 may include a deep neural network including an input layer, an output layer, and one or more hidden layers located between the input layer and the output layer. The nodes of the deep neural network may be partially connected or fully connected between layers.

[0040] In some cases, the deep neural network can be a recurrent deep neural network (e.g., a long short-term (LSTM) recurrent deep neural network) in which the connections between nodes form a loop to retain information from a previous portion of the input data sequence for a subsequent portion of the input data sequence. In other cases, the deep neural network can be a feedforward deep neural network in which the connections between nodes do not form a loop. Additionally or alternatively, the user detection module 222 and the gesture detection module 224 can include another type of neural network, such as a convolutional neural network. Reference Figure 7 An example deep neural network is further described. The user detection module 222 and the gesture detection module 224 may also include one or more types of regression models, such as a single linear regression model, a multivariate linear regression model, a logistic regression model, a stepwise regression model, a multivariate adaptive regression spline, a locally estimated scatterplot-smoothing model, and the like.

[0041] In general, the machine learning architecture can be customized based on available power, available memory, or computing power. For user detection module 222, the machine learning architecture can also be customized based on the number of radar features that radar system 108 is designed to recognize. For gesture detection module 224, the machine learning architecture can be customized based on the number of gestures that radar system 108 is designed to recognize.

[0042] Computing device 102 may optionally (not depicted) include at least one additional sensor (other than antenna 214) to improve the fidelity of user detection module 222 and / or gesture detection module 224. In some cases, for example, user detection module 222 may distinguish the presence of user 104 with a lower certainty (e.g., an amount of certainty below a threshold). This may occur, for example, when user 104 is far away from computing device 102 or when a large object (e.g., furniture) obscures user 104. To improve the accuracy of user identification, computing device 102 may use information such as that regarding Figure 8 The one or more additional sensors described above may be used to verify the low-certainty result. The additional sensors may include, for example, microphones, speakers, ultrasonic sensors, ambient light sensors, cameras, barometers, thermostats, optical sensors, etc. Similar to the operations performed by radar system 108, these sensors may forgo detecting, collecting, or enabling storage of a human identification of user 104 while distinguishing the user from other users of computing device 102. Multiple computing devices 102-X may also be connected (e.g., wirelessly) to create a computing system, such as reference 102. Figure 3 Further described. Example computing system

[0043] Figure 3 An example environment 300 is shown in which multiple computing devices 102-X are connected via a communication network 302 to form a computing system. The example environment 300 depicts a home having a first room 304-1 (living room) and a second room 304-2 (kitchen). The first room 304-1 is equipped with a first computing device 102-1 including a first radar system 108-1, and the second room 304-2 is equipped with a second computing device 102-2 including a second radar system 108-2. In this example, the first room 304-1 is separated from the second room 304-2, but connected by a door in the home. The first computing device 102-1 in the first room 304-1 can detect users 104 and gestures within a first vicinity 106-1. And the second computing device 102-2 in the second room 304-2 can detect users 104 and gestures within a second vicinity 106-2.

[0044] The home of the example environment 300 may not be limited to the arrangement and number of computing devices 102 shown. In general, an environment (e.g., a home, a building, a workplace, a car, an airplane) may include one or more computing devices 102 distributed across one or more rooms 304. For example, a room 304 may include two or more computing devices 102 positioned close to or far from each other in the room 304. Although the first proximate area 106-1 depicted in the example environment 300 does not spatially overlap with the second proximate area 106-2, in general, the proximate areas 106 may also be positioned to partially overlap. Although Figure 3 The environment depicted in the example is a home, but in general, the environment may include any private or public indoor and / or outdoor space, such as a library, office, workplace, factory, garden, restaurant, terrace, airplane, vehicle, etc.

[0045] For environments with two or more computing devices 102, these devices can communicate with each other through one or more communication networks 302. The communication network 302 can be a LAN, WAN, mobile or cellular communication network, extranet, intranet, Internet, Wi-Fi, or other network. ® network, etc. In some embodiments, the computing device 102 may communicate using short-range communications such as, for example, near field communication (NFC), radio frequency identification (RFID), Bluetooth, etc. In addition, the computing system may include one or more memories that are separate from or integrated into one or more of the constituent computing devices 102-1 or 102-2. In one example, the first computing device 102-1 and the second computing device 102-2 may include a first memory and a second memory, respectively, wherein the contents of each memory are shared between devices using the communication network 302. In another example, the memory may be separate from the first computing device 102-1 and the second computing device 102-2 (e.g., cloud storage), but accessible to both devices. The memory may be used, for example, to store radar characteristics, user preferences, security settings, training history, unregistered user identification, etc. of the user 104.

[0046] In an example, the first computing device 102-1 may connect to the communication network 302 using the first network interface 210-1 to exchange information with the second computing device 102-2. Using the communication network 302, the computing devices 102 may exchange stored information about one or more users 104, which may include radar characteristics, training history, user settings, etc. In addition, the computing devices 102 may exchange information about ongoing operations (e.g., timers, music being played) to maintain continuity of operations and / or information about operations across various rooms 304. These operations may be performed simultaneously or independently by one or more computing devices 102, for example, based on the detection of the presence of a user in the room 304. The computing devices 102 may use the radar system 108 to detect the presence of a user, such as regarding Figure 4 Further described. Radar-enabled user detection

[0047] Figure 4An example environment 400 is shown in which a radar system 108 is used by a computing device 102 to detect the presence of a user 104. Similar to the discussion regarding the example sequence 100, the radar system 108 may transmit one or more radar transmit signals 402 to detect a vicinity 106 of the user 104. For example, the radar system 108 may illuminate the user 104 entering the vicinity 106 with a wide 150° radar pulse beam (e.g., radar transmit signals) operating at a frequency of 1 to 10 kilohertz (kHz). Although reference may be made to the radar transmit signals 402 in the present disclosure, it should be understood that the radar system 108 may also transmit a group of radar transmit signals 402 over a period of time.

[0048] When encountering user 104, a portion of the energy associated with radar transmit signal 402 may be reflected back to radar system 108 in one or more radar receive signals 404-Y (where Y may represent an integer value of 1, 2, 3, ...). In example environment 400, three radar receive signals 404-1, 404-2, and 404-3 are depicted as being reflected from three discrete dynamic scattering centers. Each radar receive signal 404 may represent a modified version of radar transmit signal 402, where the amplitude, time, phase, or frequency is modified at each dynamic scattering center. The superposition of these radar receive signals 404 may allow radar system 108 to distinguish users 104 using, for example, radial distance, geometry (e.g., body shape, shape, height), orientation, surface texture, material composition, etc., as described with respect to FIG. Figure 5 Further described.

[0049] Figure 5 An example implementation 500 of the antenna 214, analog circuit 216, and system processor 218 of the radar system 108 is shown. In the depicted configuration, the analog circuit 216 can be coupled between the antenna 214 and the system processor 218 to implement both user detection and gesture detection techniques. The analog circuit 216 can include a transmitter 502 equipped with a waveform generator 504 and a receiver 506 including at least one receive channel 508. The waveform generator 504 and the receive channel 508 can each be coupled between the antenna 214 and the system processor 218.

[0050] Although one antenna 214 is depicted in the example implementation 500, in general, the radar system 108 may include one or more antennas to form an antenna array. When utilizing an antenna array, the waveform generator 504 may generate similar or different waveforms for each antenna 214 to transmit into the vicinity 106. Additionally, although one receive channel 508 is depicted in the example implementation 500, in general, the radar system 108 may include one or more receive channels. Each receive channel 508 may be configured to accept a single or multiple radar receive signals 404-Y at any given time.

[0051] During operation, the transmitter 502 may communicate an electrical signal to the antenna 214, which may transmit one or more radar transmit signals 402 to detect user presence and / or gesture commands in the vicinity 106. Specifically, the waveform generator 504 may generate an electrical signal having a specified waveform (e.g., a specified amplitude, phase, frequency). The waveform generator 504 may additionally transmit information about the electrical signal to the system processor 218 for digital signal processing. If the radar transmit signal 402 interacts with the user 104, the radar system 108 may receive one or more radar receive signals 404-Y on the receive channel 508. These radar receive signals 404-Y may be sent to the system processor 218 to enable user detection (using the user detection module 222) and / or gesture detection (using the gesture detection module 224). The user detection module 222 may determine whether the user 104 is within the vicinity 106, and then distinguish the user 104 from other users. The user 104 may be distinguished (but not personally identified) based on the one or more radar receive signals 404-Y, such as with respect to Figure 6 Further described. Radar-enabled user identification

[0052] Figure 6 Example implementations 600-1 through 600-4 are shown in which the user detection module 222 can distinguish between users 104. The user detection module 222 can utilize, in part, one or more radar reception signals 404 to distinguish, for example, a first user 104-1 from a second user 104-2 without personally identifying the first user 104-1 or the second user 104-2 (e.g., legal name, facial recognition information, biometric information, personal address, personal mobile device, etc.). By distinguishing between users 104, the user detection module 222 can enable the computing device 102 to provide a customized experience for each user 104, such as retrieval of training history, preferences, privacy settings, etc. In this way, the computing device 102 can improve some VA-equipped devices by meeting each user's privacy and functionality expectations.

[0053] To distinguish the user 104, the user detection module 222 may analyze the radar received signal 404 to determine (1) topological identification, (2) temporal identification, (3) gesture identification, and / or (4) contextual identification of the user 104. The user detection module 222 is not limited to Figure 6 The four discrimination categories depicted in FIG. 1 and may include other radar characteristics and / or categories not shown. In addition, the four discrimination categories are shown as example categories and may be combined and / or modified to include subcategories that implement the techniques described herein.

[0054] In example implementation 600-1, user detection module 222 may use topological information in part to distinguish users 104 when detecting that they are located in nearby area 106. The topological information may include RCS data, which includes, for example, the height, shape, or body shape of user 104. For example, first user 104-1 (e.g., father) may be significantly larger than second user 104-2 (e.g., child). When father and child enter nearby area 106, radar system 108 may acquire radar reception signals 404 indicating the presence of each user. These radar reception signals 404 may partially include radar characteristics indicating the height, shape, or body shape of each user. In this example, the father's radar characteristics may be different from those of the child. Then, user detection module 222 may compare the radar characteristics of each user with the stored radar characteristics to determine whether father and child are registered users, unregistered persons with stored radar characteristics, or unregistered persons without stored radar characteristics.

[0055] In this example, user detection module 222 can determine that first user 104-1 (father) is a registered user who has previously interacted with computing device 102. Specifically, user detection module 222 can correlate the father's stored radar characteristics (e.g., saved to a memory shared by multiple computing devices 102-X) with one or more radar reception signals 404 to determine that he is a registered user. Upon determining that first user 104-1 is a registered user, computing device 102 can activate the father's settings, prompt the father to continue gesture training (based on the father's training history), etc.

[0056] The user detection module 222 may also determine that the second user 104-2 (child) is an unregistered person who has not previously interacted with the computing device 102. Specifically, the user detection module 222 may compare the stored radar characteristics of the registered users and the unregistered persons with the one or more radar received signals 404 that include the radar characteristics of the second user 104-2. Upon determining that the radar characteristics of the second user 104-2 are not correlated with each of the stored radar characteristics (e.g., assuming a certain level of fidelity), the user detection module 222 may determine that the child is an unregistered person. Instead of personally identifying the child, the radar system 108 assigns the child an unregistered user identification that includes these radar characteristics so that the child can be distinguished from other users (e.g., the father) at a future time. The computing device 102 may also prompt the child to start gesture training and / or implement predetermined settings (e.g., standard preferences, settings programmed by the owner of the computing device 102).

[0057] In general, stored radar characteristics of registered users (and unregistered persons) may be collected and saved. For example, radar system 108 may record radar characteristics each time user 104 interacts with computing device 102 to improve user identification. The stored radar characteristics of user 104 may include topological, temporal, gesture, and / or contextual information inferred from one or more radar received signals 404 associated with the user 104. In addition, radar system 108 may utilize one or more models used by user detection module 222 to distinguish each user 104 based on their corresponding radar characteristics. These one or more models may include machine learning (ML) models, predicate logic, hysteresis logic, etc. to improve user identification.

[0058] In example implementation 600-2, user detection module 222 may use temporal information in part to distinguish users 104 when detecting their location within proximity 106. Unlike conventional radar detectors, which may require high spatial resolution, radar system 108 of the present disclosure may rely more on temporal resolution (rather than spatial resolution) to detect and distinguish users 104. In this manner, radar system 108 may detect movement of users 104 into proximity 106 by detecting, for example, motion signatures (e.g., the unique way in which users 104 typically move). Motion signatures may include gait (depicted in the graph of example implementation 600-2), limb movements (e.g., corresponding arm movements), weight distribution, breathing characteristics, unique habits, and the like. For example, a user's motion signature may include a limp, an energetic gait, pigeon-toed gait, knock-kneed, bow-legged, and the like. Using this information, user detection module 222 may be able to detect a user's motion (e.g., their hand movements) without identifying details that may be considered private (e.g., facial features to enable facial recognition). Detection of motion signatures by radar system 108 may enable a user to maintain greater anonymity when compared, for example, to devices performing facial recognition or speech-to-text technology.

[0059] In example implementation 600-3, user detection module 222 may also use, in part, gesture performance information to distinguish users 104. User detection module 222 may communicate with gesture detection module 224 when utilizing radar characteristics associated with gesture performance. A user may perform a gesture (e.g., a push-pull gesture) in a unique manner that helps distinguish that user 104. For example, a push-pull gesture may include pushing a user's hand in one direction and then immediately pulling their hand in the opposite direction. Although radar system 108 may expect push-pull motions to be complementary (e.g., equal range of motion, equal velocity), user 104 may perform motions that are different than expected. Each user may perform the gesture in a unique manner that is recorded on the device for user identification and gesture interpretation.

[0060] As depicted in example implementation 600-3, a first user 104-1 (e.g., a father) may perform a push-pull gesture differently when compared to a second user 104-2 (e.g., a child). For example, the first user 104-1 may push their hand to a first range (e.g., distance) at a first rate, but pull their hand back to a second range at a second rate. The second range may include a shorter distance than the first range, and the second rate may be much slower than the first rate. The radar system 108 may be configured to recognize this unique push-pull gesture based on the first user's training history (if available).

[0061] When distinguishing the first user 104-1, the radar system 108 may receive one or more radar receive signals 404, which include radar characteristics of the first user 104-1 associated with its execution of the push-pull gesture. The user detection module 222 may compare these radar characteristics with the stored radar characteristics of the registered users to determine whether there is a correlation. If there is a correlation (e.g., assuming a certain level of fidelity), the user detection module 222 may determine that the first user 104-1 is a registered user (father) based on the execution of the push-pull gesture. Similar to the discussion of the example implementation 600-1, the computing device 102 may then activate the father's settings, prompt the father to continue gesture training, etc. In this example, it is assumed that the father has performed a push-pull gesture at least once in the past, and the radar characteristics of the execution are recorded in the father's training history to subsequently partially implement the discrimination of the father's presence from the presence of other users.

[0062] As depicted in example implementation 600-3, second user 104-2 (child) may also attempt to perform a push-pull gesture. Second user 104-2 may push their hand to a first range at a first rate, but pull their hand back to a third range at a third rate. In this example, the third range may be much larger than the first range, and the third rate may be much faster than the first rate. Specifically, radar system 108 may receive one or more radar receive signals 404, which include radar characteristics of the second user 104-2 associated with the execution of the push-pull gesture. User detection module 222 may compare these radar characteristics with stored radar characteristics to determine whether there is a correlation. Similar to the discussion about example implementation 600-1, user detection module 222 may determine that the push-pull gesture of second user 104-2 is not related to the stored radar characteristics (e.g., training history) of a registered user. Therefore, user detection module 222 may determine that the child is an unregistered person and assign an unregistered user identifier to the child. However, the radar characteristics associated with the child's push and pull gestures may be included in the unregistered user identification to enable future identification.

[0063] The user detection module 222 may also use context information in part to distinguish the user 104. The context information may include relevant details (e.g., complementary to radar characteristics) determined by the user detection module 222 using, for example, the antenna 214, another sensor of the computing device 102, data stored on the memory (e.g., user habits), local information (e.g., time, relative position), etc. In the example implementation 600-4, the user detection module 222 may use the local time as the context to achieve recognition of the user's presence. If the user 104 (e.g., the father) continues to sit on the sofa in the living room at 5:30 p.m. every day, the user detection module 222 may note the habit to improve user recognition. Whenever the user 104 is detected on the sofa at 5:30 p.m., the radar system 108 may use the context information in part to distinguish the user 104 as the father. In another example, if the computing device 102 is located in a child's room, the radar system 108 may determine over time that the child is the most common user in the room. The context information may be used to achieve user recognition. Similarly, if computing device 102 is located in a shared space (e.g., backyard, entryway), radar system 108 may determine over time that unregistered persons (e.g., guests, nannies, housekeepers, gardeners, friends) are common in the area.

[0064] The context information collected by the user detection module 222 can be used alone to distinguish users 104, or in combination with topological information, time information, and / or gesture information. In general, the user detection module 222 can use any one or more of the depicted identification categories in any combination at any time to distinguish users 104. For example, the radar system 108 can collect topological and time information about users 104 who have entered the nearby area 106 but lack gesture and context information. In this case, the user detection module 222 can distinguish users 104 based on the analysis of the topological information and time information. In another case, the radar system 108 can collect topological and time information, but determine that the information is not sufficient to correctly distinguish users 104 (e.g., to a certain level of certainty). If context information is available, the radar system 108 can use the context to distinguish users 104 (similar to the example implementation 600-4). Figure 6 Any one or more of the categories depicted may take precedence over another.

[0065] The user detection module 222 can utilize one or more logic systems (e.g., including predicate logic, hysteresis logic, etc.) to improve user identification. The logic system can be used to prioritize certain user identification techniques over other techniques (e.g., prioritize temporal identification over contextual information), add weights (e.g., confidence) to certain results when relying on two or more identification categories, etc. For example, the user detection module 222 can determine with a low certainty that the first user 104-1 may be a registered user. The logic system can determine that the low certainty is below an allowed threshold (e.g., a limit) and instead prompt the radar system 108 to issue a second radar transmission signal 402 (or a set of signals transmitted over a period of time) to again detect the nearby area 106. The user detection module 222 can also include one or more ML models to improve user identification, as shown in reference Figure 7 Further described.

[0066] Figure 7 An example implementation 700 of an ML model for distinguishing users 104 of a computing device 102 is shown. In the depicted configuration, the ML model is implemented as a deep neural network and may include an input layer 702, a plurality of hidden layers 704, and an output layer 706. The input layer 702 may include a plurality of inputs 708-1, 708-2, ... 708-N, where N represents a positive integer equal to the number of radar characteristics 710 associated with one or more radar receive signals 404. The plurality of hidden layers 704 may include layers 704-1, 704-2, ... 704-M, where M represents a positive integer. Each hidden layer 704 may include a plurality of neurons, such as neurons 712-1, 712-2, ... 712-Q, where Q represents a positive integer. Each neuron 712 may be connected to at least one other neuron 712 in the previous hidden layer 704 or the next hidden layer 704. The number of neurons 712 may be similar or different between different hidden layers 704. In some cases, the hidden layer 704 may be a copy of a previous layer (e.g., layer 704-2 may be a copy of layer 704-1). The output layer 706 may include outputs 714-1, 714-2, ... 714-N associated with differentiated users 716 (e.g., registered users, unregistered persons) that may be detected within the vicinity 106.

[0067] In general, a variety of different deep neural networks can be implemented with various numbers of inputs 708, hidden layers 704, neurons 712, and outputs 714. The number of layers within the ML model can be based on the radar characteristics and / or the number of classes to be distinguished (e.g. Figure 6As an example, the ML model may include four layers (e.g., one input layer 702, one output layer 706, and two hidden layers 704) to distinguish the first user 104-1 from the second user 104-2, as described with respect to the example sequence 100 and the example implementation 600. Alternatively, the number of hidden layers may be approximately 100.

[0068] When utilized by the user detection module 222, the ML model can improve the fidelity of user identification. The ML model can collect multiple inputs 708 (e.g., radar characteristics 710 associated with one or more radar reception signals 404) over time, the multiple inputs including topological, temporal, gesture, and / or contextual information about the user 104. For example, the second user 104-2 (e.g., a child) can be located far away from the radar system 108 during a first interaction with the computing device 102, thereby generating a first set of inputs 708 for distinguishing the child as an unregistered person. The first set of inputs 708 can be included in the unregistered user identification assigned to the child. In a second interaction, the child can be sitting closer to the radar system 108, thereby generating a second set of inputs 708 for distinguishing the child. The second set of inputs 708 is different from the first set and can also be included in the unregistered user identification of the child. This process can continue over time, thereby providing the ML model with more inputs 708 to better (e.g., with higher accuracy, faster rate) distinguish the child at a future time.

[0069] The gesture detection module 224 may also collect gesture execution information of the user 104 during training as an input to the ML model to enable the user detection module 222 to distinguish users based on gesture execution. If the first user 104-1 performs the push and pull gesture four times during training, the ML model may have at least four inputs 708-1, 708-2, 708-3, and 708-4. When the first user 104-1 (e.g., father) performs the push and pull gesture at a future time, the user detection module 222 may partially utilize one or more outputs 714 of the ML model to distinguish him.

[0070] In general, the ML model may be integrated into the user detection module 222, the radar system 108, the computing device 102, or located separately from the computing device 102 (e.g., a shared server). The gesture detection module 224 may also include a similar ML model that may improve the detection and recognition of gestures performed by the user 104. For example, the gesture detection module 224 may detect a push or pull gesture performed by the first user 104-1, and then interpret the gesture as a command using the ML model output 714. The operations of the gesture detection module 224 may be performed simultaneously with the operations performed by the user detection module 222 or at a separate time. Additional sensors (e.g., non-radar sensors) of the computing device 102 may also be used to improve the fidelity of user recognition, as described in reference to FIG. Figure 8 Further described. Optional sensor for improved fidelity of user identification

[0071] Figure 8 An example implementation 800 of a computing device 102 using an additional sensor (e.g., a microphone 802) to improve the fidelity of user identification is shown. In some cases, radar characteristics associated with nearby objects (e.g., registered users or unregistered persons) may not provide enough information to distinguish users 104 with a certain level of certainty. As depicted in the example implementation 800, the user detection module 222 may not be able to determine with certainty that the first user 104-1 is a registered user (e.g., a father) based solely on radar and / or the other characteristics described above. In this case, the computing device 102 can bootstrap an audio signal 804 (e.g., a sound wave emitted by the first user 104-1) to enable the radar system 108 to determine that the father is present.

[0072] As depicted in the example implementation 800, the user detection module 222 may receive the audio signal 804 via the microphone 802 and analyze the characteristics (e.g., wavelength, amplitude, time period, frequency, speed, velocity) of these sound waves to determine which user is present in the vicinity 106. The analysis may be performed automatically when triggered, concurrently with the radar characteristics analysis, or after the radar characteristics analysis, etc. The audio signal 804 may be modified by additional circuitry and / or components before being received by the user detection module 222.

[0073] User detection module 222 may analyze audio signal 804 to distinguish users without accessing private information (e.g., content of the conversation). For example, radar system 108 may characterize audio signal 804 to distinguish the presence of first user 104-1 without identifying the words spoken (e.g., performing speech-to-text). Radar system 108 may also characterize audio signal 804 in terms of pitch, loudness, tone, timbre, rhythm, consonance, dissonance, pattern, etc. As a result, users 104 may comfortably discuss private information near computing device 102 without having to worry about whether the device identifies the words, sentences, thoughts, etc. spoken.

[0074] The computing device 102 may store the audio detection characteristics of one or more users 104 (e.g., on a shared memory) to enable identification of the presence of users. When a registered user (e.g., a father) enters the vicinity 106 of the radar system 108, the user detection module 222 may then use, in part, the stored audio detection characteristics of the father to distinguish him from other users of the device. When an unregistered person enters the vicinity 106, the user detection module 222 may then use, in part, the stored audio detection characteristics of the registered user to determine that this is an unregistered person who has not yet provided an audio signal 804 to the computing device 102. The radar system 108 may then generate an unregistered user identification for the unregistered person, the unregistered user identification including the audio detection characteristics associated with the one or more audio signals 804 emitted by the unregistered person. Thus, the radar system 108 may be able to use the audio detection characteristics stored in the unregistered user identification of the unregistered person to distinguish the unregistered person at a later time.

[0075] In general, additional sensor inputs (e.g., audio signals 804 from microphone 802) are optional, and user 104 may be provided with privacy controls to limit the use of such additional sensors. For example, user 104 may modify their personal settings, general settings, default settings, etc. to include and / or exclude any additional sensors (e.g., sensors that supplement antenna 214 for radar). In addition, user detection module 222 may implement these personal settings when distinguishing the presence of a user. Although example implementation 800 describes the additional sensor as microphone 802, in general, the user 104 may modify their personal settings, general settings, default settings, etc. to include and / or exclude any additional sensors (e.g., sensors that supplement antenna 214 for radar). In addition, user detection module 222 may implement these personal settings when distinguishing the presence of a user. Figure 8 The described technology can be applied to, for example, ultrasonic sensors, ambient light sensors, accelerometers, gyroscopes, magnetometers, proximity sensors, ambient temperature sensors, light sensors, pressure sensors, touch sensors, etc. Fig. 9 Further describes privacy controls. Adaptive Privacy

[0076] Fig. 9Example sequences 900-1 and 900-2 are shown in which privacy settings are modified based on user presence. In example sequence 900-1, user detection module 222 of computing device 102 detects that first user 104-1 is present within proximity 106. Instead of detecting the presence of one or more other users (e.g., registered users, unregistered persons), radar system 108 may implement first privacy settings 902 for first user 104-1. The first privacy settings 902 may include information about, for example, allowed sensors (see Figure 8 ), audio reminders, calendar information, music, media, settings for home objects (e.g., lighting preferences), etc. For example, when the first privacy setting 902 is implemented, the first user 104-1 can receive audio reminders for calendar events.

[0077] In the example sequence 900-2, in addition to the continued presence of the first user 104-1, the computing device 102 may also detect the presence of the second user 104-2 (e.g., another registered user) at a later time. The radar system 108 may now implement a second privacy setting 904 to adjust the privacy of the first user 104-1 based on the presence of the second user. The implementation may be automatic or triggered based on a command from the first user 104-1. For example, the second privacy setting 904 may limit an audio reminder to prevent private information from being broadcast in the presence of others. The second privacy setting 904 may be based on, for example, a preset condition, a user input, etc.

[0078] The second privacy setting 904 may also be implemented to protect the privacy of the second user's information and may be adjusted based on the users in the room. For example, the presence of another registered user (e.g., a family member) may require fewer privacy restrictions than the presence of an unregistered person (e.g., a guest). Adaptive privacy settings may also be customized for each user 104. For example, the first user 104-1 may have a stricter privacy setting (e.g., limiting audio reminders in the presence of others), while the second user 104-2 may have a less strict privacy setting (e.g., not limiting audio reminders in the presence of others).

[0079] Adaptive privacy may also be applied to ongoing operations. In an example, a first user 104-1 (e.g., a father) may start an oven and set a timer for 20 minutes. If a second user 104-2 (e.g., a child) attempts to turn off the oven before the timer expires, the radar system 108 may prevent the child from changing the operation. This may allow the father to control the operation of the oven during those 20 minutes to prevent toasting interruptions. Specifically, the radar system 108 may associate the ongoing operation with the user 104 executing the command to prevent another user from modifying its operation. In another example, a mother may execute a command to turn off the bedroom lights at 9:00 p.m. to ensure that her child goes to bed on time. If the child executes a command to keep the lights on after bedtime, the radar system 108 may prevent the child from modifying the mother's command. Techniques for user identification of radar-based gesture detectors are not limited to Figures 1 to 9 The examples shown, and more generally about Figure 10-1 , Figure 10-2 and Figure 10-3 Give a description. Example Method

[0080] Figure 10-1 , Figure 10-2 and Figure 10-3 An example method 1000 for user identification of a radar-based gesture detector is shown. The method 1000 is shown as various sets of operations (or actions) performed and is not necessarily limited to the order or combination of operations shown herein. In addition, any one or more of the operations may be repeated, combined, reorganized, or linked to provide a wide range of additional methods and / or alternative methods. In the following discussion, reference may be made to Figures 1 to 9 The example environments or example sequences of the present invention are referenced by way of example only. The techniques are not limited to being performed by one entity or multiple entities operating on one computing device 102.

[0081] At 1002, a radar transmission signal is transmitted from a radar system of a computing device. Figure 1 , a radar transmission signal 402 may be transmitted from the radar system 108 of the computing device 102 to detect whether the user 104 is present in the nearby area 106. The radar transmission signal 402 may include a single signal, a plurality of similar or different signals, a burst of signals, a continuous signal, etc., as shown in FIG. Figure 4 described.

[0082] At 1004, a radar receive signal is received at a computing device. For example, radar system 108 of computing device 102 may receive one or more radar receive signals 404-Y (where Y may represent an integer value 1, 2, 3, ...).

[0083] At 1006, the computing device determines radar characteristics of the object from which the radar reception signal was reflected. For example, one or more radar characteristics of the object (e.g., user 104) from which radar reception signal 404-Y was reflected are determined by radar system 108 of computing device 102. The one or more radar characteristics may be used to distinguish between users and include, for example, information about Figure 6 The computing device 102 may forgo determining personally identifiable information of the user 104 detected in the nearby area 106. At the same time or at a different time, at 1008, the computing device may (optionally) receive an audio signal associated with the object. For example, the computing device 102 may use the microphone 802 to receive one or more audio signals 804 associated with the object (e.g., the user 104).

[0084] At 1010, the computing device may determine that the object is an unregistered person. For example, the user detection module 222 of the radar system 108 may determine the presence of an unregistered person based on one or more radar characteristics of the radar reception signal 404-Y. Specifically, the radar system 108 may compare the first radar characteristic of the first radar reception signal 404-1 with one or more stored radar characteristics. The stored radar characteristics may include one or more radar characteristics of a registered user or a previously detected unregistered person, which one or more radar characteristics may be saved to, for example, a memory. If the first radar characteristic is correlated with one or more stored radar characteristics, the user detection module 222 may determine that there is a registered user. However, at 1010, the radar system 108 determines that the detected radar characteristic is not correlated (e.g., not similar or identical) to the one or more stored radar characteristics. Therefore, the radar system 108 determines that this lack of correlation indicates the presence of an unregistered person. This may be done without requiring, for example, a determination as to whether the user is a registered user. Figure 1 The user detection module 222 may optionally utilize one or more audio signals 804 received by the computing device 102 to determine the presence of an unregistered person.

[0085] At 1012 , the unregistered person is assigned an unregistered user identification. For example, radar system 108 of computing device 102 may assign an unregistered user identification (eg, a mock identity, a pseudo identity) to the unregistered person.

[0086] Before proceeding to block 1020, the computing device may also optionally perform any one or more of the operations identified at 1014, 1016, and 1018. At 1014, the computing device may provide gesture training. For example, radar system 108 of computing device 102 may provide gesture training for unregistered persons, as described in reference to FIG. Figure 1For example, radar system 108 may assume that the new user (unregistered person) has not yet performed gesture training, and may prompt the unregistered person to begin training one or more gestures. At 1016, computing device may maintain a first training history. For example, computing device 102 may store one or more radar characteristics associated with the manner in which the unregistered person performed gestures during training to an unregistered user identification of the unregistered person.

[0087] At 1018, the computing device may optionally activate a predetermined user setting. Fig. 9 , the radar system 108 of the computing device 102 may activate predetermined user settings (e.g., default preferences, default privacy settings). The method 1000 is described in Figure 10-2 Continue, such as Figure 10-1 1012 is indicated by the letter "A" after the box 1012, which is the same as Figure 10-2 Corresponding to the letter “A” before box 1020 of .

[0088] At 1020, the radar characteristics of the object are associated with the unregistered user identification. For example, one or more radar characteristics of the object (e.g., an unregistered person) can be associated with the unregistered user identification. The unregistered user identification can also include the settings and training history of the unregistered person.

[0089] At 1022, the computing device stores the unregistered user identification and the associated radar characteristics. For example, radar system 108 of computing device 102 may store both the unregistered user identification and one or more radar characteristics associated with the unregistered person in a memory (e.g., local, shared, remote memory). In this manner, the unregistered user identification may be used, in part, to distinguish the unregistered person from other users at a future time.

[0090] The computing device 102 may optionally perform any one or more steps after block 1022. These steps may be performed in any order and / or may be repeated. Although blocks 1024 to 1048 are depicted as being after blocks 1002 to 1022, they may be performed before or during any one or more of the blocks 1002 to 1022.

[0091] At 1024, a second radar transmission signal is transmitted from the computing device. Figure 1 and Figure 4 , a second radar transmission signal 402 - 2 may be transmitted from the radar system 108 of the computing device 102 to detect whether another user 104 is within the nearby area 106 .

[0092] At 1026, a second radar receive signal is received at the computing device. For example, radar system 108 of computing device 102 may receive second radar receive signal 404-2. At the same time or at a different time, at 1028, the computing device may (optionally) receive a second audio signal. For example, computing device 102 may use microphone 802 to receive second audio signal 804-2.

[0093] At 1030, the computing device may determine the presence of a registered user based on the second radar reception signal. For example, the user detection module 222 of the radar system 108 may determine the presence of a registered user based on the second radar reception signal 404-2. Specifically, the radar system 108 may compare the second radar characteristic of the second radar reception signal 404-2 with one or more stored radar characteristics. If the second radar characteristic is correlated with one or more stored radar characteristics, the user detection module 222 may determine that a registered user (or an unregistered user with stored radar characteristics) exists. At 1030, the radar system 108 determines that the detected radar characteristic is correlated with (e.g., similar or identical to) one or more stored radar characteristics of a registered user. Therefore, the radar system 108 determines the presence of a registered user. The presence of a registered user may also be determined without the need for personally identifiable information, such as with respect to Figure 1 The user detection module 222 may optionally utilize the second audio signal 804 - 2 received by the computing device 102 to determine the presence of a registered user.

[0094] Optionally, the computing device may perform any one or more of the operations identified at 1032, 1034, and 1036 before proceeding to block 1038. At 1032, the computing device may provide gesture training. For example, radar system 108 of computing device 102 may provide gesture training to registered users, as described in reference to FIG. Figure 1 As described. Gesture training can be customized for a registered user based on the training history of the registered user. For example, if the registered user has previously trained one of the four gestures, the radar system 108 (e.g., using the gesture detection module 224 or another module of the system medium 220) can prompt the registered user to continue training the remaining three of the four gestures. In this way, training can be customized and prevent the registered user from retraining unnecessarily. When the registered user resumes training on the remaining three gestures (e.g., at 1032), the computing device 102 can maintain the second training history of the registered user.

[0095] At 1036, the computing device may optionally activate user settings of the registered user upon determining the presence of the registered user. For example, radar system 108 of computing device 102 may reference information about Fig. 9The discussion of activating user settings (e.g., preferences, privacy settings) of a registered user. These operations can also be performed for unregistered persons with radar characteristics (such as radar characteristics associated with the unregistered user identification of the unregistered person).

[0096] Method 1000 is described in Figure 10-3 Continue, such as Figure 10-2 The letter "B" after the box 1030 is shown, which is the same as Figure 10-3 The letter "B" preceding box 1038 corresponds to the letter "B".

[0097] At 1038, a third radar transmission signal is transmitted from the computing device. Figure 1 and Figure 4 , a third radar transmission signal 402 - 3 may be transmitted from the radar system 108 of the computing device 102 to detect whether there is an unregistered person or a registered user within the nearby area 106 .

[0098] At 1040, a third radar receive signal is received at the computing device. For example, radar system 108 of computing device 102 may receive third radar receive signal 404-3. At the same time or at a different time, at 1042, the computing device may (optionally) receive a third audio signal. For example, computing device 102 may use microphone 802 to receive third audio signal 804-3.

[0099] At 1044, the computing device may determine the presence of a registered user or an unregistered person based on the third radar reception signal. For example, user detection module 222 of radar system 108 may determine the presence of a registered user or an unregistered person based on one or more radar characteristics of third radar reception signal 404-3. Specifically, similar to the discussion of 1010 and 1030, radar system 108 may compare the third radar characteristic of third radar reception signal 404-3 with one or more stored radar characteristics.

[0100] Optionally, the computing device may perform any one or more of the operations identified at 1046 and 1048. At 1046, the computing device may activate a predetermined setting or a user setting of a registered user, respectively. For example, if (at 1044) the device detects the presence of an unregistered person, the radar system 108 may activate a predetermined user setting. Similarly, if (at 1044) the device detects the presence of a registered user, the radar system 108 of the computing device 102 may activate the user setting of the registered user. At 1048, the computing device may resume gesture training based on the first training history or the second training history, respectively. For example, if (at 1044) the device detects the presence of an unregistered person, the radar system 108 may provide gesture training to the unregistered person based on the first training history associated with the unregistered user identification. Similarly, if (at 1044) the device detects the presence of a registered user, the radar system 108 of the computing device 102 may provide gesture training to the registered user based on the second training history. in conclusion

[0101] Although techniques and apparatus for user recognition of radar-based gesture detectors have been described in language specific to features and / or methods, it is to be understood that the subject matter of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as example implementations of user recognition of radar-based gesture detectors.

[0102] Some examples are described below.

[0103] Example 1: A method includes: transmitting a radar transmit signal from a radar system associated with a computing device; receiving a radar receive signal at the radar system or another radar system associated with the computing device; determining, based on the radar receive signal, a radar characteristic of an object from which the radar receive signal was reflected; determining, based on the radar characteristic of the object, that the object is an unregistered person who is not a registered user associated with the computing device; assigning an unregistered user identification to the unregistered person; associating the radar characteristic of the object with the unregistered user identification; and storing the unregistered user identification and the associated radar characteristic, the associated radar characteristic being usable to determine the presence of the unregistered person at a future time by correlating future received radar receive signals having similar or identical radar characteristics with the associated radar characteristic of the unregistered person.

[0104] Example 2: As described in Example 1, the method further includes: transmitting a second radar transmission signal from the radar system or another radar system associated with the computing device; receiving a second radar reception signal associated with a second radar characteristic at the radar system or another radar system, the second radar characteristic being similar to or identical to the radar characteristic associated with the unregistered person; and comparing the second radar characteristic with the radar characteristic associated with the stored unregistered user identification, the comparison being used to determine the presence of the unregistered person.

[0105] Example 3: A method as described in any of the preceding examples, further comprising: transmitting a third radar transmit signal from the radar system or another radar system associated with the computing device; receiving a third radar receive signal associated with a third radar characteristic at the radar system or another radar system; and comparing the third radar characteristic with one or more stored radar characteristics of a registered user, the comparison being used to correlate the third radar characteristic with a stored first radar characteristic of the registered user, the correlation indicating the presence of the registered user.

[0106] Example 4: A method as described in any of the preceding examples, wherein the presence of an unregistered person or the presence of a registered user is determined without using personally identifiable information, the personally identifiable information comprising at least one of: facial recognition information; biometric information; content of a private conversation; legally identifiable information; identification of an electronic tag; identification of a personal electronic device; or private information.

[0107] Example 5: A method as described in any preceding example, wherein the radar characteristics include topological information of the object, the topological information including at least one of the following: radar cross section data; geometric information; texture information; surface smoothness; or structural configuration.

[0108] Example 6: A method as in any preceding example, wherein the radar characteristic includes temporal information of the object, the temporal information associated with one or more movements of the object over time.

[0109] Example 7: A method as described in any of the preceding examples, wherein the radar characteristic includes gesture information associated with the manner in which the object performs one or more gestures to control or change a display, function, or capability of a computing device or a display, function, or capability associated with the computing device.

[0110] Example 8: A method as described in any of the preceding examples, further comprising associating contextual information with an unregistered user identifier, the contextual information comprising at least one of: a state of an operation performed by the computing device at a current time, a past time, or a scheduled time; a location of the computing device; or recorded habits or preferences of a registered user or an unregistered person.

[0111] Example 9: A method as described in Example 8, wherein the contextual information includes recorded habits or preferences of unregistered persons, the recorded habits or preferences including time of day or day of week, and the method further includes determining the presence of unregistered persons at future times based on the stored associated radar characteristics and time of day or day of week.

[0112] Example 10: A method as described in any of the preceding examples, the method further comprising: in response to determining that the object is an unregistered person, providing gesture training to teach the unregistered person to perform gestures to control or change a display, function or capability of a computing device or a display, function or capability associated with a computing device; storing a training history of gestures performed by the unregistered person during gesture training for the unregistered person, the stored training history being associated with an unregistered user identifier; and in response to determining the presence of the unregistered person at a future time, performing any of the following: resuming gesture training based on the stored training history; or recognizing gestures received in the future as gestures for controlling or changing a display, function or capability based on the stored training history.

[0113] Example 11: A method as described in any of the preceding examples, wherein determining the presence of an unregistered person at a future time is further based on a machine learning model that utilizes associated radar characteristics of the unregistered person.

[0114] Example 12: A method as described in any of the preceding examples, further comprising: detecting a first gesture performed by an unregistered person at a computing device, the first gesture being associated with an operation to be performed by the computing device within a duration; implementing the operation; detecting a second gesture performed by another user during the duration at the computing device to change the operation; and avoiding changing the operation based on the second gesture being performed by another user who is not the unregistered person.

[0115] Example 13: A method as described in any of the preceding examples, the method further comprising: receiving an audio signal of an object at a computing device; determining audio characteristics of the audio signal without determining one or more words associated with the audio signal; and in response to determining that the object is an unregistered person: associating the audio characteristics with an unregistered user identification; and storing the associated audio characteristics so that the presence of the unregistered person can be determined at a future time by correlating future received audio signals having similar or identical audio characteristics with the associated audio characteristics.

[0116] Example 14: A method as described in any preceding example, wherein the radar system is configured to transmit and receive radar signals in a radio frequency range.

[0117] Example 15: A computing device comprising: at least one antenna; a radar system configured to transmit radar transmit signals and receive radar receive signals using the at least one antenna; at least one processor; and a computer-readable storage medium comprising instructions that, in response to being executed by the processor, direct the computing device to perform any of the methods described in Examples 1 to 14.

[0118] Example 16: A computing system comprising a first computing device and a second computing device, the first computing device and the second computing device being connected to a communication network so that: the first computing device or the second computing device is capable of performing any one of the methods described in Examples 1 to 14; and is capable of exchanging information between the first computing device and the second computing device, the information comprising at least one of the following: one or more detected radar characteristics; one or more stored radar characteristics; topological information; time information; gesture information; context information; or one or more audio signals.

Claims

1. A method comprising: transmitting a radar transmission signal from a radar system associated with a computing device; receiving a radar receive signal at the radar system or another radar system associated with the computing device; determining, based on the radar reception signal, a radar characteristic of an object from which the radar reception signal was reflected; determining, based on the radar characteristic of the object, that the object is an unregistered person, the unregistered person not being a registered user associated with the computing device; Assigning an unregistered user identification to the unregistered person; associating the radar characteristic of the object with the unregistered user identifier; as well as The unregistered user identification and the associated radar characteristic are stored, the associated radar characteristic being usable to determine the presence of the unregistered person at a future time by correlating future received radar receive signals having similar or identical radar characteristics with the associated radar characteristic of the unregistered person.

2. The method of claim 1, further comprising: transmitting a second radar transmission signal from the radar system or another radar system associated with the computing device; receiving, at the radar system or the other radar system, a second radar receive signal associated with a second radar characteristic that is similar or identical to the radar characteristic associated with the unregistered person; and The second radar characteristic is compared with radar characteristics associated with stored unregistered user identifications, the comparison being used to determine the presence of the unregistered person.

3. The method according to any preceding claim, further comprising: transmitting a third radar transmission signal from the radar system or the other radar system associated with the computing device; receiving, at the radar system or the other radar system, a third radar receive signal associated with a third radar characteristic; as well as The third radar characteristic is compared to one or more stored radar characteristics of the registered user, the comparison being used to correlate the third radar characteristic with a stored first radar characteristic of the registered user, the correlation indicating a presence of the registered user.

4. A method as claimed in any preceding claim, wherein the presence of the unregistered person is determined without using personally identifiable information, the personally identifiable information comprising at least one of: Facial recognition information; Biometric information; the content of private conversations; Legally identifiable information; The identification of the electronic tag; or Identification of personal electronic devices.

5. A method as claimed in any preceding claim, wherein the radar characteristics include topological information of the object, the topological information comprising at least one of: Radar cross section data; Geometric information; Texture information; Surface smoothness; or Structural configuration.

6. A method as claimed in any preceding claim, wherein the radar characteristics include temporal information of the object, the temporal information being associated with one or more movements of the object over time.

7. A method as claimed in any preceding claim, wherein the radar characteristic includes gesture information associated with the manner in which the object performs one or more gestures to control or change a display, function or capability of the computing device or a display, function or capability associated with the computing device.

8. A method as claimed in any preceding claim, further comprising associating contextual information with the unregistered user identification, the contextual information comprising at least one of: the status of an operation performed by the computing device at a current time, a past time, or a scheduled time; the location of the computing device; or The recorded habits or preferences of the registered user or the unregistered person.

9. The method of claim 8, wherein the contextual information includes the recorded habits or preferences of the unregistered person, the recorded habits or preferences including time of day or day of week and the method further includes determining the presence of the unregistered person at the future time based on the stored associated radar characteristics and the time of day or day of week.

10. The method of any preceding claim, further comprising: In response to determining that the subject is the unregistered person, providing gesture training to teach the unregistered person to perform gestures to control or change a display, function, or capability of or associated with the computing device; storing, for the unregistered person, a training history of gestures performed by the unregistered person during gesture training, the stored training history being associated with the unregistered user identifier; as well as In response to determining the presence of the unregistered person at the future time, any of the following is performed: Resuming gesture training based on the stored training history; or Future received gestures are recognized as the gesture for controlling or changing the display, function, or capability based on the stored training history.

11. The method of any preceding claim, wherein determining the presence of the unregistered person at the future time is further based on a machine learning model utilizing associated radar characteristics of the unregistered person.

12. The method of any preceding claim, further comprising: detecting, at the computing device, a first gesture performed by the unregistered person, the first gesture being associated with an operation to be performed by the computing device for a duration; Implementing the operation; detecting, at the computing device, a second gesture performed by another user during the duration to change the operation; as well as Based on the second gesture being performed by another user who is not the unregistered person, changing the operation is avoided.

13. The method of any preceding claim, further comprising: receiving, at the computing device, an audio signal of the subject; determining an audio characteristic of the audio signal without determining one or more words associated with the audio signal; as well as In response to determining that the subject is the unregistered person: Associating the audio characteristic with the unregistered user identifier; and The associated audio characteristics are stored to enable determination of the presence of the unregistered person at the future time by correlating future received audio signals having similar or identical audio characteristics with the associated audio characteristics.

14. A computing device comprising: at least one antenna; a radar system configured to transmit radar transmit signals and receive radar receive signals using the at least one antenna; at least one processor; as well as A computer-readable storage medium comprising instructions which, in response to being executed by the processor, direct the computing device to perform any one of the methods of claims 1 to 13.

15. A computing system comprising a first computing device and a second computing device, the first computing device and the second computing device being connected to a communication network such that: The first computing device or the second computing device is capable of performing any one of the methods of claims 1 to 13; and Information can be exchanged between the first computing device and the second computing device, the information comprising at least one of: one or more detected radar characteristics; one or more stored radar characteristics; Topological information; Time information; Gesture information; Contextual information; or One or more audio signals.

Images