A smart home device using a single radar transmission mode for activity recognition of active users and vital signs monitoring of inactive users.

Abstract

Various configurations are provided for non-contact human interaction and health monitoring using a single radar transmission modulation mode. The radar chirp may be output by a radar sensor operating in burst mode. The burst mode radar data stream may be monitored for non-contact human interactions performed by an active user. The burst mode radar data stream may be converted into a virtual continuous mode radar data stream. Health monitoring of an inactive user may be performed using the virtual continuous mode radar data stream.

Description

[Technical Field]

[0001] REFERENCE TO RELATED APPLICATIONS This application is related to PCT application No. PCT / US2019 / 031290, filed May 8, 2019, entitled "Sleep Tracking and Vital Sign Monitoring Using Low Power Radio Waves," U.S. patent application Ser. No. 16 / 990,714, filed August 11, 2020, entitled "Contactless Sleep Detection and Disturbance Attribution for Multiple Users," U.S. patent application Ser. No. 16 / 990,705, filed August 11, 2020, entitled "Contactless Sleep Detection and Disturbance Attribution," U.S. patent application Ser. No. 16 / 990,720, filed August 11, 2020, entitled "Contactless Cough Detection and Attribution," and U.S. patent application Ser. No. 16 / 990,720, filed August 11, 2020, entitled "Precision Sleep Tracking Using a Contactless Sleep Tracking System," This application is related to U.S. patent application Ser. No. 16 / 990,726, entitled "Initializing Sleep Tracking on a Contactless Health Tracking Device," filed August 11, 2020, U.S. patent application Ser. No. 16 / 990,746, entitled "Initializing Sleep Tracking on a Contactless Health Tracking Device," and PCT application PCT / US2020 / 048388, filed August 28, 2020, entitled "Precision Sleep Tracking Using a Contactless Sleep Tracking Device," the entire disclosures of which are incorporated herein by reference for all purposes. [Background technology]

[0002] background It may be convenient for a user to interact with a device without any physical contact with the device (or another input device). For example, hand gestures may be performed, detected by the device, and interpreted as commands. While such contactless interaction is useful, the device's ability to perform other tasks without contact may also be desirable. Such other tasks may have competing requirements that must be balanced with the detection of contactless interaction. Summary of the Invention [Means for solving the problem]

[0003] overview Various embodiments related to a contactless health monitoring device are described. In some embodiments, a contactless health monitoring device is described. The device may include a housing. The device may include a radar sensor housed by the housing, where the radar sensor is configured to operate in a burst mode, in which the radar sensor emits multiple bursts of radar chirps. A first amount of time elapsed between radar chirps of a burst of the multiple bursts of radar chirps may be less than a second amount of time elapsed between subsequent bursts of the multiple bursts of radar chirps. The radar sensor outputs a burst mode radar data stream that may be based on radar chirp reflections of the multiple bursts of radar chirps. The device may include a processing system housed by the housing, including one or more processors, that may be in communication with the radar sensor. The processing system may be configured to receive the burst mode radar data stream from the radar sensor. The processing system may be configured to analyze the burst mode radar data stream to identify contactless human interactions. The processing system may be configured to convert the burst mode radar data stream into a virtual continuous mode radar data stream. The processing system may be configured to perform health monitoring of a user using the virtual continuous mode radar data stream.

[0004] Embodiments of such a device may include one or more of the following features: the virtual continuous mode radar data stream may be composed of multiple virtual returns of radar chirps equally spaced in time; configuring the processing system to convert the burst mode radar data stream to the virtual continuous mode radar data stream may include configuring the processing system to create a virtual return of the radar chirp based on multiple radar chirps of a burst of the multiple bursts; the virtual return of the radar chirp may be part of multiple virtual returns of radar chirps equally spaced in time; configuring the processing system to create a virtual return of the radar chirp based on multiple radar chirps of the burst of the multiple bursts may include configuring the processing system to perform an averaging process; the processing system may include sampling multiple samples of each radar chirp of the multiple chirps of the burst; the processing system may include averaging each sample of the multiple samples with corresponding samples from other chirps of the multiple chirps of the burst to create multiple averaged samples; and the processing system may include assembling the averaged samples to create a virtual return of the radar chirp. A non-contact human interaction may be detected while the health monitoring is being performed. The radar sensor may output a frequency modulated continuous wave (FMCW) radar. The non-contact human interaction may be a gesture. The non-contact human interaction may be presence detection. The health monitoring may include user sleep monitoring. The virtual continuous mode radar data stream may have higher resolution than a burst mode radar data stream due to dithering. The device may further comprise a wireless network interface housed by the housing. The device may further comprise an electronic display housed by the housing. The device may further comprise a microphone housed by the housing. The device may further comprise a speaker housed by the housing.The wireless network interface, the electronic display, the microphone, and the speaker may be in communication with a processing system. The processing system may be further configured to receive a spoken command via the microphone. The processing system may be further configured to send an indication of the spoken command to a cloud-based server system via the wireless network interface. The processing system may be further configured to receive a response from the cloud-based server system via the wireless network interface. The processing system may be further configured to output information obtained from the performed health monitoring via the electronic display, the speaker, or both based on the response from the cloud-based server system. The processing system may be further configured to output a report based on the health monitoring.

[0005] In some embodiments, a method for monitoring non-contact human interaction and health monitoring using a single radar modulation mode is described. The method may include emitting radar chirps such that a radar sensor operating in burst mode emits multiple bursts of radar chirps. A first amount of time elapsed between subsequent radar chirps of a burst of the multiple bursts of radar chirps may be less than a second amount of time elapsed between subsequent bursts of the multiple bursts of radar chirps. The radar sensor outputs a burst mode radar data stream that may be based on radar chirp reflections of the multiple bursts of radar chirps. The method may include a processing system receiving the burst mode radar data stream from the radar sensor. The method may include the processing system analyzing the burst mode radar data stream for non-contact human interaction. The method may include the processing system converting the burst mode radar data stream into a virtual continuous mode radar data stream. The method may include the processing system performing health monitoring of a user using the virtual continuous mode radar data stream.

[0006] Embodiments of such a method may include one or more of the following features: the virtual continuous mode radar data stream may be composed of multiple virtual returns of radar chirps equally spaced in time; configuring the processing system to convert the burst mode radar data stream into the virtual continuous mode radar data stream may include configuring the processing system to create a virtual return of the radar chirp based on multiple radar chirps of a burst of the multiple bursts; the virtual return of the radar chirp may be part of multiple virtual returns of radar chirps equally spaced in time; creating a virtual return of the radar chirp based on multiple radar chirps of a burst of the multiple bursts may include sampling multiple samples of each radar chirp of the multiple chirps of the burst; creating a virtual return of the radar chirp based on multiple radar chirps of the burst of the multiple bursts may include averaging each sample of the multiple samples with corresponding samples from other chirps of the multiple chirps of the burst to create multiple averaged samples. Creating a virtual reflection of a radar chirp based on multiple radar chirps of the one of the multiple bursts may include assembling averaged samples to create a virtual reflection of the radar chirp. The contactless human interaction may be a gesture. The contactless human interaction may be presence detection. The health monitoring may include sleep monitoring of the user.

[0007] In some embodiments, various configurations of the smart home device exist. The smart home device may include a housing configured for placement of the smart home device in a user activity area, a user sleep area, or both areas. The smart home device may include a radar sensor housed by the housing and configured to operate in a burst mode to transmit multiple bursts of radar chirps, receive reflections of the multiple bursts of radar chirps, and output a single radar data stream based on the reflections of the multiple bursts of radar chirps. A first amount of time that may elapse between adjacent radar chirps of a burst of the multiple bursts of radar chirps is less than a second amount of time that elapses between adjacent bursts of the multiple bursts of radar chirps. The smart home device may include a processing system housed by the housing, including one or more processors, and in communication with the radar sensor. The processing system may be configured to perform a first set of operations on the single radar data stream to perform user activity recognition. The processing system may be configured to perform a second set of operations on the single radar data stream to perform user vital sign detection, and no change in radar transmission mode is required to perform both user activity recognition and user vital sign detection using the single radar data stream.

[0008] Embodiments of such a smart home device may include one or more of the following features: the second set of operations may include instructions to convert the single radar data stream into a virtual continuous mode radar data stream; the user activity recognition may be gesture detection; the virtual continuous mode radar data stream may include multiple virtual returns of radar chirps equally spaced in time; the instructions to convert the single radar data stream into a virtual continuous mode radar data stream may include instructions to cause a processing system to create a virtual return of the radar chirp based on multiple radar chirps of a burst of the multiple bursts, the virtual return of the radar chirp being a portion of the multiple virtual returns of the radar chirps equally spaced in time.

[0009] Configuring the processing system to create a virtual radar chirp reflection based on multiple radar chirps of the one burst of the multiple bursts may include configuring the processing system to perform an averaging process, including sampling a plurality of samples of each radar chirp of the multiple chirps of the one burst, averaging each sample of the plurality of samples with corresponding samples from other chirps of the multiple chirps of the one burst to create a plurality of averaged samples, and assembling the averaged samples to create a virtual radar chirp reflection. User activity recognition may be performed while detection of user vital signs is being performed. The radar sensor may output a frequency-modulated continuous wave (FMCW) radar having a frequency between 57 and 64 GHz and a peak EIRP of less than 20 dBm.

[0010] BRIEF DESCRIPTION OF THE DRAWINGS A better understanding of the nature and advantages of various embodiments may be realized by reference to the following figures. In the accompanying figures, similar components or features may have the same reference label. Furthermore, various components of the same type may be distinguished by following the reference label with a dash and a second label that distinguishes between the similar components. When only a first reference label is used in this specification, the specification is applicable to any one of the similar components having the same first reference label, regardless of the second reference label. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 illustrates one embodiment of a system for performing contactless health and user interaction monitoring. [Figure 2A] 1 illustrates an embodiment of a health tracking system that uses radar to detect human interaction and provide health monitoring. [Figure 2B] 1 illustrates an embodiment of a health tracking system that uses radar to detect human interaction and provides health monitoring with integrated beam targeting. [Figure 2C] 1 illustrates an embodiment of a frequency modulated continuous wave radar beam output by a radar subsystem. [Figure 3A] 1 illustrates an embodiment of a contactless health tracking device. [Figure 3B] 1 illustrates an exploded view of an embodiment of a contactless health tracking device. [Figure 4] FIG. 1 illustrates one embodiment of a state system for determining when a person is asleep. [Figure 5] 1 shows a simplified diagram of a radar chirp received in burst mode. [Figure 6] 1 shows a simplified diagram of a radar chirp received in burst mode and converted to virtual continuous mode. [Figure 7] FIG. 1 illustrates an embodiment of a method for monitoring non-contact human interaction and health monitoring using a single radar modulation mode. DETAILED DESCRIPTION OF THE INVENTION

[0012] Detailed Description The embodiments detailed herein focus on devices that perform contactless health monitoring, such as contactless sleep monitoring, while also monitoring contactless human interactions. Contactless health monitoring may include tracking a user's sleep (for an inactive user), monitoring a user's vital signs, tracking a user's cough, etc. Contactless health monitoring may be performed using radar. Contactless human interactions may be gestures performed by an active user (e.g., a user moves their hand or arm in a particular way to be interpreted as a command) or presence (e.g., a user is detected as being present in the vicinity of the device). The device may use radar to monitor contactless human interactions.

[0013] Because both contactless health monitoring and contactless human interaction monitoring may use radar, competing demands may exist between the two types of monitoring. For example, contactless health monitoring may be effectively performed when the radar sensor is operating in continuous mode. In continuous mode, radar chirps may be received periodically. However, to effectively perform human interaction monitoring, the radar sensor may be operated in burst mode. In burst mode, a series of radar chirps ("bursts") are emitted, followed by a period of waiting before initiating the next burst of radar chirps. The period of waiting between bursts may limit the total amount of energy emitted into the user's environment, thereby reducing the user's exposure to RF energy.

[0014] In the embodiments detailed herein, the radar sensor may be operated in burst mode. Thus, the radar data stream output by the radar sensor may be optimized for human interaction monitoring. Processing may be performed on the radar data stream to create a virtual continuous mode radar data stream. This virtual continuous mode radar data stream may be created such that it contains higher quality data than if the radar sensor itself were operating in continuous mode.

[0015] By creating a virtual continuous mode radar data stream, a single device or system receiving the radar data stream can effectively monitor human interactions in burst mode and perform health monitoring in continuous mode. Thus, the single device may be able to effectively respond to contactless human interactions while health monitoring is being performed without the user having to switch the device or system mode. Similarly, the device or system may operate the radar sensor continuously in burst mode without having to change the mode in which the radar sensor operates. More precisely, the device or system continuously creates a virtual continuous mode radar data stream, thereby enabling contactless user interaction monitoring and contactless health monitoring to occur simultaneously and independently of each other.

[0016] Further details regarding such and additional embodiments may be understood in connection with the drawings. FIG. 1 illustrates an embodiment of a system 100 for performing contactless health and user interaction monitoring. The system 100 may include a contactless health and human interaction monitoring device 101 (“device 101”); a network 160; and a cloud-based server system 170. The device 101 may include a processing system 110; a sleep data storage device 118; a radar subsystem 120; an environmental sensor suite 130; an electronic display 140; a wireless network interface 150; and a speaker 155. Generally, the device 101 may include a housing that houses all of the components of the device 101. Further details regarding such possible housings are provided in connection with FIGS. 3A and 3B. The device 101 may also be referred to as a health monitoring device, a health tracking device, a sleep monitoring device, a sleep tracking device, or a home assistant device.

[0017] The processing system 110 may include one or more processors configured to perform various functions, such as the functions of the radar processing module 112, the sleep state detection engine 114, and the environmental event correlation engine 116. The processing system 110 may include one or more special-purpose processors or general-purpose processors. Such special-purpose processors may include processors specifically designed to perform the functions detailed herein. Such special-purpose processors may also be ASICs or FPGAs, which are general-purpose components physically and electrically configured to perform the functions detailed herein. Such general-purpose processors may execute special-purpose software stored using one or more non-transitory processor-readable media, such as random access memory (RAM), flash memory, a hard disk drive (HDD), or a solid-state drive (SSD).

[0018] The radar subsystem 120 (also called a radar sensor) may be a single integrated circuit (IC) that emits, receives, and outputs a radar data stream indicative of received and reflected radio waves. The output of the radar subsystem 120 may be analyzed using the radar processing module 112 of the processing system 110. Further details regarding the radar subsystem 120 and the radar processing module 112 are provided in connection with FIG. 2.

[0019] The device 101 may include one or more environmental sensors, such as all, one, or any combination of environmental sensors provided as part of the environmental sensor suite 130. The environmental sensor suite 130 may include a light sensor 132, a microphone 134, a temperature sensor 136, and a passive infrared (PIR) sensor 138. In some embodiments, multiple instances of some or all of these sensors may be present. For example, in some embodiments, multiple microphones may be present. The light sensor 132 may be used to measure the amount of ambient light present in the device's 101's general environment. The microphone 134 may be used to measure the ambient noise level present in the device's 101's general environment. The temperature sensor 136 may be used to measure the ambient temperature of the device's 101's general environment. The PIR sensor 138 may be used to detect moving, living objects (e.g., people, pets) in the device's 101's general environment. Other types of environmental sensors are also possible. For example, a camera and / or a humidity sensor may be incorporated as part of the environmental sensor suite 130. In some embodiments, some data, such as humidity data, may be obtained from a nearby weather station that has data available over the internet.

[0020] Device 101 may include various interfaces. Display 140 may enable processing system 110 to present information for viewing by one or more users. Wireless network interface 150 may enable communication using a wireless local area network (WLAN), such as a WiFi-based network. Speaker 155 may enable output of sounds, such as synthesized speech. For example, responses to voice commands received via microphone 134 may be output via speaker 155 and / or display 140. The voice commands may be analyzed locally by device 101 and transmitted to cloud-based server system 170 via wireless network interface 150 for analysis. Responses based on the analysis of the voice commands may be sent back to device 101 via wireless network interface 150 for output via speaker 155 and / or display 140. Additionally or alternatively, speaker 155 and microphone 134 may be collectively configured for active acoustic sensing, including ultrasonic acoustic sensing. Additionally or alternatively, other forms of wireless communication may be possible, such as using low-power wireless mesh network radios and protocols (e.g., Thread®) to communicate with various smart home devices. In some embodiments, a wired network interface, such as an Ethernet connection, may be used for communication with the network. Furthermore, the evolution of wireless communications to fifth-generation (5G) and sixth-generation (6G) standards and technologies provides greater throughput with lower latency, enhancing mobile broadband services. 5G and 6G technologies also offer new classes of services on control and data channels for vehicular networks (V2X), fixed wireless broadband, and the Internet of Things (IoT). Such standards and technologies may be used for communication by device 101.

[0021] Low-power wireless mesh network radios and protocols may be used to communicate with power-limited devices. Power-limited devices may be exclusively battery-powered devices. Such devices may rely exclusively on one or more batteries for power and, therefore, may keep the amount of power used for communication low to reduce the frequency at which the battery or batteries must be replaced. In some embodiments, power-limited devices may have the ability to communicate over a relatively high-power network (e.g., WiFi) and a low-power mesh network. Power-limited devices may use the relatively high-power network less frequently to conserve power. Examples of such power-limited devices include environmental sensors (e.g., temperature sensors, carbon monoxide sensors, smoke sensors, motion sensors, presence detectors) and other forms of remote sensors.

[0022] The wireless network interface 150 may enable wireless communication with a network 160. The network 160 may include one or more public and / or private networks. The network 160 may include a private local wired or wireless network, such as a home WLAN. The network 160 may also include a public network, such as the Internet. The network 160 may enable the device 101 to communicate with a remotely located cloud-based server system 170.

[0023] Cloud-based server system 170 may provide various services to device 101. With respect to sleep data, cloud-based server system 170 may include processing and storage services for sleep-related data. While the embodiment of FIG. 1 includes processing system 110 performing sleep state detection and environmental event correlation, in other embodiments, such functions may be performed by cloud-based server system 170. Also, in addition to or as an alternative to sleep data storage device 118 being used to store sleep data, sleep-related data may be stored by cloud-based server system 170, such as by being mapped to a common user account to which device 101 is linked. When multiple users are monitored, sleep data may be stored and mapped to a master user account or the accounts of corresponding users.

[0024] Cloud-based server system 170 may additionally or alternatively provide other cloud-based services. For example, device 101 may additionally function as a home assistant device. The home assistant device may respond to vocal queries from a user. In response to detecting a vocal trigger phrase being spoken through microphone 134, device 101 may record audio. The stream of audio may be transmitted to cloud-based server system 170 for analysis. Cloud-based server system 170 may perform speech recognition processing and use a natural language processing engine to understand queries from the user and provide responses output by device 101 as synthesized speech, output presented on electronic display 140, and / or commands executed by device 101 (e.g., increase the volume of device 101) or transmitted to some other smart home device. Furthermore, queries or commands may be submitted to cloud-based server system 170 via electronic display 140, which may be a touchscreen. For example, device 101 may be used to control various smart home or home automation devices. Such commands may be sent directly by the device 101 to the controlled device or may be sent via a cloud-based server system 170.

[0025] Based on the data output by the radar processing module 112, the sleep state detection engine 114 may be used to determine whether the user is likely asleep or awake. The sleep state detection engine 114 may proceed through a state machine, such as that described in detail in connection with FIG. 4, and may use states identified using such a state machine to determine whether the user is likely awake or asleep. For example, if the user is determined to be stationary in bed for at least a defined period of time, the user may be identified as asleep. The output of the sleep state detection engine 114 may be used by the environmental event correlation engine 116. The environmental event correlation engine 116 may analyze data received from the environmental sensor suite 130. Data from each environmental sensor device may be monitored for 1) an increase in an environmental condition above a fixed, predefined threshold and / or 2) an increase in the environmental condition by at least a predefined amount or percentage. As an example, data indicating the light level of the ambient environment may be continuously or periodically output by the light sensor 132. The environmental event correlation engine 116 may determine 1) whether the amount of ambient lighting increases from below a fixed, predefined threshold to above the fixed, predefined threshold, and / or 2) whether the amount of ambient lighting increases by at least a predefined amount or percentage. If option 1, 2, or both occur, an environmental event may be determined to have occurred. This environmental event may be time-stamped by the environmental event correlation engine 116. The environmental event correlation engine 116 may then determine whether the user's awakening can be attributed to the identified environmental event.

[0026] The radar data stream output by the radar subsystem 120 may be output to the human interaction engine 245 in addition to the radar processing module 112. The human interaction engine 245 may use the radar data stream to determine whether a human interaction, such as a person moving or gesture, has been performed within a defined distance of the device 101. Further details regarding how the radar subsystem 120, radar processing module 112, and human interaction engine 245 function and interact are detailed in conjunction with Figures 2A and 2B.

[0027] 2A illustrates an embodiment of a health monitoring system 200A ("system 200A") that uses radar to detect human interactions and perform health monitoring. System 200A may include a radar subsystem 205 (which may represent an embodiment of radar subsystem 120); a radar processing module 210 (which may represent an embodiment of radar processing module 112); a beam steering module 230; a virtual continuous chirp generator 240; and a human interaction engine 245.

[0028] The radar subsystem 205 may include an RF emitter 206, an RF receiver 207, and radar processing circuitry 208. The RF emitter 206 may emit radio waves, such as in the form of continuous wave (CW) radar. The RF emitter 206 may use frequency-modulated continuous wave (FMCW) radar. FMCW radar may operate in a continuous sparse sampling mode for relatively long periods of time. The RF emitter 206 may include one or more antennas and may transmit at or about 60 GHz. The frequency of the transmitted radio waves may repeatedly sweep from low to high frequencies (or vice versa). The power level used for transmission may be very low so that the radar subsystem 205 has an effective range of a few meters or even shorter. Further details regarding the radio waves generated and emitted by the radar subsystem 205 are provided in connection with FIG. 2C.

[0029] The RF receiver 207 may include one or more antennas distinct from the transmitting antenna(s) and may receive reflections of radio waves from nearby objects of the radio waves emitted by the RF emitter 206. The reflected radio waves may be interpreted by the radar processing circuit 208 by mixing the transmitted radio waves with the reflected received radio waves to generate a mixed signal that can be analyzed for distance. Based on this mixed signal, the radar processing circuit 208 may output raw waveform data, sometimes referred to as a raw chirp waterfall, for analysis by another processing entity. The radar subsystem 205 may be implemented as a single integrated circuit (IC), and the radar processing circuit 208 may be a separate component from the RF emitter 206 and the RF receiver 207. In some embodiments, the radar subsystem 205 is integrated as part of the device 101 such that the RF emitter 206 and the RF receiver 207 face in the same direction as the display 140. In other embodiments, an external device including the radar subsystem 205 may be connected to the device 101 via wired or wireless communication. For example, the radar subsystem 205 may be an add-on device to a home assistant device.

[0030] The RF subsystem 205 outputs a radar data stream. Because the RF subsystem 205 is operated in burst mode, the radar data stream output may be referred to as a burst mode radar data stream. The burst mode radar data stream may be output by the radar subsystem 205 to the human interaction engine 245 and the virtual continuous chirp generator 240. The human interaction engine 245 and the virtual continuous chirp generator 240 may be implemented as software running using the same processing system as the radar processing module 210. In other embodiments, separate processing systems may be used for the virtual continuous chirp generator 240, the human interaction engine 245, or both. In other embodiments, dedicated hardware may be used to perform the functions of the virtual continuous chirp generator 240, the human interaction engine 245, or both.

[0031] The human interaction engine 245 may use the burst-mode radar data stream to determine whether a human interaction has occurred. The processing performed by the human interaction engine 245 may be independent of the processing performed by the virtual continuous chirp generator 240 and the radar processing module 210. Therefore, whether health monitoring is performed by the radar processing module 210 or a component that uses the data output by the radar processing module 210 may be irrelevant to the human interaction engine 245 monitoring non-contact human interactions such as presence and gestures. For clarity, "non-contact" refers to the absence of physical contact between the user and the device on which the system 200A is implemented. More precisely, remote sensing, such as radar, is used to monitor the user.

[0032] Virtual continuous chirp generator 240 also receives the burst mode radar data stream. Virtual continuous chirp generator 240 performs processing on the received burst mode radar data stream and outputs a virtual continuous mode radar data stream. Details regarding the processing performed by virtual continuous chirp generator 240 to generate the virtual continuous mode radar data stream are provided in connection with FIG. 6. The output of virtual continuous chirp generator 240 is provided to radar processing module 210. Radar processing module 210 performs processing on the virtual continuous mode radar data stream as if the continuous mode radar data stream had been output directly to radar processing module 210 by radar subsystem 205.

[0033] For the radar subsystem 205, if FMCW is used, a well-defined FMCW range can be defined. Within this range, the distance to an object can be accurately determined. However, outside this range, a detected object may be erroneously interpreted as closer than an object within the well-defined range. This erroneous interpretation may be due to the frequency of the mixed signal and the sampling rate of the ADC used by the radar subsystem to convert the received analog signal to a digital signal. If the frequency of the mixed signal is above the Nyquist rate of the ADC sampling, the digital data output by the ADC representing the reflected radar signal may be erroneously represented (e.g., as a lower frequency indicating a closer object).

[0034] The radar processing module 210 may include one or more processors. The radar processing module 210 may include one or more special-purpose processors or general-purpose processors. The special-purpose processor may include a processor specifically designed to perform the functions detailed herein. Such a special-purpose processor may be an ASIC or FPGA, which are general-purpose components physically and electrically configured to perform the functions detailed herein. The general-purpose processor may execute special-purpose software stored using one or more non-transitory processor-readable media, such as random access memory (RAM), flash memory, a hard disk drive (HDD), or a solid-state drive (SSD). The radar processing module 210 may include a motion filter 211, a frequency emphasis circuit 212, a range-vital transform engine 213, a range gating filter 214, a spectral summation engine 215, and a neural network 216. Each component of the radar processing module 210 may be implemented using software, firmware, or as dedicated hardware.

[0035] The virtual continuous mode radar data stream from the virtual continuous chirp generator 240 may be received by the radar processing module 210 and first processed using the motion filter 211. It is important to note that in some embodiments, the motion filter 211 is the initial component used to perform filtering. That is, the processing performed by the radar processing module 210 is not interchangeable in some embodiments. Typically, vital sign determination and sleep monitoring may occur while the monitored user is sleeping or attempting to sleep in bed. In such an environment, there may typically be little motion. Such motion may be due to the user moving in bed (e.g., rolling over while trying to fall asleep or while asleep) and the user's vital signs, including breathing motion and motion due to the monitored user's heartbeat. In such an environment, a large portion of the radio waves emitted from the RF emitter 206 may be reflected by stationary objects near the monitored user, such as a mattress, box springs, bed frame, walls, furniture, and bedding. Therefore, a large portion of the raw waveform data received from the radar subsystem 205 may be unrelated to the user's motion and the user's vital measurements.

[0036] The motion filter 211 may include a waveform buffer that buffers "chirps" or slices of received raw waveform data. For example, sampling may occur at a rate of 10 Hz. In other embodiments, sampling may be slower or faster. In a particular embodiment, the motion filter 211 may buffer 20 seconds of received raw waveform chirps. In other embodiments, shorter or longer durations of buffered raw waveform data are buffered. This buffered raw waveform data can be filtered to remove raw waveform data indicative of stationary objects. That is, for a moving object, such as the chest of a monitored user, the user's heart rate and breathing rate will affect the distance and velocity measurements made by the radar subsystem 205 and output to the motion filter 211. This user movement introduces "jitter" into the received raw waveform data over the buffered period. More specifically, jitter refers to a phase shift caused by a moving object reflecting emitted radio waves. Rather than using reflected FMCW radio waves to determine the velocity of a moving object, it is possible to use motion-induced phase shifts in the reflected radio waves to measure vital statistics, including heart rate and respiratory rate, as detailed herein.

[0037] For stationary objects, such as furniture, zero phase shift (i.e., no jitter) will be present in the raw waveform data over the buffered period. The motion filter 211 can subtract raw waveform data corresponding to stationary objects so that raw waveform data indicative of motion is passed to the frequency emphasis circuit 212 for further analysis. The raw waveform data corresponding to stationary objects can be discarded or otherwise ignored for the remainder of processing by the radar processing module 210.

[0038] In some embodiments, an infinite impulse response (IIR) filter is incorporated as part of the motion filter 211. Specifically, a single-pole IIR filter may be implemented to filter raw waveform data that does not indicate motion. Therefore, the single-pole IIR filter may be implemented as a high-pass / low-blocking filter that prevents raw waveform data indicating motion below a certain frequency from being passed to the frequency emphasis circuit 212. The cutoff frequency may be set based on known limits for human vital signs. For example, the respiratory rate may be expected to be between 10 and 60 breaths per minute. Motion data indicating frequencies lower than 10 breaths per minute may be filtered out. In some embodiments, a band-pass filter may be implemented to filter out raw waveform data indicating high-frequency motion that is impossible or unlikely to represent a human vital sign. For example, a heart rate that can be expected to exceed the respiratory rate may not exceed 150 beats per minute for a person in a resting or near-resting state. Raw waveform data indicating higher frequencies may be filtered out by the band-pass filter.

[0039] In some embodiments, it may be possible to further fine-tune the frequency of raw waveform data that the motion filter 211 passes to the frequency emphasis circuit 212. For example, during an initial setup phase, the user may provide information about the user being monitored (e.g., themselves, their child), such as age data. Table 1 shows typical breathing rates for various ages. Similar data may exist for heart rate. Filters may be configured to filter out data that falls outside of expected breathing rate, heart rate ranges, or both.

[0040] [Table 1]

[0041] The monitored user's vital signs are periodic impulse events. That is, although the user's heart rate may vary over time, the user's heart can be expected to continue beating periodically. This beating can be understood as an impulse event that is not a sinusoidal function but more similar to a square wave with a relatively low duty cycle that induces movement in the user's body. Similarly, although the user's respiratory rate may vary over time, breathing is a periodic function performed by the user's body that is similar to a sinusoidal function, except that the user's exhalation is generally longer than their inhalation. Furthermore, at any given time, a specific window of waveform data is analyzed. Because a specific time window of waveform data is analyzed, even a perfect sine wave within that window may exhibit spectral leakage in the frequency domain. The frequency components due to this spectral leakage should be de-emphasized.

[0042] The frequency emphasis circuit 212 may work in conjunction with the range-vital transform engine 213 to determine one (e.g., respiration) or two (e.g., respiration and heart rate) frequency components of the raw waveform data. The frequency emphasis circuit 212 may use frequency windowing, such as a 2D Hamming window (other forms of windowing, such as a Hann window, are also possible), to emphasize important frequency components of the raw waveform data and de-emphasize or remove waveform data due to spectral leakage outside the defined frequency window. Such frequency windowing may reduce the magnitude of the raw waveform data that may be due to processing artifacts. The use of frequency windowing may help reduce the effects of data-dependent processing artifacts while preserving data that allows the heart rate and respiration rate to be determined separately.

[0043] For a stationary bedside FMCW radar-based monitoring device (e.g., using a radar such as that emitted in Figure 2C) that can be placed within 1-2 meters of one or more users being monitored to detect breathing and heart rate, a 2D Hamming window that emphasizes frequencies in the range of 10-60 bpm (0.16 Hz-1 Hz) for breathing and 30-150 bpm (0.5-2.5 Hz) for heart rate provides a signal that is good enough to perform reliable measurements without requiring prior knowledge of the subject's age or medical history.

[0044] Because heart rate and respiration rate are periodic impulse events, the heart rate and respiration rate in the frequency domain may be represented by different fundamental frequencies, but each may have many harmonic components at higher frequencies. One of the primary purposes of the frequency emphasis circuit 212 may be to prevent frequency ripple of the harmonics of the monitored user's respiration rate from affecting the frequency measurement of the monitored user's heart rate (or vice versa). While the frequency emphasis circuit 212 may use a 2D Hamming window, it should be understood that other window or separation functions can be used to help separate the frequency ripple of the monitored user's respiration rate from the frequency ripple of the monitored user's heart rate.

[0045] The range-vital transform engine 213 analyzes the motion-filtered received waveform data to identify and quantify the magnitude of motion at specific frequencies. More specifically, the range-vital transform engine 213 analyzes phase jitter over time to detect relatively small motions resulting from user vital signs, such as respiratory rate and heart rate, which have relatively low frequencies. The analysis of the range-vital transform engine 213 may assume that the frequency components of the motion waveform data are sinusoidal. Furthermore, the transform used by the range-vital transform engine 213 may also identify the distance at which the frequency is observed. Because the radar subsystem 205 uses an FMCW radar system, frequency, magnitude, and distance can all be determined, at least in part.

[0046] Prior to applying the transforms of the range-vital transform engine 213, a zero-padding process may be performed by the range-vital transform engine 213 to add a large number of zeros to the motion-filtered raw waveform data. Performing a zero-padding process can effectively increase resolution in the frequency domain, enabling more accurate low-rate measurements (e.g., low heart rate, low respiratory rate). For example, zero-padding may be useful for numerically increasing resolution to detect a difference of half a breath per minute compared to a resolution of one breath per minute without zero-padding. In some embodiments, three to four times as many zeros may be added compared to the buffered sample size of the raw waveform data. For example, if 20 seconds of buffered raw waveform data is analyzed, 60 to 80 seconds of zero-padding may be added to the samples. Specifically, it has been found that zero-padding the samples in the range of three to four times can substantially improve resolution without overly complicating the transform process (and therefore increasing processor usage).

[0047] To determine the amount of zero padding to be performed, Equations 1-3 may be used: In Equation 1, RPM_resolution is ideally less than 1.

[0048]

number

[0049]

number

[0050] In some embodiments, a chirp_rate of 30 Hz may be used. Such a frequency may have sufficient margin from the upper Nyquist limits of respiratory rate and heart rate. Thus, n_FFT_slow_time_min may be 2048. Given a 20-second window for estimating respiratory statistics, Equation 3 results in a value of 600.

[0051]

number

[0052] This value 600 is less than the required vital-to-FFT size, causing the range-vital transform engine 213 to perform 3-4 times zero padding. The balance of how much zero padding to perform may be based on the increase in frequency resolution and the associated increase in the amount of calculations required to perform the FFT. 3-4 times zero padding has been found to provide sufficient resolution for heart rate and respiration rate while modestly limiting the amount of calculations that need to be performed.

[0053] The range vital transform engine 213 may perform a series of Fourier transforms (FTs) to determine the frequency components of the received raw waveform data output by the frequency emphasis circuit 212. Specifically, a series of fast Fourier transforms (FFTs) may be performed by the range vital transform engine 213 to determine particular frequencies and the magnitude of the waveform data at such frequencies.

[0054] Waveform data acquired over a period of time can be represented in multiple dimensions. The first dimension (e.g., along the y-axis) may relate to multiple samples of waveform data from a particular chirp, and the second dimension (e.g., along the x-axis) relates to a particular sample index of the waveform data collected over multiple chirps. There is a third dimension of data (e.g., along the z-axis) that indicates the intensity of the waveform data.

[0055] Multiple FFTs may be performed based on the first and second dimensions of the waveform data. An FFT may be performed along each of the first and second dimensions. That is, an FFT may be performed for each chirp, or an FFT may be performed for each specific sample index across multiple chirps occurring during a period. An FFT performed on waveform data for a specific reflected chirp may indicate one or more frequencies that indicate the distance at which an object reflecting the emitted radio waves is located in an FMCW radar. An FFT performed for specific sample indexes across multiple chirps may measure the frequency of phase jitter across the multiple chirps. Thus, an FFT on the first dimension may provide the distance at which a vital statistic value resides, and an FFT on the second dimension may provide the frequency of the vital statistic. The output of the FFT performed across the two dimensions indicates 1) the frequency of the vital statistic, 2) the range at which the vital statistic was measured, and 3) the magnitude of the measured frequency. In addition to values ​​attributable to the vital statistic present in the data, noise may be present that has been filtered out, such as using the spectral summation engine 215. The noise may be due in part to the fact that heart rate and breathing are not perfect sine waves.

[0056] For clarity, the transform performed by range-vital transform engine 213 differs from a range-Doppler transform. Rather than analyzing changes in velocity (as in a range-Doppler transform), periodic changes in phase shift over time are analyzed as part of the range-vital transform. The range-vital transform is tuned to identify small movements (e.g., breathing rate, heart rate) that occur over relatively long periods of time by tracking changes in phase, referred to as phase jitter. As detailed above, zero padding is performed to allow sufficient resolution to accurately determine heart rate and breathing rate.

[0057] The range gating filter 214 is used to monitor a defined range of interest and filter out waveform data due to motion beyond the defined range of interest. For the deployment detailed herein, the defined range of interest may be 0 to 1 meter. In some embodiments, this defined range of interest may be different and possibly set by the user (e.g., via a training or setup process) or by the service provider. In some embodiments, the goal of this deployment may be to monitor one person closest to the device (and filter out or isolate data related to another person who is farther away, such as someone sleeping next to the person being monitored). In other embodiments, if both people are monitored, the data may be isolated, as described in more detail in connection with FIG. 12. Thus, the range-vital transformation engine 213 and range gating filter 214 serve to isolate, filter out, or remove motion data due to objects outside the defined range of interest and to sum the energy of motion data due to objects within the defined range of interest. The output of the range gating filter 214 may include data having a determined range within the tolerance of the range gating filter 214. The data may further have a frequency dimension and a magnitude dimension. Thus, the data may have three dimensions.

[0058] The spectral summing engine 215 may receive the output from the range gating filter 214. The spectral summing engine 215 may transmit the measured energy of harmonic frequencies of the heart rate and respiration rate and function to sum the harmonic frequency energy onto the energy of the fundamental frequency. This function is sometimes referred to as the harmonic sum spectrum (HSS). Because the heart rate and respiration rate are not sinusoidal, in the frequency domain, harmonics will exist at frequencies higher than the fundamental frequency of the user's respiration rate and the fundamental frequency of the user's heart rate. One of the primary purposes of the spectral summing engine 215 is to prevent harmonics of the monitored user's respiration rate from affecting the frequency measurement of the monitored user's heart rate (or vice versa). HSS may be performed in the second order by summing the original spectrum with a downsampled (by a factor of 2) instance of the spectrum. This process may also be applied to higher harmonics, such that their respective spectra are added to the spectrum of the fundamental frequency.

[0059] At this stage, for a person lying still in bed (excluding movements due to breathing and heartbeat), two major frequency peaks are expected to be present in the frequency data. However, if the monitored user is physically moving, such as tossing and turning in bed, the energy will be more widely distributed (broader distribution) across the frequency spectrum. Such large physical movements may manifest themselves in the frequency data as many smaller peaks. If the bed is empty rather than occupied, there may be no or very few frequency components above the noise floor because the motion filter 211 has pre-filtered the raw waveform data corresponding to stationary objects. The distribution and magnitude of the frequency peaks across the spectrum may be used to determine whether the user is likely to be awake or asleep.

[0060] The spectral summation engine 215 may output a feature vector indicating the heart rate (e.g., beats per minute) and respiration rate (e.g., breaths per minute). The feature vector may indicate frequency and magnitude. The neural network 216 may be used to determine whether the heart rate and / or respiration rate indicated in the feature vector output from the spectral summation engine 215 should be considered valid. Thus, the heart rate and respiration rate output by the spectral summation engine 215 may be stored, presented to a user, and / or treated as valid based on the output of the neural network 216. By performing spectral analysis, the neural network 216 may be trained (e.g., using supervised learning performed using a training set of data) to output one of three states as shown in Table 2. Vital statistics data may be considered valid if a user is determined to be present and the detected movement is attributable to the user's vital signs.

[0061] Each state in Table 2 is associated with a different spectral energy and spectral sparsity profile. Spectral energy refers to the sum of energy across the frequency spectrum detected due to motion present in the monitored area. Spectral sparsity describes whether motion tends to be distributed across a wide range of frequencies or concentrated in a few specific frequencies. For example, if energy peaks occur infrequently, such as when a user's vital signs are detected (and no other motion is detected), the spectrum has high sparsity. However, if peaks (above threshold) or other forms of threshold-based determination (based at least in part on magnitude) occur frequently, the spectrum has low sparsity.

[0062] As an example, motion due to a vital sign such as a heartbeat may indicate significant motion (e.g., high spectral energy) at a particular frequency (e.g., high spectral sparsity), while motion due to a user moving their limbs may also indicate significant motion (high spectral energy) but low spectral sparsity. A neural network may be trained to distinguish between each state based on the spectral energy profile output by the spectral summation engine 215. Thus, the neural network 216 may be provided with two features: a first value representing spectral energy and a second value representing spectral sparsity.

[0063] The output of the spectral summation engine 215 may be characterized as a feature vector having a first dimension of frequency and a second dimension of amplitude. A first value representing spectral energy may be calculated by determining the maximum amplitude present in the feature vector output by the spectral summation engine 215. This maximum amplitude value may be normalized to a value between 0 and 1. A second value representing spectral sparsity may be calculated by subtracting the median amplitude of the feature vector from the maximum amplitude. Again, the calculated sparsity may be normalized to a value between 0 and 1.

[0064] Table 2 presents a generalization of how the spectral energy and spectral sparsity features are used as features by the trained neural network to classify the state of the monitored region.

[0065] [Table 2]

[0066] The state of the monitored area classified by the neural network 216 may be used in determining the sleep state of the monitored user, or more generally, whether the user is moving or stationary in bed. The state of the monitored area determined by the classification performed by the neural network 216 may further be used to determine whether the vital statistics output by the spectral summation engine 215 should be trusted or ignored. For accurate vital statistics determination, heart rate and respiration rate may be identified as likely to be accurate if the neural network 216 determines that the user is present and stationary (i.e., there is no significant physical movement, but movement is occurring due to breathing and / or heartbeat). In some embodiments, the vital statistics output by the spectral summation engine 215 may be stored exclusively locally (e.g., to alleviate privacy concerns), while in other embodiments, the vital statistics output may be transmitted to the cloud-based server system 170 for remote storage (instead of, or in addition to, storing such data locally).

[0067] The neural network 216 may be initially trained using a large set of training data of amplitude and frequency feature vectors appropriately tagged with classifications that map spectral energy and spectral sparsity to corresponding ground truth states of the monitored region. Alternatively, the neural network 216 may be initially trained using a large set of training data of amplitude and frequency feature vectors appropriately labeled to map constituent spectral energy and spectral sparsity pairs, each appropriately tagged to a corresponding ground truth state of the monitored region. The neural network may be a time-independent fully connected neural network. In some embodiments, machine learning arrangements, classifiers, or forms of artificial intelligence other than neural networks may be used.

[0068] In other embodiments, spectral energy values ​​and spectral sparsity values ​​are not the features used by the neural network, and a neural network, possibly with extra front-end convolutional layers, can be trained to directly use the output of range gating filter 214. Instead, an embodiment of the convolutional network can analyze the frequency and magnitude data output by range gating filter 214 to classify the user's state. The convolutional neural network can be trained using offline training based on a set of spectral measurements mapped to ground truth states of the monitored area before system 200B is used by an end user.

[0069] The sleep state determined by the neural network 216 may be stored in the sleep data storage device 118, along with the time data. The vital statistics output by the spectral summation engine 215 may be stored in a vital statistics data store if the neural network 216 indicates that the monitored user is present and motionless. Other vital statistics may be discarded or, in some cases, flagged to indicate that they are unlikely to be correct. The data stored in the sleep data storage device 118 and the vital statistics data store may be stored locally on the device 101. In some embodiments, storage occurs solely on the device 101. Such an implementation may help alleviate concerns about health-related data being transmitted and stored remotely. In some embodiments, the monitored user may choose to have the sleep data and vital statistics data transmitted via a network interface (e.g., wireless network interface 150) stored and analyzed externally, such as by a cloud-based server system 170. Storage by the cloud-based server system 170 may have important advantages, such as the ability for the user to access such data remotely, grant access to a healthcare provider, or participate in research studies. Users may retain the ability to delete or otherwise remove data from the cloud-based server system 170 at any time.

[0070] In some embodiments, the radar processing module 210 may be located in whole or in part remotely from the device 101. While the radar subsystem 205 may need to be local to the monitored user, the processing of the radar processing module 210 may be moved to the cloud-based server system 170. In other embodiments, a smart home device in local communication with the device 101 (e.g., via a LAN or WLAN) may perform some or all of the processing of the radar processing module 210. In some embodiments, a local communication protocol, such as a mesh network, may be used to transmit the raw waveform data to a local device that will perform the processing. Such communication protocols may include Wi-Fi, Bluetooth, Thread, or the IEEE 802.11 and 802.15.4 families of communication protocols. Storage of the sleep data and vital statistics data, as well as processing, may occur on the cloud-based server system 170 or another smart home device in the home where the device 101 is located. In still other embodiments, the radar processing module 210 may be combined with the radar subsystem 205 as a single component or system of components.

[0071] The stored sleep data and vital statistics data of the sleep data storage device 118 may be used by the sleep data compilation engine 119 to provide the user with short-term and long-term trends regarding the user's sleep patterns, vital statistics, or both. For example, each morning, graphs, statistics, and trends may be determined by the sleep data compilation engine 119 based on the data stored in the sleep data storage device 118 and output for display by the sleep data compilation engine 119 via the display 140. A graph showing sleep data from the previous night, and possibly one or more graphs showing the previous night's respiratory rate and heart rate, may be presented. Similar graphs, trends, and statistics may be output by the sleep data compilation engine 119 over significantly longer periods of time, such as over weeks, months, years, and even multiple years. Other uses of the sleep data and vital statistics are also possible. For example, a medical professional may be notified if a particular trigger regarding heart rate, respiratory rate, and / or sleep patterns is triggered. Additionally or alternatively, a notification may be output to the user indicating that the collected data is of potential concern or indicates a healthy individual. In some examples, a particular sleep problem, such as sleep apnea, may be identified. Sleep data may be output via speaker 155 using synthesized speech (e.g., in response to the user waking up, in response to a spoken user command, or in response to the user providing input via a touchscreen such as display 140). Such sleep data may also be represented graphically and / or textually on display 140.

[0072] System 200A may additionally include a beam steering module 230. The beam steering module 230 may include a channel weighting engine 231, which may be implemented using software, firmware, and / or hardware similar to the components of radar processing module 210. The beam steering module 230 is illustrated as separate from radar processing module 210 because it processes data received from radar subsystem 205 to emphasize data received from certain directions and de-emphasize data received from other directions. The beam steering module 230 may be implemented using the same hardware as radar processing module 210. For example, the beam steering module 230 may be a software process that modifies radar data received from radar subsystem 205 before motion filter 211 is applied. Device 101 may be a surface-top device intended to be placed in a specific location, connected to a continuous power source (e.g., a household power outlet), and interacted with via voice and / or a touchscreen. Thus, radar subsystem 205 may remain pointed at a portion of the surrounding environment for a significant period of time (e.g., hours, days, weeks, or months). In general, the beam steering module 230 may be used to map the environment (e.g., a room) in which the device 101 is located and direct the detection direction of the radar subsystem 205 to a zone within the field of view of the radar subsystem 205 where the user is most likely to be present.

[0073] Targeting an area within the field of view of the radar subsystem 205 can help reduce the amount of false negatives and false positives caused by the movement of objects other than the user. Additionally, targeting can help compensate for the angle and position of the device 101 relative to where the user is sleeping. (For example, the device 101 may be placed on a nightstand that is at a different height than the user's bed. Additionally or alternatively, the device 101, radar subsystem 205, may not be pointed directly at the location in the bed where the user is sleeping.)

[0074] If it is determined that no users are present, such as based on low spectral energy and low spectral density in Table 2, optimal beam steering processing may be performed by channel weighting engine 231 and beam steering system 232. While no users are present, an analysis may be performed to determine which directional arrangement of radar subsystem 205 results in the least clutter.

[0075] 2B illustrates an embodiment of a health tracking system 200B (“system 200B”) that uses radar to detect human interaction and provides health monitoring with integrated beam targeting. The beam targeting performed by using beam steering module 230 may focus radar returns from areas where a user may be present and ignore or at least reduce radar returns from interfering objects, such as nearby walls or large objects. In particular, beam steering module 230 may perform processing on a virtual continuous mode radar data stream, so that the burst mode radar data stream analyzed by human interaction engine 245 can remain unaffected by the targeting performed by beam steering module 230.

[0076] Virtual continuous chirp generator 240 and human interaction engine 245 may function as described in detail in connection with FIG. 2A . However, the virtual continuous mode radar stream output by virtual continuous chirp generator 240 is input to beam steering module 230 rather than being input directly to radar processing module 210. Radar subsystem 205 may output separate data for each antenna of radar subsystem 205. Thus, there may be a separate burst mode radar data stream for each antenna (or a single burst mode radar data stream may indicate the specific antenna that received the reflected radio waves). Virtual continuous chirp generator 240 may maintain separation of the data for each antenna. Thus, a virtual continuous mode radar data stream may be created for each antenna, or a single virtual continuous mode radar data stream may be created that indicates which portions of the virtual continuous mode radar data stream correspond to which antenna of radar subsystem 205.

[0077] The radar subsystem 205 may include multiple antennas to receive reflected radar waves. In some embodiments, there may be three antennas. These antennas may be arranged in an "L" pattern, with two antennas orthogonal in the horizontal direction and two antennas orthogonal in the vertical direction, with one antenna being used in both horizontal and vertical orientations. Weighting may be applied to target the receive radar beams in the vertical and / or horizontal directions by analyzing the phase difference of the received radar signals. In other embodiments, the antennas may be arranged in different patterns.

[0078] Vertical targeting may be performed to compensate for vertical tilt of the device in which the system 200B is incorporated. For example, as described below in connection with FIG. 3A, the face of the contactless health tracking device 300 may be tilted relative to where a user typically sleeps.

[0079] Horizontal targeting may be performed to compensate for emitted radar being aimed at objects that cause interference. For example, if a user's bed headboard is against a wall, the headboard and / or wall may occupy a significant portion of the field of view of the radar subsystem 120. Radar reflections from the headboard and / or wall may not be useful for determining data about the user, and therefore it may be beneficial to de-emphasize reflections from the wall and / or headboard and to emphasize reflections obtained away from the wall and / or headboard. Therefore, by weighting applied to the received radar signal, the receive beam may be directed horizontally away from the wall and headboard.

[0080] In system 200B, beam steering module 230 is present to perform processing on the raw chirp waterfall of the virtual continuous mode radar data stream received from radar subsystem 205 via virtual continuous chirp generator 240. Thus, beam steering module 230 can function as a pre-processing module prior to the analysis of radar processing module 210 and can help highlight areas where one or more users are expected to be present. Beam steering module 230 may be implemented using hardware, software, or firmware, and thus beam steering module 230 may be implemented using the same processor or processors as radar processing module 210.

[0081] The beam steering module 230 may include a channel weighting engine 231 and a beam steering system 232. The channel weighting engine 231 can be used to perform a training process that determines a set of weights to be applied to the received radar signals from each antenna before the received radar signals are combined. The channel weighting engine 231 may perform the training process when the monitored area is determined to be empty. During such times, the strength of signals received from large, stationary objects (e.g., walls, headboards) can be analyzed and weights can be set to direct the beam horizontally (and possibly vertically) away from such objects. Thus, the channel weighting engine 231 can manipulate the direction of the received radar beam to minimize the amount of reflection in a static environment for a particular distance range from the device (e.g., up to 1 meter). Such training may also be performed when a user is present. That is, the receive beam of the radar subsystem 205 can be directed toward locations where motion is detected or, more specifically, where the user's vital signs are present.

[0082] The weights determined by the channel weighting engine 231 may be used by the beam steering system 232 to apply a weight to the received reflected radar signal of each antenna individually. After the received signals from each antenna have been weighted, they may be combined for processing by the radar processing module 210.

[0083] FIG. 2C illustrates an embodiment of a chirp timing diagram 200C of frequency-modulated continuous wave (FMCW) radar radio waves output by the radar subsystem. The chirp timing diagram 200C is not to scale. The radar subsystem 205 may generally output radar waves in the pattern of the chirp timing diagram 200C. The chirp 250 represents a continuous pulse of radio waves sweeping up in frequency from low to high. In other embodiments, individual chirps may sweep continuously down from high to low, from low to high, and back down, or from high to low, and back up. In some embodiments, the low frequency is 58 GHz and the high frequency is 63.5 GHz (at such frequencies, the radio waves are sometimes referred to as millimeter waves). In some embodiments, the frequency is between 57 GHz and 64 GHz. The low and high frequencies may vary depending on the embodiment. For example, the low and high frequencies may be between 45 GHz and 80 GHz. The selected frequencies may be selected at least in part to comply with government regulations. In some embodiments, each chirp comprises a linear sweep from low to high frequency (or vice versa), while in other embodiments, an exponential or other pattern may be used to sweep the frequency from low to high or high to low.

[0084] Chirp 250, which may represent all chirps in chirp timing diagram 200C, may have a chirp duration 252 of 128 μs. In other embodiments, chirp duration 252 may be longer or shorter, such as from 50 μs to 1 ms. In some embodiments, a period of time may elapse before a subsequent chirp is emitted. Inter-chirp pause 256 may be 205.33 μs. In other embodiments, inter-chirp pause 256 may be longer or shorter, such as from 10 μs to 1 ms. In the illustrated embodiment, chirp period 254, which includes chirp 250 and inter-chirp pause 256, may be 333.33 μs. This duration varies based on the selected chirp period 252 and inter-chirp pause 256.

[0085] A number of chirps output, separated by inter-chirp pauses, may be referred to as a frame 258 or frame 258. A frame 258 may include 20 chirps. In other embodiments, the number of chirps in a frame 258 may be more or less, such as 1 to 100. The number of chirps present in a frame 258 may be determined based on the average amount of power desired to be output within a given period. The FCC or other regulatory body may set a maximum amount of power allowed to be radiated into the environment. For example, there may be duty cycle requirements that limit the duty cycle to less than 10% in any 33 ms period. In one specific example where there are 20 chirps per frame, each chirp may have a duration of 128 us and each frame may be 33.33 ms in duration. The corresponding duty cycle is (20 frames) * (.128 ms) / (33.33 ms), or approximately 7.8%. Limiting the number of chirps in a frame 258 before the inter-frame pause may limit the average output power. In some embodiments, the peak EIRP (effective isotropic radiated power) may be 13 dBm (20 mW) or less, for example, 12.86 dBm (19.05 mW). In other embodiments, the peak EIRP is 15 dBm or less and the duty cycle is 15% or less. In some embodiments, the peak EIRP is 20 dBm or less. That is, at any given time, the average power radiated over a period of time by the radar subsystem may be limited to never exceed such a value. Furthermore, the total power radiated over a period of time may be limited. In some embodiments, a duty cycle may not be required.

[0086] The frames may be transmitted at a frequency of 30 Hz (33.33 ms), as indicated by period 260. In other embodiments, the frequency may be higher or lower. The frame frequency may depend on the number of chirps in a frame and the duration of the inter-frame pause 262. For example, the frequency may be between 1 Hz and 50 Hz. In some embodiments, chirps may be transmitted continuously, such that the radar subsystem outputs a continuous stream of chirps separated by inter-chirp pauses. A tradeoff may be made to conserve the average power consumed by the device due to transmitting chirps and processing received reflections of the chirps. The inter-frame pause 262 represents a period during which no chirps are output. In some embodiments, the inter-frame pause 262 is significantly longer than the duration of the frames 258. For example, the duration of the frames 258 may be 6.66 ms (with a chirp period 254 of 333.33 μs and 20 chirps per frame). If 33.33 ms occurs between frames, the inter-frame pause 262 may be 26.66 ms. In other embodiments, the duration of the inter-frame pause 262 may be longer or shorter, such as 15 ms to 40 ms.

[0087] In the illustrated embodiment of FIG. 2C , one frame 258 and the start of a subsequent frame are shown. It should be understood that each subsequent frame can be configured similarly to frame 258. Furthermore, the transmission mode of the radar subsystem may be fixed; that is, chirps may be transmitted according to chirp timing diagram 200C regardless of whether a user is present, the time of day, or other factors. Thus, in some embodiments, the radar subsystem always operates in one transmission mode, regardless of the state of the environment or the activity being monitored. A continuous train of frames similar to frame 258 may be transmitted while device 101 is powered on.

[0088] 3A shows an embodiment of a contactless sleep tracking device 300 ("device 300"). Device 300 may have a front face that includes a front transparent screen 340 through which a display can be viewed. Such a display may be a touchscreen. Surrounding front transparent screen 340 may be an optically opaque area called a bezel 330, through which radar subsystem 205 may have a view of the environment in front of device 300.

[0089] For the purposes of the immediate discussion, the terms vertical and horizontal generally describe orientations relative to a bedroom, with vertical referring to a direction perpendicular to the floor and horizontal referring to a direction parallel to the floor. The radar subsystem, which may be an Infineon® BGT60 radar chip, is generally planar and mounted generally parallel to the bezel 330 for spatial compactness of the overall device, and because the antenna within the radar chip is in the plane of the chip, it may direct the receive beam of radar subsystem 120 in direction 350 that is generally perpendicular to bezel 330 without beam targeting. Due to a departure tilt of bezel 330 away from a purely vertical orientation, provided to be approximately 25 degrees in some embodiments to facilitate easy user interaction with the touchscreen functionality of transparent screen 340, direction 350 may be directed upward from horizontal by departure angle 351. Assuming that device 300 is typically placed on a bedside platform (e.g., a nightstand) that is approximately level with the top surface of a mattress on which a user sleeps, it may be beneficial to aim the receive beam of radar subsystem 120 in a horizontal direction 352 or near horizontal (e.g., between −5° and 5° from horizontal). Thus, vertical beam targeting can be used to compensate for the departure angle 351 of the portion of device 300 where radar subsystem 120 resides.

[0090] 3B is an exploded view illustrating an embodiment of a contactless sleep tracking device 300. Device 300 may include a display assembly 301, a display housing 302, a main circuit board 303, a neck assembly 304, a speaker assembly 305, a base plate 306, a mesh network communication interface 307, a top daughter board 308, a button assembly 309, a radar assembly 310, a microphone assembly 311, a rocker switch bracket 312, a rocker switch board 313, a rocker switch button 314, a Wi-Fi assembly 315, a power supply board 316, and a power supply bracket assembly 317. Device 300 may represent an embodiment of how device 101 can be implemented.

[0091] The display assembly 301, display housing 302, neck assembly 304, and base plate 306 may collectively form a housing that houses all of the remaining components of device 300. The display assembly 301 may include an electronic display, which may also be a touchscreen, that presents information to a user. The display assembly 301 may therefore include a display screen, which may include a metal plate of the display that can serve as a ground plane. The display assembly 301 may include a transparent portion separate from the metal plate that provides various sensors with a view of the general direction in which the display assembly 301 is facing. The display assembly 301 may include an exterior surface made of glass or clear plastic that functions as part of the housing of device 300.

[0092] The display housing 302 may be plastic or other rigid or semi-rigid material that serves as a housing for the display assembly 301. Various components, such as a main circuit board 303, a mesh network communication interface 307, a top daughter board 308, a button assembly 309, a radar assembly 310, and a microphone assembly 311, may be mounted on the display housing 302. The mesh network communication interface 307, the top daughter board 308, the radar assembly 310, and the microphone assembly 311 may be connected to the main circuit board 303 using a flat wire assembly. The display housing may be attached to the display assembly 301 using an adhesive.

[0093] The mesh network communication interface 307 may include one or more antennas and may enable communication with a mesh network, such as a Thread-based mesh network. The Wi-Fi assembly 315 may be located at a distance from the mesh network communication interface 307 to reduce the possibility of interference. The Wi-Fi assembly 315 may enable communication with a Wi-Fi-based network.

[0094] The radar assembly 310, which may include the radar subsystem 120 or the radar subsystem 205, may be positioned such that its RF emitter and RF receiver are spaced away from the metal plate of the display assembly 301 and a significant distance from the mesh network communication interface 307 and the Wi-Fi assembly 315. These three components may be arranged in a generally triangular configuration to increase the distance between the components and reduce interference. For example, in the device 300, a distance of at least 74 mm may be maintained between the Wi-Fi assembly 315 and the radar assembly 310. A distance of at least 98 mm may be maintained between the mesh network communication interface 307 and the radar assembly 310. Additionally, a distance between the radar assembly 310 and the speaker 318 may be desired to minimize the effects of vibrations on the radar assembly 310 that the speaker 318 may generate. For example, in the device 300, a distance of at least 79 mm may be maintained between the radar assembly 310 and the speaker 318. Additionally, a distance between the microphone and the radar assembly 310 may be desired to minimize potential interference from the microphone on the received radar signal. The top daughterboard 308 may include multiple microphones. For example, at least 12 mm may be maintained between the closest microphone on the top daughterboard 308 and the radar assembly 310.

[0095] Other components may also be present. A third microphone assembly, microphone assembly 311, may be present and may face rearward. Microphone assembly 311 may function in conjunction with the microphone on top daughter board 308 to separate voice commands from background noise. Power supply board 316 may convert power received from an AC power source to DC to power the components of device 300. Power supply board 316 may be mounted within device 300 using power supply bracket assembly 317. Rocker switch bracket 312, rocker switch board 313, and rocker switch button 314 may collectively be used to receive user input, such as an up / down input. Such input may be used, for example, to adjust the volume of sound output via speaker 318. As another user input, button assembly 309 may include a toggle button that the user can activate. Such user input may be used to activate and deactivate all microphones, such as for cases where the user desires privacy and / or does not want device 300 to respond to voice commands.

[0096] 4 shows an embodiment of a state machine 400 for determining when a person is asleep. Based on the data output by the radar processing module 112, the sleep state detection engine 114 may use the state machine 400 to determine whether the person is asleep. It should be understood that in some embodiments, the sleep state detection engine 114 is incorporated as part of the functionality of the radar processing module 112 and does not exist as a separate module. The state machine 400 may include five potential sleep states: in bed 401, not in bed 402, moving in bed 403, not moving in bed 405, and out of bed 404.

[0097] If there is no waveform data indicating movement, this may indicate that the user is not in bed. A user in bed can be expected to be constantly moving, at least slightly, based on vital signs. Therefore, if no movement is observed, the user may be determined to be in state 401. After state 401 is determined, the next possible state to be determined is state 402. In state 402, the monitored user is getting into bed. For example, according to Table 2, significant user movement may be detected. This may indicate that the user is getting into bed, which may cause the state to transition from state 401 to state 402.

[0098] From state 402, motion in bed may continue to be detected due to the user turning over, being positioned, moving pillows, sheets, and / or blankets, reading a book, etc. State 402 may transition to state 403 while such motion continues to be detected. Alternatively, if motion is detected followed by no more motion, this may indicate that the monitored user has entered state 405 by leaving bed. If this occurs, state 402 may transition to state 405 and then back to state 401. In general, state 404 may be interpreted as a state in which the user is asleep, and state 403 may be interpreted as a state in which the user is awake. In some embodiments, a time greater than a threshold in state 404 (or some other form of determination using at least partially time-based form of threshold criteria) is required to classify the user as asleep, and a time greater than a threshold in state 403 (or some other form of determination using at least partially time-based form of threshold criteria) is required to classify the user as awake. For example, movement in bed for less than five seconds may be interpreted as movement while the user was still asleep if the user had previously been determined to be asleep. Thus, if a user transitions from state 404 to state 403, experiences a number of movement events, and then returns to state 404 within less than the duration, the user may be identified as having experienced a "sleep arousal" in which their sleep was disturbed but they were not awakened. Such sleep arousals may be tracked along with episodes in which the user is determined to be fully awakened, or separate data may be maintained for such episodes.

[0099] From state 403, the monitored user may be determined to be getting out of bed in state 405 and may transition to a state of immobility in state 404. "Immobility" in state 404 refers to the absence of significant movement by the monitored user, but the user continuing to exhibit minor movements as measured by vital signs. In some embodiments, vital signs are treated as accurate and / or stored, recorded, or otherwise used to measure the user's vital signs only if the monitored user's state is determined to be in state 404. Data collected between states 403 and 404 may be used to determine the monitored user's general sleep patterns (e.g., how much time was spent tossing and turning, what the quality of sleep was, when deep sleep occurred, when REM sleep occurred, etc.). After the user has entered state 404 for a predetermined period, the user may be assumed to be asleep until they exit state 404. When a user first transitions to state 404, the user may be required to remain in state 404 for a certain period of time, such as 2-5 minutes, to be considered asleep. If the user is in state 403 for at least the defined period, the user may be identified as awake. However, if the user enters state 403 from state 404 and returns to state 404 for less than the defined period, the user may be identified as having only moved within their sleep and continuing to sleep.

[0100] FIG. 5 shows a diagram 500 of reflected radar chirps transmitted and received in burst mode. Diagram 500 represents reflected and received radar chirps when radar subsystem 205 (or, more generally, radar subsystem 120) is operating in burst mode. In burst mode, a burst 510 (which may also be referred to as a frame) is received by radar subsystem 205. A single burst includes a number of chirps. A reflected chirp is received for each chirp emitted by radar subsystem 205. As shown, burst 510-1 may include five reflected chirps: chirp 511-1, chirp 511-2, chirp 511-3, chirp 511-4, and chirp 511-5. In other embodiments, a burst includes a greater or lesser number of chirps. For example, in some embodiments, a burst includes 3, 4, 10, 15, 20, or other numbers of chirps. A burst may contain any number of chirps from 2 to 100.

[0101] Within a burst, such as burst 510-1, a chirp may be transmitted, and thus reflected and received, approximately every 0.333 ms. Thus, time 512 may be 0.333 ms. In other embodiments, time 512 may be greater or less. For example, time 512 may be between 0.01 ms and 5 ms. Time 514, which represents the burst duration, i.e., the amount of time between the start of burst 510-1 and the start of burst 510-2, may be 33.3 ms. In other embodiments, time 514 may be greater or less, such as between 15 ms and 500 ms. A relatively longer time may elapse between each burst, such as between burst 510-1 and burst 510-2, than between adjacent chirps within the burst. For example, the amount of time between burst 510-1 and burst 510-2, time 513-1 (i.e., the amount of time elapsed between chirp 511-5 and chirp 511-6), may be 26.7 ms. In other embodiments, time 513-1 is greater or less, such as any value between 1 ms and 200 ms. Regardless of the embodiment, time 513-1 is greater than time 512, i.e., the time between adjacent chirps within a burst or frame is less than the time between bursts.

[0102] The timing 513 between bursts may be fixed, so time 513-1, time 513-2, and time 513-4 may be the same. Similarly, the time between chirps within a burst may be constant. The radar subsystem may operate continuously in this mode and does not need to change modes for vital detection.

[0103] FIG. 6 shows a diagram 600 of a radar chirp received as part of a burst mode radar data stream and converted into a virtual continuous mode radar data stream. When a reflected radar chirp is received by the radar subsystem, it may be mixed with the currently emitted frequency, thus generating a mixed, received radar chirp. The conversion of diagram 600 may be performed by virtual continuous chirp generator 240 based on the raw chirp waterfall output by the radar subsystem. As detailed in connection with FIG. 5, a radar sensor may operate in burst mode and therefore receive reflected radar chirps in a burst pattern. The waveform data or raw chirp waterfall output by a radar subsystem operating in burst mode may be referred to as a burst mode radar data stream. This burst mode radar data stream is illustrated in FIG. 6 by a diagrammatic representation of bursts 510 and chirps 511.

[0104] Transform 610 is performed to convert each reflected burst in the raw chirp waterfall into a single, representative virtual continuous radar chirp 620. Transform 610-1 (represented by arrows) converts reflected burst 510-1 into virtual continuous radar chirp 620-1; transform 610-2 converts burst 510-2 into virtual continuous chirp 620-2; transform 610-3 converts burst 510-3 into virtual continuous chirp 620-3; transform 610-4 converts burst 510-4 into virtual continuous chirp 620-4, and so on.

[0105] To perform transform 610-1, each of chirps 511-1 through 511-5 is sampled a certain number of times from the burst mode radar data stream. This sampling may occur at a certain number of chirp sampling points, such as chirp sampling point 605-1 through chirp sampling point 605-7. (The arrows representing sampling points 605-3, 605-4, 605-5, and 605-6 are not labeled to simplify FIG. 6.) As an example, if five chirps are present in burst 510-1 and seven chirp sampling points are used, a total of 35 samples (seven samples for each chirp) may be sampled from burst 510-1. The number of chirp sampling points used may vary depending on the embodiment and may be more or less than the seven chirp sampling points shown in FIG. 6.

[0106] As part of the transformation 610-1, an averaging or combining process may be performed. In general, each of the chirps (chirps 511-1 to 511-5) of burst 510-1 may be averaged together. To achieve this averaging, the samples of each chirp of burst 510-1 taken at chirp sampling point 605-1 may be averaged together; the samples of each chirp of burst 510-1 taken at chirp sampling point 605-2 may be averaged together; the samples of each chirp of burst 510-1 taken at chirp sampling point 605-3 may be averaged together; the samples of each chirp of burst 510-1 taken at chirp sampling point 605-4 may be averaged together; the samples of each chirp of burst 510-1 taken at chirp sampling point 605-5 may be averaged together; the samples of each chirp of burst 510-1 taken at chirp sampling point 605-6 may be averaged together; and the samples of each chirp of burst 510-1 taken at chirp sampling point 605-7 may be averaged together. More generally, each sample of a chirp is averaged with the corresponding sample of every other chirp in the burst, e.g., the third sample of a chirp is averaged with the third sample of every other chirp in the burst.

[0107] In the illustrated embodiment, the seven averaged samples may then be used to construct virtual continuous radar chirp 620-1. For example, the first average from chirp sampling point 605-1 is used as the first sample 621-1 of virtual continuous radar chirp 620-1; the second average from chirp sampling point 605-2 is used as the second sample 621-2 of virtual continuous radar chirp 620-1, and so on. More generally, the averaged samples of burst 510-1 are concatenated to generate virtual continuous radar chirp 620-1.

[0108] In other words, sample-by-sample averaging is performed for each of the bursts 510. Mathematically, each chirp within a burst can be expressed as a i,j where i is the index for the sample of the chirp and j is the index for the chirp within the burst. 2,3 would be the second sample of the third chirp in a given burst. Equation 4 can be used to find the mean value used to create the virtual continuous radar chirp: V k = average(c k,1 ,c k,2 ,c k,3 ,…c k,N ) Equation 4 In Equation 4, N represents the number of chirps in the burst, and k represents the number of samples. k represents the mean value corresponding to the kth sample. Equation 4 expresses all V used to construct the virtual continuous radar chirp. k It is repeated k times to obtain the value.

[0109] The created virtual continuous radar chirps 620 may be equally spaced in time. For example, time 630-1 may be 33.3 ms, which may coincide with the burst period described in detail in connection with FIG. 5 . Times 630-2, 630-3, etc. may coincide with time 630-1 so that the virtual continuous radar chirps are output periodically. More generally, time 630 may be the same as the period of burst 510, since a single virtual continuous radar chirp is generated for each burst of burst 510. By equally spaced in time, the virtual continuous radar chirps may be more effective than burst-mode radar data streams for use in health monitoring. The virtual continuous radar chirps 620 may be output as a virtual continuous-mode radar data stream to other components for processing. If such other components are implemented as software, the processing may be performed using the same processor or processing system used to perform the functions of virtual continuous chirp generator 240.

[0110] By averaging the chirps 511 of the burst 510-1 together, the virtual continuous radar chirp 620-1 may have higher resolution than any of the individual chirps 511. The reflected radar received by the radar subsystem 120 (or 205) may be converted to digital data using an analog-to-digital converter (ADC). The ADC may have a limited resolution, such as 8 bits. Thus, all values ​​output by the ADC may be limited to a range of 0 to 256, corresponding to 8 binary bits. Dithering is a concept in signal processing that can trade signal fidelity for increased resolution. By averaging or otherwise combining multiple reflected, noisy chirps, dithering can be used to obtain higher resolution than that output by the ADC used to convert the analog radar signal to the digital waveform data present in the digital burst-mode radar data stream. Intuitively, the samples in a series of noisy chirps will dither slightly around their true values ​​with the added ADC noise. When multiple chirps are aggregated, measurements are captured in a statistically accurate manner. Without noise, even if there are many chirps in a burst, there can be no statistical advantage because each sample across the chirp will have the same value, subject to ADC quantization. The resulting higher resolution signal with dithering has higher fidelity in detecting weak signatures such as human vital signs (e.g., breathing, heart rate).

[0111] Various methods may be performed using the systems and configurations detailed in connection with Figures 1-6. Figure 7 shows an embodiment of a method 700 for monitoring non-contact human interactions and monitoring health using a single radar modulation mode. Method 700 may be performed using the devices and systems detailed in connection with Figures 1-3B.

[0112] In block 705, radio waves are emitted by a radar subsystem or radar sensor. The emitted radio waves may be continuous wave radar, such as FMCW, as described in more detail in connection with FIG. 2C. The radar sensor may operate in burst mode, such that radar chirps are emitted (and therefore received) in a pattern similar to that of FIG. 2C. The emitted radio waves may be emitted by an RF emitter 206 of the radar subsystem 205, which may emit the radio waves using one or more antennas. In block 710, reflections of the radio waves may be received, such as by an RF receiver 207 of the radar subsystem 205. The reflections received in block 810 may be reflected from moving objects (e.g., a person breathing with a heartbeat) and stationary objects. The pattern of the received, reflected radio waves may be approximately a mirror image of the pattern in which the radio waves were emitted; therefore, diagram 500 may approximately represent a pattern in which a burst of chirps is transmitted and a burst of reflected chirps is also received.

[0113] At block 715, a raw chirp waterfall, which may be referred to as a burst mode radar data stream, is generated based on the received and reflected radio waves. The radar subsystem or sensor may convert the received and reflected radar signals to the digital domain using an onboard ADC. In some embodiments, data received from different antennas of the radar subsystem is kept separate. The digital burst mode radar data stream may be output by the radar subsystem to a processing system, such as a processing system performing the functions of virtual continuous chirp generator 240 and / or human interaction engine 245.

[0114] In block 720, the burst mode radar data stream may be analyzed to determine whether any non-contact human interaction, such as a gesture or human presence, is present. The burst mode radar data stream may be directly analyzed for this human interaction, regardless of the state of the health monitoring performed in blocks 725 and 730. If a non-contact human interaction is detected in block 720, one or more actions may be taken in response to the detection in block 722. For example, a command may be executed or output to another device based on the detected gesture. If human presence is detected, actions may be taken such as illuminating the electronic display 140 (and presenting information) or outputting audio, such as synthesized voice.

[0115] Independently of blocks 720 and 722, method 700 may include execution of block 725. Thus, block 725 may be executed while blocks 720 and / or 722 are being executed. In block 725, the same burst mode radar data stream that was, or will be, analyzed for non-contact human interaction in block 720 may be processed to create a virtual continuous chirp radar data stream. As detailed in connection with FIG. 6 , a conversion, which may involve an averaging process, may be performed to convert a burst of radar chirps into a virtual continuous radar chirp. Multiple virtual continuous chirps are assembled into a virtual continuous radar data stream, which is output (as digital data) to one or more other components, which may be implemented using the same processing system that generated it or a separate processing system. By combining multiple chirps in a burst into a single virtual continuous mode chirp, the virtual continuous mode chirp can benefit from dithering and therefore have higher resolution than the individual chirps of the burst present in the burst mode radar data stream.

[0116] In block 730, health monitoring, such as sleep tracking, vital signs monitoring, cough monitoring, or sleep disorder attribution, may be performed as detailed in connection with the functionality of radar processing module 210. Health monitoring may function, or may function more effectively, based on a continuous radar data stream as opposed to a burst mode radar data stream. In other embodiments, some other form of monitoring or tracking may be performed based on a continuous radar data stream.

[0117] The methods, systems, and devices described above are exemplary. Various configurations may omit, substitute, or add various procedures or components as appropriate. For example, in alternative configurations, methods may be performed in an order different from that described, and / or various steps may be added, omitted, and / or combined. Also, features described with respect to particular configurations may be combined in various other configurations. Different aspects and elements of the configurations may be similarly combined. Also, because technology evolves, many elements are exemplary and do not limit the scope of the disclosure or claims.

[0118] Specific details are given in the description to provide a thorough understanding of example configurations (including implementations). However, the configurations may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques are shown without unnecessary detail so as not to obscure the configurations. This description only provides example configurations and does not limit the scope, applicability, or configurations of the claims. Rather, the preceding description of the configurations will provide those skilled in the art with an effective description for implementing the described techniques. Various changes can be made in the function and arrangement of elements without departing from the spirit or scope of the present disclosure.

[0119] Configurations may also be described as processes depicted as flow diagrams or block diagrams. While each operation may be described as a sequential process, many operations may be performed in parallel or simultaneously. The order of operations may also be rearranged. A process may have additional steps not included in the diagrams. Furthermore, example methods may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. If implemented by software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored in a non-transitory computer-readable medium, such as a storage medium. A processor may perform the described tasks.

[0120] While several example configurations have been described, various modifications, alternative constructions, and equivalents may be used without departing from the spirit of the disclosure. For example, the above elements may be components of larger systems, and other rules may take precedence over or modify the application of the present invention in other aspects. Also, many steps may be performed before, during, and after the above elements are considered.

Claims

1. A smart home device, a housing configured for placement of the smart home device in a user activity area, a user sleep area, or both; a radar sensor contained by the housing and configured to operate in a burst mode to transmit multiple bursts of radar chirps, receive reflections of the multiple bursts of radar chirps, and output a single radar data stream based on the reflections of the multiple bursts of radar chirps; a first amount of time elapsed between adjacent radar chirps of a burst of the plurality of bursts of the radar chirps is less than a second amount of time elapsed between adjacent bursts of the plurality of bursts of the radar chirps, and the smart home device further a processing system housed by the housing, the processing system including one or more processors, the processing system being in communication with the radar sensor; configured to perform a first set of operations on the single radar data stream to perform user activity recognition; a second set of operations configured to perform on the single radar data stream to perform user vital sign detection, the second set of operations including instructions to convert the single radar data stream into a virtual continuous mode radar data stream, the virtual continuous mode radar data stream including multiple virtual reflections of radar chirps equally spaced in time, and wherein no change in radar transmission mode is required to perform both the user activity recognition using the single radar data stream and the user vital sign detection using the virtual continuous mode radar data stream.

2. The smart home device of claim 1 , wherein the user activity recognition is gesture detection.

3. The instructions for converting the single radar data stream to the virtual continuous mode radar data stream include instructions for the processing system including:

3. The smart home device of claim 1 or 2, further comprising instructions for creating a virtual reflection of a radar chirp based on multiple radar chirps of a burst among the multiple bursts, the virtual reflection of the radar chirp being part of multiple virtual reflections of the radar chirp that are equally spaced in time.

4. wherein the processing system being configured to create the virtual return of the radar chirp based on the multiple radar chirps of the one of the plurality of bursts includes the processing system being configured to perform an averaging process; sampling a plurality of samples of each radar chirp of the plurality of radar chirps of the burst; averaging each sample of the plurality of samples with corresponding samples from other ones of the plurality of radar chirps for the burst to produce a plurality of averaged samples; and assembling the averaged samples to create the virtual reflection of the radar chirp.

5. The smart home device of claim 4 , wherein the user activity recognition is performed while the user vital signs detection is performed.

6. 6. The smart home device of claim 5, wherein the radar sensor outputs a frequency modulated continuous wave (FMCW) radar having a frequency between 57-64 GHz and a peak EIRP of less than 20 dBm.

7. A contactless health monitoring device, comprising: The housing and a radar sensor housed by the housing, the radar sensor configured to operate in a burst mode in which the radar sensor emits multiple bursts of radar chirps; a first amount of time elapsed between adjacent radar chirps of a burst of the plurality of radar chirp bursts is less than a second amount of time elapsed between adjacent bursts of the plurality of radar chirp bursts; The radar sensor outputs a burst mode radar data stream based on received radar chirp reflections of a plurality of bursts of the radar chirps, and the non-contact health monitoring device further comprises: a processing system housed by the housing, the processing system including one or more processors, the processing system being in communication with the radar sensor; configured to receive the burst mode radar data stream from the radar sensor; configured to analyze the burst mode radar data stream to identify non-contact human interactions; configured to convert the burst mode radar data stream into a virtual continuous mode radar data stream, the virtual continuous mode radar data stream including a plurality of virtual returns of radar chirps equally spaced in time; A non-contact health monitoring device configured to perform health monitoring of a user using the virtual continuous mode radar data stream.

8. The processing system being configured to convert the burst mode radar data stream to the virtual continuous mode radar data stream means that the processing system is:

8. The non-contact health monitoring device of claim 7, further configured to create a virtual reflection of a radar chirp based on multiple radar chirps of a burst among the plurality of bursts, the virtual reflection of the radar chirp being part of the plurality of virtual reflections of the radar chirp equally spaced in time.

9. wherein the processing system being configured to create the virtual return of the radar chirp based on the multiple radar chirps of the one of the plurality of bursts includes the processing system being configured to perform an averaging process; sampling a plurality of samples of each radar chirp of the plurality of radar chirps of the burst; averaging each sample of the plurality of samples with corresponding samples from other ones of the plurality of radar chirps for the burst to produce a plurality of averaged samples; and assembling the averaged samples to create the virtual reflection of the radar chirp.

10. The contactless health monitoring device of claim 9 , wherein the contactless human interaction is detected while health monitoring is being performed.

11. The non-contact health monitoring device of any one of claims 7 to 10, wherein the radar sensor outputs a frequency modulated continuous wave (FMCW) radar having a frequency between 57 and 64 GHz and a peak EIRP of less than 20 dBm.

12. The contactless health monitoring device according to any one of claims 7 to 11, wherein the contactless human interaction is a gesture.

13. The contactless health monitoring device according to any one of claims 7 to 11, wherein the contactless human interaction is presence detection.

14. The non-contact health monitoring device according to any one of claims 7 to 13, wherein the health monitoring includes monitoring the user's sleep.

15. 15. The non-contact health monitoring device of claim 7, wherein the virtual continuous mode radar data stream has higher resolution than the burst mode radar data stream due to dithering.

16. moreover, a wireless network interface housed by the housing; an electronic display housed by the housing; a microphone housed by the housing; and a speaker housed by the housing, wherein the wireless network interface, the electronic display, the microphone, and the speaker are in communication with the processing system.

17. The processing system further comprises: configured to receive spoken commands via the microphone; configured to transmit an indication of the spoken command to a cloud-based server system via the wireless network interface; configured to receive a response from the cloud-based server system via the wireless network interface; 17. The non-contact health monitoring device of claim 16, configured to output information obtained from the performed health monitoring via the electronic display, the speaker, or both based on the response from the cloud-based server system.

18. The non-contact health monitoring device of any one of claims 7 to 17, wherein the processing system is further configured to output a report based on the health monitoring.

19. 1. A method for monitoring non-contact human interactions and health monitoring using a single radar modulation mode, comprising: emitting radar chirps such that a radar sensor operating in burst mode emits multiple bursts of radar chirps; a first amount of time elapsed between subsequent radar chirps of a burst of the plurality of bursts of the radar chirp is less than a second amount of time elapsed between subsequent bursts of the plurality of bursts of the radar chirp; The radar sensor outputs a burst mode radar data stream based on radar chirp reflections of multiple bursts of the radar chirp, and the method further comprises: a processing system receiving the burst mode radar data stream from the radar sensor; the processing system analyzing the burst mode radar data stream for non-contact human interaction; and the processing system converting the burst mode radar data stream into a virtual continuous mode radar data stream, the virtual continuous mode radar data stream consisting of a plurality of virtual returns of radar chirps equally spaced in time, the method further comprising: The method for monitoring non-contact human interactions and health monitoring using a single radar modulation mode includes the processing system performing health monitoring of a user using the virtual continuous mode radar data stream.

20. The processing system being configured to convert the burst mode radar data stream to the virtual continuous mode radar data stream means that the processing system is:

20. The method of claim 19 for monitoring non-contact human interaction and health monitoring using a single radar modulation mode, comprising: configuring to create a virtual reflection of a radar chirp based on multiple radar chirps of a burst of the plurality of bursts, the virtual reflection of the radar chirp being part of multiple virtual reflections of the radar chirps equally spaced in time.

21. generating the virtual return of the radar chirp based on the multiple radar chirps of the one of the plurality of bursts, sampling a plurality of samples of each radar chirp of the plurality of radar chirps of the burst; averaging each sample of the plurality of samples with corresponding samples from other ones of the plurality of radar chirps for the burst to produce a plurality of averaged samples; and assembling the averaged samples to create the virtual reflection of the radar chirp.

22. 22. The method for monitoring non-contact human interaction and health monitoring using a single radar modulation mode according to claim 21, wherein the non-contact human interaction is a gesture.

23. 22. The method for monitoring non-contact human interaction and health monitoring using a single radar modulation mode as claimed in claim 21, wherein the non-contact human interaction is presence detection.

24. The method for monitoring non-contact human interaction and health monitoring using a single radar modulation mode according to any one of claims 21 to 23, wherein the health monitoring includes monitoring the user's sleep.

Images