Smart home device using a single radar transmission mode for activity recognition of active users and vital sign monitoring of inactive users

Abstract

Various arrangements are provided for monitoring contactless human interaction and health using a single radar transmission modulation mode. A radar sensor operating in a burst mode may output a radar chirp. A burst mode radar data stream may be monitored for contactless human interaction performed by an active user. The burst mode radar data stream may be converted into a virtual continuous mode radar data stream. The virtual continuous mode radar data stream may be used for health monitoring of an inactive user.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is related to the following applications: PCT application PCT / US2019 / 031290, entitled “Sleep Tracking and Vital Sign Monitoring Using Low Power Radio Waves,” filed on May 8, 2019; U.S. patent application Ser. No. 16 / 990,714, entitled “Contactless Sleep Detection and Disturbance Attribution for Multiple Users,” filed on August 11, 2020; U.S. patent application Ser. No. 16 / 990,705, entitled “Contactless Sleep Detection and Disturbance Attribution,” filed on August 11, 2020; and U.S. patent application Ser. No. 16 / 990,705, entitled “Contactless Cough Detection and Attribution,” filed on August 11, 2020. Attribution); U.S. patent application No. 16 / 990,720, entitled “Precision Sleep Tracking Using a Contactless Sleep Tracking Device,” filed on August 11, 2020; U.S. patent application No. 16 / 990,746, entitled “Initializing Sleep Tracking on a Contactless Health Tracking Device,” filed on August 11, 2020; and PCT application PCT / US2020 / 048388, entitled “Precision Sleep Tracking Using a Contactless Sleep Tracking Device,” filed on August 28, 2020. The entire disclosures of these applications are hereby incorporated by reference for all purposes. Background Art

[0003] A user can conveniently interact with a device without making any physical contact with the device (or another input device). For example, a gesture can be performed and detected by the device and interpreted as a command. While such contactless interaction is useful, the ability of the device to perform other tasks without contact may also be desired. Such other tasks may have competing requirements that need to be balanced with the detection of contactless interaction. Summary of the invention

[0004] Various embodiments related to a non-contact health monitoring device are described. In some embodiments, a non-contact health monitoring device is described. The device may include a housing. The device may include a radar sensor housed by the housing, the radar sensor configured to operate in a burst mode, in which the radar sensor transmits multiple bursts of radar chirps. A first amount of time that passes between a plurality of radar chirps of a burst in the plurality of bursts of radar chirps may be less than a second amount of time that passes between subsequent bursts of the plurality of bursts of radar chirps. The radar sensor outputs a burst mode radar data stream, which may be based on the plurality of bursts of radar chirps or reflections of the radar chirps. The device may include a processing system housed by the housing, the processing system including one or more processors that may communicate with the radar sensor. The processing system may be configured to receive a burst mode radar data stream from the radar sensor. The processing system may be configured to analyze the burst mode radar data stream to identify non-contact human interactions. The processing system may be configured to convert the burst mode radar data stream into a virtual continuous mode radar data stream. The processing system may be configured to perform health monitoring of a user using the virtual continuous mode radar data stream.

[0005] Embodiments of this apparatus may include one or more of the following features: a virtual continuous mode radar data stream may be composed of multiple virtual reflections of radar chirps equally spaced in time. The processing system is configured to convert the burst mode radar data stream into a virtual continuous mode radar data stream, which may include the processing system being configured to create virtual reflections of radar chirps based on multiple radar chirps of a burst in multiple bursts. The virtual reflection of the radar chirp may be a portion of multiple virtual reflections of radar chirps equally spaced in time. The processing system is configured to create virtual reflections of radar chirps based on multiple radar chirps of a burst in multiple bursts, which may include the processing system being configured to perform averaging. The processing system may include sampling multiple samples of each radar chirp in the multiple chirps of the burst. The processing system may include averaging each of the multiple samples with corresponding samples of other chirps from the multiple chirps of the burst to create multiple average samples. The processing system may include assembling the average samples to create virtual reflections of radar chirps. When performing health monitoring, non-contact human interaction may be detected. The radar sensor may output a frequency modulated continuous wave (FMCW) radar. The contactless human interaction may be a gesture. The contactless human interaction may be presence detection. The health monitoring may include sleep monitoring of the user. Due to jitter, the virtual continuous mode radar data stream may have a higher resolution than the burst mode radar data stream. The device may further include a wireless network interface housed by the housing. The device may further include an electronic display housed by the housing. The device may further include a microphone housed by the housing. The device may further include a speaker housed by the housing. The wireless network interface, the electronic display, the microphone, and the speaker may communicate with the processing system. The processing system may be further configured to receive verbal commands via the microphone. The processing system may be further configured to transmit an indication of the verbal command to a cloud-based server system via the wireless network interface. The processing system may be further configured to receive a response from the cloud-based server system via the wireless network interface. The processing system may be further configured to output information obtained from the performed health monitoring via an electronic display, a speaker, or both based on a response from the cloud-based server system. The processing system may be further configured to output a report based on the health monitoring.

[0006] In some embodiments, a method for monitoring non-contact human interaction and monitoring health using a single radar modulation mode is described. The method may include transmitting a radar chirp by a radar sensor operating in a burst mode, so that the radar sensor transmits multiple bursts of radar chirps. A first amount of time that passes between subsequent radar chirps of a burst in the multiple bursts of radar chirps may be less than a second amount of time that passes between subsequent bursts of the multiple bursts of radar chirps. The radar sensor outputs a burst mode radar data stream, which may be based on multiple bursts of radar chirps or reflections of radar chirps. The method may include receiving a burst mode radar data stream from the radar sensor by a processing system. The method may include analyzing the burst mode radar data stream by the processing system for non-contact human interaction. The method may include converting the burst mode radar data stream into a virtual continuous mode radar data stream by the processing system. The method may include performing health monitoring of a user using the virtual continuous mode radar data stream by the processing system.

[0007] Embodiments of this method may include one or more of the following features: the virtual continuous mode radar data stream may be composed of multiple virtual reflections of radar chirps that are equally spaced in time. The processing system is configured to convert the burst mode radar data stream into the virtual continuous mode radar data stream, which may include the processing system being configured to create virtual reflections of radar chirps based on multiple radar chirps of a burst in multiple bursts. The virtual reflections of the radar chirps may be a portion of multiple virtual reflections of radar chirps that are equally spaced in time. Creating virtual reflections of radar chirps based on multiple radar chirps of a burst in multiple bursts may include sampling multiple samples of each radar chirp in the multiple chirps of the burst. Creating virtual reflections of radar chirps based on multiple radar chirps of a burst in multiple bursts may include averaging each of the multiple samples with corresponding samples of other chirps from the multiple chirps of the burst to create multiple average samples. Creating a virtual reflection of the radar chirp based on a plurality of radar chirps of a burst in a plurality of bursts may include assembling average samples to create a virtual reflection of the radar chirp. The contactless human interaction may be a gesture. The contactless human interaction may be presence detection. The health monitoring may include sleep monitoring of the user.

[0008] In some embodiments, there are various arrangements of smart home devices. The smart home device may include a housing configured to place the smart home device in an area where a user is active, a user is sleeping, or both. The smart home device may include a radar sensor housed by the housing, the radar sensor configured to operate in a burst mode, in which the radar sensor transmits multiple bursts of radar chirps, receives reflections of multiple bursts of radar chirps, and outputs a single radar data stream based on the reflections of multiple bursts of radar chirps. A first amount of time that can pass between adjacent radar chirps of a burst in the multiple bursts of radar chirps is less than a second amount of time that passes between adjacent bursts of the multiple bursts of radar chirps. The smart home device may include a processing system housed by the housing, the processing system including one or more processors that communicate with the radar sensor. The processing system may be configured to perform a first set of operations on a single radar data stream to perform user activity recognition. The processing system may be configured to perform a second set of operations on a single radar data stream to perform user vital sign detection, wherein performing both user activity recognition and user vital sign detection using a single radar data stream does not require changing the radar transmission mode.

[0009] Embodiments of this smart home device may include one or more of the following features: The second set of operations may include instructions for converting a single radar data stream into a virtual continuous mode radar data stream. The user activity recognition may be gesture detection. The virtual continuous mode radar data stream may include multiple virtual reflections of radar chirps that are equally spaced in time. The instructions for converting the single radar data stream into the virtual continuous mode radar data stream may include instructions for causing the processing system to create virtual reflections of the radar chirp based on multiple radar chirps of a burst in multiple bursts, wherein the virtual reflection of the radar chirp is a portion of the multiple virtual reflections of the radar chirp that are equally spaced in time.

[0010] The processing system is configured to create a virtual reflection of the radar chirp based on multiple radar chirps of a burst in multiple bursts may include the processing system is configured to perform an averaging process, including: sampling multiple samples of each radar chirp in the multiple chirps of the burst; averaging each of the multiple samples with corresponding samples of other chirps from the multiple chirps of the burst to create multiple average samples; and assembling the average samples to create a virtual reflection of the radar chirp. User activity recognition can be performed when performing detection of user vital signs. The radar sensor can output a frequency modulated continuous wave (FMCW) radar with a frequency between 57-64GHz and a peak EIRP less than 20dBm. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] A further understanding of the nature and advantages of the various embodiments may be achieved by reference to the following drawings. In the drawings, similar components or features may have the same reference label. In addition, various components of the same type may be distinguished by following the reference label with a dash and a second label that distinguishes between similar components. If only the first reference label is used in the specification, the specification applies to any one of the similar components having the same first reference label, regardless of the second reference label.

[0012] Figure 1 An embodiment of a system for performing contactless health and user interaction monitoring is illustrated.

[0013] Figure 2A An embodiment of a health tracking system that uses radar to detect human interactions and perform health monitoring is illustrated.

[0014] Figure 2B An embodiment of a health tracking system using radar to detect human interactions and integrated beam aiming to perform health monitoring is illustrated.

[0015] Figure 2C An embodiment of a frequency modulated continuous wave radar radio wave output by a radar subsystem is shown.

[0016] Figure 3A An embodiment of a contactless health tracking device is illustrated.

[0017] Figure 3B An exploded view of an embodiment of a contactless fitness tracking device is illustrated.

[0018] Figure 4 An embodiment of a status system for determining when a person is sleeping is illustrated.

[0019] Figure 5 Schematic diagram illustrating radar chirps received in burst mode.

[0020] Figure 6 Schematic diagram illustrating a radar chirp received in burst mode and converted into a virtual continuous mode.

[0021] Figure 7 An embodiment of a method for monitoring contactless human interactions and monitoring health using a single radar modulation pattern is illustrated. DETAILED DESCRIPTION

[0022] The embodiments detailed herein focus on devices that perform contactless health monitoring (e.g., contactless sleep monitoring) while also monitoring contactless human interactions. Contactless health monitoring may involve tracking the sleep of a user (non-active user), monitoring the user's vital signs, tracking the user's coughs, and the like. Contactless health monitoring may be performed using radar. Contactless human interactions may be gestures performed by an active user (e.g., a user moves his hand or arm in a specific way that will be interpreted as a command) or presence (e.g., detecting the presence of a user near the device). The device may monitor contactless human interactions using radar.

[0023] Since both non-contact health monitoring and non-contact human interaction monitoring can use radar, there may be competing needs between these two types of monitoring. For example, non-contact health monitoring can be effectively performed when the radar sensor is operated in continuous mode. In continuous mode, radar chirps can be received periodically. However, to effectively perform human interaction monitoring, the radar sensor can be operated in burst mode. In burst mode, a series of radar chirps (a "burst") is transmitted, followed by a wait period before the next radar chirp burst is started. The waiting period between bursts can limit the total amount of energy emitted into the user's environment, thereby reducing the user's exposure to RF energy.

[0024] In the embodiments detailed herein, the radar sensor can be operated in burst mode. Thus, the radar data stream output by the radar sensor can be optimized for human interaction monitoring. Processing can be performed on the radar data stream to create a virtual continuous mode radar data stream. This virtual continuous mode radar data stream can be created so that the virtual continuous mode radar data stream includes higher quality data than if the radar sensor itself were operated in continuous mode.

[0025] By creating a virtual continuous mode radar data stream, a single device or system receiving the radar data stream can effectively monitor human interactions in burst mode and perform health monitoring in continuous mode. Therefore, a single device can effectively respond to non-contact human interactions while performing health monitoring without the user having to switch the mode of the device or system. Similarly, the device or system can continuously operate the radar sensor in burst mode without having to change the mode in which the radar sensor operates. More specifically, the device or system continuously creates a virtual continuous mode radar data stream, thereby allowing monitoring of non-contact user interactions and non-contact health monitoring to occur simultaneously and independently of each other.

[0026] Further details regarding such embodiments and additional embodiments can be understood in conjunction with the accompanying drawings. Figure 1An embodiment of a system 100 for performing contactless health and user interaction monitoring is illustrated. System 100 may include: a contactless health and human interaction monitoring device 101 ("device 101"); a network 160; and a cloud-based server system 170. Device 101 may include: a processing system 110; a sleep data storage device 118; a radar subsystem 120; an environmental sensor suite 130; an electronic display 140; a wireless network interface 150; and a speaker 155. In general, device 101 may include a housing that houses all of the components of device 101. Further details regarding such possible housings are provided with reference to FIG. Figure 3A and Figure 3B The device 101 may also be referred to as a health monitoring device, a health tracking device, a sleep monitoring device, a sleep tracking device, or a home assistance device.

[0027] The processing system 110 may include one or more processors configured to perform various functions, such as the radar processing module 112; the sleep state detection engine 114; and the functions of the environmental event correlation engine 116. The processing system 110 may include one or more dedicated or general-purpose processors. Such dedicated processors may include processors specifically designed to perform the functions detailed herein. Such dedicated processors may be ASICs or FPGAs, which are general-purpose components that are physically and electrically configured to perform the functions detailed herein. Such general-purpose processors may execute dedicated software stored using one or more non-transitory processor-readable media, such as random access memory (RAM), flash memory, hard disk drive (HDD) or solid-state drive (SSD).

[0028] Radar subsystem 120 (also referred to as a radar sensor) may be a single integrated circuit (IC) that transmits, receives, and outputs a radar data stream indicative of received reflected radio waves. The output of radar subsystem 120 may be analyzed using radar processing module 112 of processing system 110. Further details regarding radar subsystem 120 and radar processing module 112 are provided with respect to FIG. 2.

[0029] Device 101 may include one or more environmental sensors, such as all, one, or some combination of environmental sensors provided as part of environmental sensor suite 130. Environmental sensor suite 130 may include: light sensor 132; microphone 134; temperature sensor 136; and passive infrared (PIR) sensor 138. In some embodiments, there may be multiple instances of some or all of these sensors. For example, in some embodiments, there may be multiple microphones. Light sensor 132 may be used to measure the amount of ambient light present in the general environment of device 101. Microphone 134 may be used to measure the level of ambient noise present in the general environment of device 101. Temperature sensor 136 may be used to measure the ambient temperature of the general environment of device 101. PIR sensor 138 may be used to detect moving living bodies (e.g., people, pets) within the general environment of device 101. Other types of environmental sensors are also possible. For example, a camera and / or humidity sensor may be incorporated as part of environmental sensor suite 130. In some embodiments, some data, such as humidity data, may be obtained from a nearby weather station that has data available via the Internet.

[0030] The device 101 may include various interfaces. The display 140 may allow the processing system 110 to present information for viewing by one or more users. The wireless network interface 150 may allow communication using a wireless local area network (WLAN) such as a WiFi-based network. The speaker 155 may allow output of sounds such as synthesized speech. For example, a response to a verbal command received via the microphone 134 may be output via the speaker 155 and / or the display 140. The verbal command may be analyzed locally by the device 101, or may be transmitted to a cloud-based server system 170 via the wireless network interface 150 for analysis. Based on the analysis of the verbal command, the response may be sent back to the device 101 via the wireless network interface 150 to be output via the speaker 155 and / or the display 140. Additionally or alternatively, the speaker 155 and the microphone 134 may be configured together for active acoustic sensing, including ultrasonic acoustic sensing. Additionally or alternatively, other forms of wireless communication are also possible, such as using a low-power wireless mesh network radio and a protocol (e.g., Thread) to communicate with various smart home devices. In some embodiments, a wired network interface such as an Ethernet connection may be used for communication with the network. In addition, wireless communication is developing towards fifth generation (5G) and sixth generation (6G) standards and technologies to provide greater throughput and lower latency, thereby enhancing mobile broadband services. 5G and 6G technologies also provide new service categories for vehicle-to-everything (V2X), fixed wireless broadband, and the Internet of Things (IoT) through control and data channels. Such standards and technologies can be used for communication of device 101.

[0031] Low power wireless mesh network radios and protocols can be used to communicate with power-limited devices. Power-limited devices can be devices that are specifically powered by batteries. Such devices can rely entirely on one or more batteries for power, so the amount of power used for communication can be kept low to reduce the frequency of needing to replace one or more batteries. In some embodiments, the power-limited device can have the ability to communicate via a relatively high power network (e.g., WiFi) and a low power mesh network. The power-limited device can use a relatively high power network infrequently to save power. Examples of such power-limited devices include environmental sensors (e.g., temperature sensors, carbon monoxide sensors, smoke sensors, motion sensors, presence detectors) and other forms of remote sensors.

[0032] Wireless network interface 150 may allow wireless communication with network 160. Network 160 may include one or more public and / or private networks. Network 160 may include a private local wired or wireless network, such as a home WLAN. Network 160 may also include a public network such as the Internet. Network 160 may allow device 101 to communicate with a remotely located cloud-based server system 170.

[0033] The cloud-based server system 170 may provide various services to the device 101. With respect to sleep data, the cloud-based server system 170 may include processing and storage services for sleep-related data. Figure 1 The embodiments of the present invention involve the processing system 110 performing sleep state detection and environmental event association, but in other embodiments, such functions may be performed by the cloud-based server system 170. Moreover, in addition to or in lieu of the sleep data storage device 118 for storing sleep data, the sleep-related data may be stored by the cloud-based server system 170, for example mapped to a public user account to which the device 101 is linked. If multiple users are monitored, the sleep data may be stored and mapped to a master user account or accounts of corresponding users.

[0034] The cloud-based server system 170 may provide other cloud-based services in addition or alternatively. For example, the device 101 may be used as a home assistance device in addition. The home assistance device may respond to voice queries from the user. In response to detecting a voice trigger phrase spoken via microphone 134, the device 101 may record audio. The audio stream may be transmitted to the cloud-based server system 170 for analysis. The cloud-based server system 170 may perform a speech recognition process, using a natural language processing engine to understand queries from the user, and provide a response to be output by the device 101 as a synthesized voice, an output to be presented on the electronic display 140, and / or a command to be executed by the device 101 (e.g., increase the volume of the device 101) or sent to some other smart home device. In addition, queries or commands may be submitted to the cloud-based server system 170 via the electronic display 140, which may be a touch screen. For example, the device 101 may be used to control various smart home devices or home automation devices. Such commands may be sent directly by the device 101 to the device to be controlled, or may be sent via the cloud-based server system 170.

[0035] Based on the data output by the radar processing module 112, the sleep state detection engine 114 can be used to determine whether the user is likely to be asleep or awake. The sleep state detection engine 114 can be performed by a state machine, such as regarding Figure 4 Detailed description, or can use the state identified using such a state machine to determine whether the user may be awake or asleep. For example, if it is determined that the user is in bed and is still for at least a defined period of time, the user can be identified as asleep. The output of the sleep state detection engine 114 can be used by the environmental event correlation engine 116. The environmental event correlation engine 116 can analyze the data received from the environmental sensor suite 130. The data from each environmental sensor device can be monitored: 1) the environmental condition increases above a fixed defined threshold; and / or 2) the environmental condition increases by at least a predetermined amount or percentage. As an example, data indicating the light level in the surrounding environment can be continuously or periodically output by the light sensor 132. The environmental event correlation engine 116 can determine 1) whether the amount of ambient lighting has increased from below a fixed defined threshold to above a fixed defined threshold; and / or 2) whether the amount of ambient lighting has increased by at least a predetermined amount of percentage. If option 1, 2 or both occur, it can be determined that an environmental event has occurred. The environmental event can be time-stamped by the environmental event correlation engine 116. The environmental event correlation engine 116 can then determine whether the user wakes up attributable to the identified environmental event.

[0036] In addition to radar processing module 112, the radar data stream output by radar subsystem 120 may also be output to human interaction engine 245. Human interaction engine 245 may use the radar data stream to determine whether a human interaction has been performed, such as a human moving within a defined distance of device 101 or a gesture. Further details on how radar subsystem 120, radar processing module 112, and human interaction engine 245 work and interact will be described in detail below. Figure 2A and 2B Describe in detail.

[0037] Figure 2A An embodiment of a health monitoring system 200A ("system 200A") that uses radar to detect human interaction and perform health monitoring is illustrated. System 200A may include a radar subsystem 205 (which may represent an embodiment of radar subsystem 120); a radar processing module 210 (which may represent an embodiment of radar processing module 112); a beam steering module 230; a virtual continuous chirp generator 240; and a human interaction engine 245.

[0038] The radar subsystem 205 may include an RF transmitter 206, an RF receiver 207, and radar processing circuitry 208. The RF transmitter 206 may transmit radio waves in the form of, for example, a continuous wave (CW) radar. The RF transmitter 206 may use a frequency modulated continuous wave (FMCW) radar. FMCW radars may operate in a continuous sparse sampling mode over a relatively long period of time. The RF transmitter 206 may include one or more antennas and may transmit at or about 60 GHz. The frequency of the transmitted radio waves may be repeatedly swept from a low frequency to a high frequency (or vice versa). The power level used for the transmission may be very low, giving the radar subsystem 205 an effective range of a few meters or even less. About Figure 2C Further details are provided regarding the radio waves generated and transmitted by radar subsystem 205 .

[0039] The RF receiver 207 includes one or more antennas different from the transmitting antenna, and can receive radio wave reflections from nearby objects of the radio waves transmitted by the RF transmitter 206. The radar processing circuit 208 can interpret the reflected radio waves by mixing the radio waves being transmitted with the reflected received radio waves, thereby generating a mixed signal that can be analyzed for distance. Based on this mixed signal, the radar processing circuit 208 can output raw waveform data, which can also be referred to as a raw chirp waterfall, to be analyzed by a separate processing entity. The radar subsystem 205 can be implemented as a single integrated circuit (IC), or the radar processing circuit 208 can be a component separate from the RF transmitter 206 and the RF receiver 207. In some embodiments, the radar subsystem 205 is integrated as part of the device 101 so that the RF transmitter 206 and the RF receiver 207 point in the same direction as the display 140. In other embodiments, an external device including the radar subsystem 205 can be connected to the device 101 via wired or wireless communication. For example, the radar subsystem 205 can be an additional device for a home auxiliary device.

[0040] The RF subsystem 205 outputs a radar data stream. Since the RF subsystem 205 operates in burst mode, the radar data stream output may be referred to as a burst mode radar data stream. The burst mode radar data stream may be output by the radar subsystem 205 to the human interaction engine 245 and the virtual continuous chirp generator 240. The human interaction engine 245 and the virtual continuous chirp generator 240 may be implemented as software executed using the same processing system as the radar processing module 210. In other embodiments, a separate processing system may be used for the virtual continuous chirp generator 240, the human interaction engine 245, or both. In other embodiments, dedicated hardware may be used to perform the functions of the virtual continuous chirp generator 240, the human interaction engine 245, or both.

[0041] The human interaction engine 245 can use the burst mode radar data stream to determine whether any human interaction has occurred. The processing performed by the human interaction engine 245 can occur independently of the processing performed by the virtual continuous chirp generator 240 and the radar processing module 210. Therefore, whether health monitoring is performed by the radar processing module 210 or a component using data output by the radar processing module 210 may be unrelated to the human interaction engine 245 monitoring non-contact human interactions (such as presence and posture). To be clear, "non-contact" means that there is no physical contact between the user and the device on which the system 200A is implemented. More specifically, remote sensing, such as radar, is used to monitor the user.

[0042] The virtual continuous chirp generator 240 also receives a burst mode radar data stream. The virtual continuous chirp generator 240 performs processing on the received burst mode radar data stream and outputs a virtual continuous mode radar data stream. Figure 6 Details are provided regarding the processing performed by virtual continuous chirp generator 240 to create a virtual continuous mode radar data stream. The output of virtual continuous chirp generator 240 is provided to radar processing module 210. Radar processing module 210 performs processing on the virtual continuous mode radar data stream as if radar subsystem 205 were outputting a continuous mode radar data stream directly to radar processing module 210.

[0043] For the radar subsystem 205, if FMCW is used, a well-defined FMCW range can be defined. Within this range, the distance to an object can be accurately determined. However, outside this range, detected objects may be incorrectly interpreted as being closer than objects within the well-defined range. This incorrect interpretation may be due to the frequency of the mixed signal and the sampling rate of the ADC used by the radar subsystem to convert the received analog signal to a digital signal. If the frequency of the mixed signal is higher than the Nyquist sampling rate of the ADC, the digital data output by the ADC representing the reflected radar signal may be incorrectly represented (e.g., represented as a lower frequency indicating a closer object).

[0044] The radar processing module 210 may include one or more processors. The radar processing module 210 may include one or more dedicated or general-purpose processors. A dedicated processor may include a processor specifically designed to perform the functions detailed herein. Such a dedicated processor may be an ASIC or FPGA, which are general-purpose components physically and electrically configured to perform the functions detailed herein. A general-purpose processor may execute dedicated software stored using one or more non-transitory processor-readable media, such as a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD). The radar processing module 210 may include: a mobile filter 211; a frequency enhancer 212; a range-vital sign transformation engine 213; a range gating filter 214; a spectrum summation engine 215; and a neural network 216. Each of the components of the radar processing module 210 may be implemented using software, firmware, or as dedicated hardware.

[0045] The virtual continuous mode radar data stream from the virtual continuous chirp generator 240 can be received by the radar processing module 210 and first processed using the mobile filter 211. In some embodiments, it is important that the mobile filter 211 is the initial component for performing filtering. That is, in some embodiments, the processing performed by the radar processing module 210 is not interchangeable. Typically, vital sign determination and sleep monitoring can be performed when the monitored user is sleeping or trying to sleep in bed. In this environment, there may usually be little movement. This movement can be attributed to the user moving in the bed (for example, turning over when trying to fall asleep or sleeping) and the user's vital signs, including movement caused by breathing and movement caused by the heartbeat of the monitored user. In this environment, a large part of the radio waves emitted from the RF transmitter 206 may be reflected by static objects near the monitored user, such as mattresses, box springs, bed frames, walls, furniture, bedding, etc. Therefore, a large part of the raw waveform data received from the radar subsystem 205 may be irrelevant to user movement and user's vital measurements.

[0046] The mobile filter 211 may include a waveform buffer that buffers the "chirp" or slices of the received raw waveform data. For example, sampling may occur at a rate of 10 Hz. In other embodiments, sampling may be slower or faster. In some embodiments, the mobile filter 211 may buffer the received raw waveform chirp for twenty seconds. In other embodiments, the buffered raw waveform data is buffered for a shorter or longer duration. This buffered raw waveform data may be filtered to remove raw waveform data indicating a fixed object. That is, for moving objects, such as the chest of a monitored user, the user's heartbeat and breathing rate will affect the distance and speed measurements made by the radar subsystem 205 and output to the mobile filter 211. This movement of the user will result in "jitter" in the raw waveform data received during the buffering period. More specifically, jitter refers to the phase shift caused by the moving object reflecting the transmitted radio waves. Instead of using the reflected FMCW radio waves to determine the speed of the moving object, the phase shift caused by movement in the reflected radio waves may be used to measure vital statistics, including heart rate and breathing rate, as described in detail herein.

[0047] For fixed objects such as furniture, zero phase shift (i.e., no shake) will appear in the raw waveform data during the buffered time period. The motion filter 211 can subtract such raw waveform data corresponding to the fixed object so that the raw waveform data indicating motion is passed to the frequency enhancer 212 for further analysis. For the remainder of the radar processing module 210 processing, the raw waveform data corresponding to the fixed object can be discarded or otherwise ignored.

[0048] In some embodiments, an infinite impulse response (IIR) filter is incorporated as part of the movement filter 211. Specifically, a single-pole IIR filter can be implemented to filter out raw waveform data that does not indicate movement. Therefore, the single-pole IIR filter can be implemented as a high-pass, low-cut filter that prevents raw waveform data indicating movement below a specific frequency from being passed to the frequency enhancer 212. The cutoff frequency can be set based on known limitations on human vital signs. For example, the respiratory rate can be expected to be between 10 and 60 breaths per minute. The filter can exclude movement data indicating a frequency lower than 10 breaths per minute. In some embodiments, a bandpass filter can be implemented to exclude raw waveform data indicating movement at high frequencies that are impossible or inconceivable for human vital signs. For example, for a person at rest or near rest, a heart rate that may be expected to be higher than the respiratory rate is unlikely to exceed 150 beats per minute. Raw waveform data indicating higher frequencies can be filtered out by a bandpass filter.

[0049] In some embodiments, the frequency of the raw waveform data that the mobile filter 211 passes to the frequency enhancer 212 can be further fine-tuned. For example, during the initial configuration phase, the user can provide information about the monitored user (e.g., himself, a child), such as age data. Table 1 indicates typical breathing rates for different ages. Similar data may exist for heart rate. The filter can be configured to exclude data outside the expected breathing rate, heart rate range, or both.

[0050] age Respiratory rate range (breaths per minute) Birth - 6 weeks 30-60 6 months 25-40 3 years old 20-30 6 years old 18-25 10 years old 17-23 Adults 12-18 65 to 80 years old 12-28 >80 years old 10-30

[0051] Table 1

[0052] The measured vital signs of a monitored user are periodic pulse events: the user's heart rate may vary over time, but the user's heart can be expected to continue to beat periodically. Such beats are not sinusoidal functions, but can be understood as pulse events, more akin to a square wave with a relatively low duty cycle that induces motion in the user's body. Similarly, the user's breathing rate may vary over time, but breathing is a periodic function performed by the user's body that is similar to a sine function, except that the user's exhalations are typically longer than their inhalations. In addition, at any given time, a specific window of waveform data is being analyzed. Due to the specific time window of waveform data being analyzed, even a perfect sinusoid within the window may result in spectral leakage in the frequency domain. Frequency components due to this spectral leakage should be weakened.

[0053] The frequency enhancer 212 can work with the range-vital sign transformation engine 213 to determine one (e.g., respiration) or two (e.g., respiration plus heartbeat) frequency components of the raw waveform data. The frequency enhancer 212 can use frequency windowing, such as a 2D Hamming window (other forms of windowing are possible, such as a Hann window) to enhance important frequency components of the raw waveform data and weaken or remove waveform data that is attributable to spectral leakage outside the defined frequency window. Such frequency windowing can reduce the amplitude of the raw waveform data that may be due to processing artifacts. The use of frequency windowing can help reduce the impact of data-dependent processing artifacts while retaining data related to being able to determine heart rate and respiratory rate separately.

[0054] For a user that can be located within 1 to 2 meters of the monitored user or users to detect breathing and heart rate (e.g., using a Figure 2C A fixed bedside FMCW radar-based monitoring device (with a mid-transmitted radar) that enhances 2D Hamming windows of respiratory rate in the range of 10 to 60 bpm (0.16 Hz to 1 Hz) and heart rate in the range of 30 to 150 bpm (0.5 to 2.5 Hz) provides a good enough signal for reliable measurement without the need for prior knowledge of the subject's age or medical history.

[0055] Since heart rate and respiration rate are periodic pulse events, the frequency domain heart rate and respiration rate can be represented by different fundamental frequencies, but each fundamental frequency may have many harmonic components at higher frequencies. One of the main purposes of the frequency enhancer 212 can be to prevent the frequency ripples of the harmonics of the monitored user's respiration rate from affecting the frequency measurement of the monitored user's heart rate (or vice versa). Although the frequency enhancer 212 can use a 2D Hamming window, it should be understood that other windowing functions or isolation functions can be used to help isolate the frequency ripples of the monitored user's respiration rate from the frequency ripples of the monitored user's heart rate.

[0056] The range-vital sign transformation engine 213 analyzes the received motion filtered waveform data to identify and quantify the amplitude of movement at a specific frequency. More specifically, the range-vital sign transformation engine 213 analyzes the phase sway over time to detect relatively small movements caused by user vital signs (e.g., breathing rate and heart rate) with relatively low frequencies. The analysis of the range-vital sign transformation engine 213 can assume that the frequency components of the motion waveform data are sinusoidal. In addition, the transformation used by the range-vital sign transformation engine 213 can also identify the distance at which the frequency is observed. The frequency, amplitude, and distance can all be determined at least in part because the radar subsystem 205 uses an FMCW radar system.

[0057] Before applying the transformation of the range-vital sign transformation engine 213, a zero padding process can be performed by the range-vital sign transformation engine 213 to add multiple zeros to the motion filtered raw waveform data. By performing the zero padding process, the resolution in the frequency domain can be effectively improved, thereby allowing more accurate low-rate measurements (e.g., low heart rate, low respiratory rate). For example, zero padding can help to numerically improve the resolution of detecting the difference of half a breath per minute compared to the resolution of breaths per minute without zero padding. In some embodiments, three to four times the number of zeros can be added compared to the buffered sample size of the raw waveform data. For example, if 20 seconds of buffered raw waveform data are analyzed, 60 to 80 seconds of zero padding can be added to the sample. Specifically, it was found that three to four times the zero padding range of the sample can significantly improve the resolution without making the transformation process too complicated (therefore, making the processor intensive).

[0058] To determine the amount of zero padding to perform, equations 1-3 may be used. In equation 1, RPM_resolution may ideally be less than 1.

[0059]

[0060] n_FFT_slow_time_min=(nearest_power_of_2)(60*chirp-rate) Equation 2

[0061] In some embodiments, a chirp rate (chirp_rate) of 30 Hz may be used. This frequency may have sufficient margin relative to the Nyquist limit of the upper limits of the respiratory rate and heart rate. Therefore, n_FFT_slow_time_min may be 2048. Given a 20 second window for estimating respiratory statistics, Equation 3 yields a value of 600.

[0062] n_chirpS_for_reSpiration = 20*chirp-rate = 600 Equation 3

[0063] This value of 600 is less than the required vital sign FFT size and causes the range-vital sign transform engine 213 to perform 3x to 4x zero padding. The balance of how much zero padding to perform can be based on the associated increase in frequency resolution and the amount of computation required to perform the FFT. It has been found that 3x to 4x zero padding can provide adequate resolution for heart rate and respiration rate while moderating the amount of computation that needs to be performed.

[0064] The range-to-vital-signs transformation engine 213 may perform a series of Fourier transforms (FTs) to determine the frequency components of the received raw waveform data output by the frequency enhancer 212. Specifically, a series of fast Fourier transforms (FFTs) may be performed by the range-to-vital-signs transformation engine 213 to determine the specific frequencies and amplitudes of the waveform data at those frequencies.

[0065] The waveform data obtained over a period of time can be represented in multiple dimensions. The first dimension (e.g., along the y-axis) can relate to multiple samples of waveform data from a particular chirp, while the second dimension (e.g., along the x-axis) relates to a specific sample index of the waveform data collected across multiple chirps. There is a third dimension of data (e.g., along the z-axis) that indicates the intensity of the waveform data.

[0066] Multiple FFTs may be performed based on a first dimension and a second dimension of the waveform data. FFTs may be performed along each of the first and second dimensions: an FFT may be performed for each chirp, and an FFT may be performed for each specific sample index across multiple chirps that occurred during the time period. An FFT performed on the waveform data of a specific reflected chirp may indicate one or more frequencies, which in an FMCW radar indicate the distance at which an object reflecting the transmitted radio waves is located. An FFT performed for a specific sample index across multiple chirps may measure the frequency of phase sloshes across multiple chirps. Thus, an FFT in the first dimension may provide a distance at which vital statistics data exist, while an FFT in the second dimension may provide a frequency of vital statistics data. The outputs of the FFTs performed in two dimensions indicate: 1) the frequency of vital statistics data; 2) the range at which vital statistics data are measured; and 3) the amplitude of the measured frequency. In addition to the values ​​generated due to the presence of vital signs statistics in the data, there may also be noise, for example, filtered using the spectral summation engine 215. The noise is generated in part because heart rate and respiration are not perfect sine waves.

[0067] It should be clear that the transform performed by the range-vital signs transform engine 213 is different from the range-Doppler transform. Rather than analyzing changes in velocity (as in the range-Doppler transform), the periodic changes in phase shift over time are analyzed as part of the range-vital signs transform. The range-vital signs transform is adjusted to identify small movements (e.g., breathing, heart rate) that occur over relatively long periods of time by tracking phase changes (called phase wobble). As previously described, zero padding is performed to allow sufficient resolution to accurately determine heart rate and respiratory rate.

[0068] The range gating filter 214 is used to monitor the defined range of interest and exclude the waveform data generated by the movement beyond the defined range of interest. For the arrangement described in detail herein, the defined range of interest can be 0 to 1 meter. In some embodiments, this defined range of interest can be different, or may be set by the user (e.g., via a training or setup process) or by a service provider. In some embodiments, the goal of this arrangement can be to monitor the person closest to the device (and exclude or separate the data of any other person farther away (e.g., the person sleeping next to the monitored person)). In other embodiments, if two people are to be monitored, the data can be separated, as described in detail with respect to Figure 12. Therefore, the range-vital sign transformation engine 213 and the range gating filter 214 are used to separate, exclude or remove the movement data of objects attributed to the defined range of interest, and to sum the energy of the movement data of objects attributed to the defined range of interest. The output of the range gating filter 214 can include data with a determined range within the allowable range of the range gating filter 214. The data can further have a frequency dimension and an amplitude. Therefore, the data can have three dimensions.

[0069] The spectrum summation engine 215 can receive the output from the range pass filter 214. The spectrum summation engine 215 can be used to transfer the measured energy of the harmonic frequencies of the heart rate and the respiratory rate, and sum the harmonic frequency energy to the energy of the fundamental frequency. This function can be called the harmonic sum spectrum (HSS). The heart rate and the respiratory rate are not sinusoidal; therefore, in the frequency domain, the harmonics will appear at frequencies higher than the fundamental frequency of the user's respiratory rate and the fundamental frequency of the user's heart rate. One of the main purposes of the spectrum summation engine 215 is to prevent the harmonics of the respiratory rate of the monitored user from affecting the frequency measurement of the heart rate of the monitored user (or vice versa). HSS can be performed in the second order by summing the original spectrum with the downsampled instance of the spectrum (multiplied by a factor of 2). This process can also be applied to higher order harmonics so that their corresponding spectra are added to the spectrum at the fundamental frequency.

[0070] At this stage, for a person lying still in bed (except for movement due to breathing and heart rate), two main frequency peaks are expected to appear in the frequency data. However, if the monitored user moves physically, such as turning over in bed, the energy will be significantly distributed across the spectrum (a wider distribution). This large physical movement can manifest itself as a large number of small peaks in the frequency data. If the bed is empty, rather than someone is present, there may be no or almost no frequency components above the background noise because the motion filter 211 has previously filtered the raw waveform data corresponding to a static object. The distribution and amplitude of the frequency peaks on the spectrum can be used to determine whether the user is likely to be awake or asleep.

[0071] The spectral summation engine 215 can output a feature vector indicating a heart rate (e.g., in beats per minute) and a respiration rate (e.g., in respirations per minute). The feature vector can indicate a frequency and an amplitude. The neural network 216 can be used to determine whether the heart rate and / or respiration rate indicated in the feature vector output of the spectral summation engine 215 should be considered valid. Therefore, the heart rate and respiration rate output by the spectral summation engine 215 can be stored, presented to the user, and / or considered valid based on the output of the neural network 216. The neural network 216 can be trained (e.g., using supervised learning performed using a training data set) to output one of three states, such as the states shown in Table 2, by performing spectral analysis. When it is determined that the user is present and the detected movement is due to the user's vital signs, the vital statistics data can be considered valid.

[0072] Each state in Table 2 is associated with a different spectral energy and spectral sparsity distribution. Spectral energy refers to the sum of energy across the spectrum detected due to the presence of motion in the monitoring area. Spectral sparsity indicates whether the movement tends to be distributed over a wide range of frequencies or is concentrated on a few specific frequencies. For example, if energy peaks occur at fewer frequencies, such as when a user's vital signs are detected (but no other movement is detected), the spectral sparsity is high. However, if peaks (exceeding a threshold) or some other form of determination based on a threshold criterion (based at least in part on amplitude) occur at many frequencies, the spectral sparsity is low.

[0073] For example, motion due to a vital sign such as a heartbeat may indicate significant movement (e.g., high spectral energy) at a particular frequency (e.g., high spectral sparsity); motion due to a user moving a limb may also indicate significant movement (high spectral energy), but may have low spectral sparsity. A neural network may be trained to distinguish each state based on the spectral energy distribution output by the spectral summation engine 215. Thus, two features may be provided to the neural network 216, namely a first value representing spectral energy and a second value representing spectral sparsity.

[0074] The output of the spectral summation engine 215 can be characterized as a feature vector having a first dimension of frequency and a second dimension of amplitude. A first value representing spectral energy can be calculated by determining the maximum amplitude present in the feature vector output by the spectral summation engine 215. This maximum amplitude value can be normalized to a value within 0 to 1. A second value representing spectral sparsity can be calculated by subtracting the median amplitude of the feature vector from the maximum amplitude. Here, the calculated sparsity can be normalized to a value within 0 to 1.

[0075] Table 2 shows an overview of how the trained neural network uses the features of spectral energy and spectral sparsity as a way to classify the status of the monitoring area.

[0076]

[0077] Table 2

[0078] The state of the monitoring area classified by the neural network 216 can be used to determine the sleep state of the monitored user, or more generally, to determine whether the user is moving in bed or stationary in bed. The state of the monitoring area determined by the classification performed by the neural network 216 can be further used to determine whether the vital statistics output by the spectral summation engine 215 should be trusted or ignored. For accurate vital statistics determination, when the neural network 216 determines that the user is present and stationary (i.e., no large body movement; however, movement occurs due to breathing and / or heartbeat), the heart rate and breathing rate can be identified as likely to be accurate. In some embodiments, the vital statistics output by the spectral summation engine 215 can be stored exclusively locally (e.g., to alleviate privacy issues); in other embodiments, the vital statistics output can be transmitted to the cloud-based server system 170 for remote storage (instead of or in addition to storing this data locally).

[0079] The neural network 216 may be initially trained using a large amount of training data of amplitude and frequency feature vectors that have been appropriately labeled by classification as mapping spectral energy and spectral sparsity to corresponding ground truth states of the monitoring area. Alternatively, the neural network 216 may be initially trained using a large amount of training data of amplitude and frequency feature vectors that have been appropriately classified as mapping included spectral energy and spectral sparsity pairs (each appropriately labeled) to corresponding ground truth states of the monitoring area. The neural network may be a non-time-dependent fully connected neural network. In some embodiments, a machine learning arrangement, classifier, or form of artificial intelligence other than a neural network may be used.

[0080] In other embodiments, instead of spectral energy values ​​and spectral sparsity values ​​being features used by the neural network, a neural network, possibly with additional front-end convolutional layers, can be trained to directly use the output of the range-pass filter 214. More specifically, an embodiment of the convolutional network can analyze the frequency and amplitude data output by the range-pass filter 214 to classify the state of the user. The convolutional neural network can be trained to utilize offline training based on a set of spectral measurements mapped to the ground truth state of the monitoring area before the end user uses the system 200B.

[0081] The sleep state determined by the neural network 216 can be stored in the sleep data storage device 118 along with the time data. When the neural network 216 indicates that the monitored user is present and stationary, the vital statistics output by the spectral summation engine 215 can be stored in the vital statistics data storage area. Other vital statistics can be discarded or possibly marked to indicate that it is unlikely to be correct. The data stored in the sleep data storage device 118 and the vital statistics data storage area can be stored locally at the device 101. In some embodiments, storage occurs only at the device 101. Such an embodiment can help alleviate concerns about health-related data transmitted and stored remotely. In some embodiments, the monitored user can choose, for example, to transmit, store and analyze sleep data and vital statistics externally by a cloud-based server system 170 via a network interface (e.g., a wireless network interface 150). Storage by a cloud-based server system 170 can have significant benefits, such as the ability of users to access such data remotely, allow access to medical providers, or participate in research. The user can retain the ability to delete or otherwise remove data from the cloud-based server system 170 at any time.

[0082] In some embodiments, the radar processing module 210 may be located in whole or in part away from the device 101. Although the radar subsystem 205 may need to be local to the monitored user, the processing of the radar processing module 210 may be moved to the cloud-based server system 170. In other embodiments, a smart home device that communicates locally with the device 101 (e.g., via a LAN or WLAN) may perform some or all of the processing of the radar processing module 210. In some embodiments, a local communication protocol, such as one involving a mesh network, may be used to transmit raw waveform data to a local device that will perform the processing. Such communication protocols may include Wi-Fi, Bluetooth, Thread, or IEEE 802.11 and 802.15.4 series of communication protocols. Similar to the processing, the storage of sleep data and vital statistics data may occur at another smart home device in the home where the cloud-based server system 170 or the device 101 is located. In other embodiments, the radar processing module 210 may be combined with the radar subsystem 205 as a single component or a system of components.

[0083] The stored sleep data and vital statistics data of the sleep data storage device 118 can be used to provide users with short-term and long-term trends related to their sleep patterns, vital statistics data, or both through the sleep data compilation engine 119. For example, each morning, graphs, statistics, and trends can be determined by the sleep data compilation engine 119 based on the data stored in the sleep data storage device 118 and output by the sleep data compilation engine 119 via the display 140 for display. A graph indicating the sleep data from the previous night can be presented, and one or more graphs indicating the breathing rate and heart rate during the previous night may be presented. Similar graphs, trends, and statistics can be output by the sleep data compilation engine 119 over significantly longer time periods, such as weeks, months, years, and even years. Other uses of sleep data and vital statistics data are also possible. For example, if certain triggers about heart rate, breathing rate, and / or sleep patterns are triggered, a medical professional may be notified. In addition or alternatively, a notification may be output to the user indicating that the collected data has potential problems or indicates a healthy person. In some cases, specific sleep problems may be identified, such as sleep apnea. The sleep data may be output via speaker 155 using synthesized speech (e.g., in response to user waking, in response to spoken user commands, or in response to user providing input via, for example, a touch screen of display 140). This sleep data may also be presented graphically and / or textually on display 140.

[0084] The system 200A may additionally include a beam steering module 230. The beam steering module 230 may include a channel weighting engine 231, which may be implemented using software, firmware, and / or hardware similar to components of the radar processing module 210. The beam steering module 230 is illustrated as being separate from the radar processing module 210 because it processes data received from the radar subsystem 205 to strengthen data received from a particular direction and weaken data received from other directions. The beam steering module 230 may be implemented using the same hardware as the radar processing module 210. For example, the beam steering module 230 may be a software process that modifies radar data received from the radar subsystem 205 before applying the mobile filter 211. The device 101 may be a surface top device that is intended to be placed in a particular location, connected to a continuous power source (e.g., a household power outlet), and interacted with via voice and / or a touch screen. Thus, the radar subsystem 205 may remain pointed at a portion of the surrounding environment for a considerable period of time (e.g., hours, days, weeks, months). Generally speaking, the beam steering module 230 can be used to map the environment (eg, room) where the device 101 is located, and steer the sensing direction of the radar subsystem 205 to an area within the field of view of the radar subsystem 205 where a user is most likely to exist.

[0085] Aiming the area within the field of view of radar subsystem 205 can help reduce the number of false negatives and false positives caused by the movement of objects other than the user. In addition, aiming can help compensate for the angle and position of device 101 relative to the user's sleeping position. (For example, device 101 may be located on a nightstand at a different height than the user's bed. Additionally or alternatively, radar subsystem 205 device 101 may not be pointed directly at the user's sleeping bed position.)

[0086] When no users are present, for example based on the low spectral energy and low spectral density of Table 2, an optimal beam steering process may be performed by channel weighting engine 231 and beam steering system 232. When no users are present, an analysis may be performed to determine which directional alignment of radar subsystem 205 provides the least clutter.

[0087] Figure 2B An embodiment of a health tracking system 200B ("system 200B") is illustrated that uses radar to detect human interactions and performs health monitoring with integrated beam aiming. Beam aiming performed using beam steering module 230 can focus on radar reflections from areas where a user may be present and ignore or at least reduce the use of radar reflections from objects that cause interference (e.g., nearby walls or large objects). Notably, beam steering module 230 can perform processing on a virtual continuous mode radar data stream; therefore, the burst mode radar data stream analyzed by human interaction engine 245 can be unaffected by the aiming performed by beam steering module 230.

[0088] The virtual continuous chirp generator 240 and the human interaction engine 245 may be as described with respect to Figure 2A Detailed description works. However, the virtual continuous mode radar stream output by the virtual continuous chirp generator 240 is input to the beam steering module 230, rather than directly to the radar processing module 210. The radar subsystem 205 can output separate data for each antenna of the radar subsystem 205. Therefore, there can be a separate burst mode radar data stream for each antenna (or within a single burst mode radar data stream, a specific antenna that receives reflected radio waves can be indicated). The virtual continuous chirp generator 240 can keep the data separate for each antenna. Therefore, a virtual continuous mode radar data stream can be created for each antenna, or a single virtual continuous mode radar data stream can be created that indicates which parts of the virtual continuous mode radar data stream correspond to which antenna of the radar subsystem 205.

[0089] Radar subsystem 205 may include multiple antennas to receive reflected radar radio waves. In some embodiments, there may be three antennas. These antennas may be aligned in an "L" pattern, so that two antennas are horizontally orthogonal and two antennas are vertically orthogonal, with one antenna being used for both horizontal and vertical arrangements. By analyzing the phase difference in the received radar signal, weighting may be applied to aim the received radar beam vertically and / or horizontally. In other embodiments, the antennas may be aligned in different patterns.

[0090] Vertical aiming can be performed to compensate for vertical tilt of the device into which the system 200B is incorporated. Figure 3A Discussed, the face of the contactless fitness tracking device 300 may be tilted relative to the position in which the user would typically sleep.

[0091] Horizontal aiming may be performed to compensate for the transmitted radar pointing toward an object causing interference. For example, if a user's headboard is against a wall, the headboard and / or wall may occupy a significant portion of the field of view of radar subsystem 120. Radar reflections from the headboard and / or wall may not be useful in determining data about the user; therefore, it may be beneficial to weaken reflections from the wall and / or headboard and strengthen reflections obtained away from the wall and / or headboard. Therefore, through the weighting applied to the received radar signal, the receive beam may be steered horizontally away from the wall and headboard.

[0092] In system 200B, there is a beam steering module 230 to perform processing on the raw chirp waterfall of the virtual continuous mode radar data stream received from radar subsystem 205 via virtual continuous chirp generator 240. Therefore, beam steering module 230 can be used as a pre-processing module before the analysis of radar processing module 210, and can be used to enhance the area where one or more users are expected to exist. Beam steering module 230 can be implemented using hardware, software, or firmware; therefore, beam steering module 230 can be implemented using the same one or more processors as radar processing module 210.

[0093] The beam steering module 230 may include a channel weighting engine 231 and a beam steering system 232. The channel weighting engine 231 may be used to perform a training process to determine a series of weights to be applied to antenna signals received from each antenna before the received radar signals are mixed together. The channel weighting engine 231 may perform a training process when the monitoring area is determined to be empty. During this time, the strength of the signals received from large static objects (e.g., walls, headboards) may be analyzed, and weights may be set to steer the beam horizontally (and possibly vertically) away from these objects. Thus, for a specific range of distances from the device (e.g., up to one meter), the receive radar beam may be steered by the channel weighting engine 231 to minimize the amount of reflections in a static environment. This training may also be performed when a user is present. That is, the receive beam of the radar subsystem 205 may be steered to a location where motion is detected, or specifically, where there are vital signs of the user.

[0094] Beam steering system 232 may use the weights determined by channel weighting engine 231 to apply weights to the received reflected radar signals of each antenna individually. The received signals from each antenna may be weighted and then mixed together for processing by radar processing module 210.

[0095] Figure 2C An embodiment of a chirp timing diagram 200C of a frequency modulated continuous wave (FMCW) radar radio wave output by a radar subsystem is illustrated. The chirp timing diagram 200C is not drawn to scale. The radar subsystem 205 can generally output radar with the pattern of the chirp timing diagram 200C. Chirp 250 represents a continuous pulse of a radio wave whose frequency is swept upward from a low frequency to a high frequency. In other embodiments, a single chirp can be continuously swept downward from a high frequency to a low frequency, from a low frequency to a high frequency, and then returned to a low frequency, or from a high frequency to a low frequency, and then returned to a high frequency. In some embodiments, the low frequency is 58 GHz and the high frequency is 63.5 GHz. (For these frequencies, radio waves can be referred to as millimeter waves.) In some embodiments, the frequency is between 57 and 64 GHz. The low frequency and high frequency can vary according to the embodiment. For example, the low frequency and high frequency can be between 45 GHz and 80 GHz. The frequency selection can be selected at least in part to comply with government regulations. In some embodiments, each chirp comprises a linear sweep from low frequency to high frequency (or vice versa).In other embodiments, an exponential or some other pattern may be used to sweep the frequency from low to high or from high to low.

[0096] A chirp 250, which may represent all the chirps in the chirp timing diagram 200C, may have a chirp duration 252 of 128 μs. In other embodiments, the chirp duration 252 may be longer or shorter, for example, between 50 μs and 1 ms. In some embodiments, a period of time may elapse before a subsequent chirp is transmitted. The inter-chirp pause 256 may be 205.33 μs. In other embodiments, the inter-chirp pause 256 may be longer or shorter, for example, between 10 μs and 1 ms. In the illustrated embodiment, the chirp period 254 including the chirp 250 and the inter-chirp pause 256 may be 333.33 μs. This duration varies based on the selected chirp duration 252 and inter-chirp pause 256.

[0097] The multiple chirps of the output separated by the pause between chirps can be referred to as frames 258 or frames 258. Frame 258 can include twenty chirps. In other embodiments, the number of chirps in frame 258 can be greater or less, for example, between 1 and 100. The number of chirps present in frame 258 can be determined based on the average power amount expected to be output in a given time period. The FCC or other regulatory agency can set the maximum power amount allowed to be radiated into the environment. For example, there may be a duty cycle requirement that limits the duty cycle to less than 10% in any 33ms time period. In a specific example where there are twenty chirps per frame, each chirp can have a duration of 128us, and the duration of each frame is 33.33ms. The corresponding duty cycle is (20 frames)*(.128ms) / (33.33ms), which is about 7.8%. By limiting the number of chirps in frame 258 before the pause between frames, the average output power can be limited. In some embodiments, the peak EIRP (effective isotropic radiated power) may be 13 dBm (20 mW) or less, such as 12.86 dBm (19.05 mW). In other embodiments, the peak EIRP is 15 dBm or less and the duty cycle is 15% or less. In some embodiments, the peak EIRP is 20 dBm or less. That is, at any given time, the average power radiated by the radar subsystem over a period of time may be limited to never exceed such a value. In addition, the total power radiated over a period of time may be limited. In some embodiments, a duty cycle may not be required.

[0098] Frames may be transmitted at a frequency of 30 Hz (33.33 ms), as shown by time period 260. In other embodiments, the frequency may be higher or lower. The frame frequency may depend on the number of chirps within a frame and the duration of the interframe pause 262. For example, the frequency may be between 1 Hz and 50 Hz. In some embodiments, chirps may be transmitted continuously so that the radar subsystem outputs a continuous stream of chirps interspersed with pauses between chirps. A trade-off may be made to save the average power consumed by the device due to transmitting chirps and processing received chirp reflections. Interframe pause 262 represents a period of time when no chirps are output. In some embodiments, interframe pause 262 is significantly longer than the duration of frame 258. For example, the duration of frame 258 may be 6.66 ms (wherein the chirp period 254 is 333.33 μs, 20 chirps per frame). If 33.33 ms occurs between frames, then interframe pause 262 may be 26.66 ms. In other embodiments, the duration of the inter-frame pause 262 may be larger or smaller, such as between 15 ms and 40 ms.

[0099] exist Figure 2C In the illustrated embodiment of , a single frame 258 and the beginning of subsequent frames are illustrated. It should be understood that each subsequent frame can be constructed similarly to frame 258. In addition, the transmission mode of the radar subsystem can be fixed. That is, chirps can be transmitted according to the chirp timing diagram 200C regardless of whether a user is present, the time of day, or other factors. Therefore, in some embodiments, the radar subsystem always operates in a single transmission mode, regardless of the environmental state or the activity attempted to be monitored. When the device 101 is powered on, a continuous string of frames similar to frame 258 can be transmitted.

[0100] Figure 3A An embodiment of a contactless sleep tracking device 300 ("device 300") is illustrated. Device 300 may have a front surface that includes a front transparent screen 340 so that a display is visible. This display may be a touch screen. An optically opaque area, referred to as a bezel 330, may surround front transparent screen 340 through which radar subsystem 205 may have a field of view of the environment in front of device 300.

[0101] For the purposes of the following description, the terms vertical and horizontal generally describe directions relative to the bedroom, where vertical refers to the direction perpendicular to the floor and horizontal refers to the direction parallel to the floor. The radar subsystem of the BGT60 radar chip is generally planar and is generally mounted parallel to the bezel 330 for the sake of spatial compactness of the entire device, and because the antenna within the radar chip is located in the plane of the chip, the receive beam of the radar subsystem 120 can be directed in a direction 350 that is generally perpendicular to the bezel 330 without beam aiming. Due to the deviation tilt of the bezel 330 away from the pure vertical direction (in some embodiments, the deviation tilt is provided to be approximately 25 degrees to facilitate the user's simple interaction with the touch screen function of the transparent screen 340), the direction 350 can be directed upward from the horizontal at a deviation angle 351. Assuming that the device 300 is generally mounted on a bedside platform (e.g., a nightstand) at approximately the same height as the top of the mattress where the user sleeps, it may be beneficial for the receive beam of the radar subsystem 120 to be aimed at a horizontal direction 352 or a substantially horizontal direction (e.g., between -5° and 5° from the horizontal direction). Therefore, vertical beam aiming can be used to compensate for the deviation angle 351 of the portion of the device 300 where the radar subsystem 120 is located.

[0102] Figure 3B An exploded view of an embodiment of a contactless sleep tracking device 300 is illustrated. Device 300 may include: display assembly 301; display housing 302; main circuit board 303; neck assembly 304; speaker assembly 305; bottom plate 306; mesh network communication interface 307; top daughter board 308; button assembly 309; radar assembly 310; microphone assembly 311; rocker switch bracket 312; rocker switch board 313; rocker switch button 314; Wi-Fi assembly 315; power board 316; and power bracket assembly 317. Device 300 may represent how an embodiment of device 101 may be implemented.

[0103] Display assembly 301, display housing 302, neck assembly 304 and base plate 306 can form a housing together, and all the remaining components of the housing accommodating device 300. Display assembly 301 can include an electronic display that presents information to the user, and the electronic display can be a touch screen. Therefore, display assembly 301 can include a display screen, and the display screen can include a metal plate of the display, and the metal plate can be used as a ground plane. Display assembly 301 can include a transparent portion away from the metal plate, and the transparent portion allows various sensors to have a field of view in the general direction that display assembly 301 faces. Display assembly 301 can include an outer surface made of glass or transparent plastic, and the outer surface is used as a part of the housing of device 300.

[0104] The display housing 302 may be a plastic or other rigid or semi-rigid material used as a housing for the display assembly 301. Various components such as the main circuit board 303; the mesh network communication interface 307; the top daughter board 308; the button assembly 309; the radar assembly 310; and the microphone assembly 311 may be mounted on the display housing 302. The mesh network communication interface 307; the top daughter board 308; the radar assembly 310; and the microphone assembly 311 may be connected to the main circuit board 303 using a flat wire assembly. The display housing may be attached to the display assembly 301 using an adhesive.

[0105] The mesh network communication interface 307 may include one or more antennas and may enable communication with a mesh network (e.g., a Thread-based mesh network). The Wi-Fi assembly 315 may be located a distance from the mesh network communication interface 307 to reduce the possibility of interference. The Wi-Fi assembly 315 may enable communication with a Wi-Fi-based network.

[0106] The radar assembly 310, which may include the radar subsystem 120 or the radar subsystem 205, may be positioned so that its RF transmitter and RF receiver are away from the metal plate of the display assembly 301 and are positioned at a significant distance from the mesh network communication interface 307 and the Wi-Fi assembly 315. These three components may be arranged in an approximately triangular shape to increase the distance between the components and reduce interference. For example, in the device 300, a distance of at least 74 mm between the Wi-Fi assembly 315 and the radar assembly 310 may be maintained. A distance of at least 98 mm between the mesh network communication interface 307 and the radar assembly 310 may be maintained. In addition, the distance between the radar assembly 310 and the speaker 318 may be expected to minimize the impact of vibrations that may be generated by the speaker 318 on the radar assembly 310. For example, for the device 300, a distance of at least 79 mm between the radar assembly 310 and the speaker 318 may be maintained. In addition, the distance between the microphone and the radar assembly 310 may be expected to minimize any possible interference of the microphone with the received radar signal. The top sub-board 308 may include a plurality of microphones. For example, at least 12 mm may be maintained between the nearest microphone of the top sub-board 308 and the radar assembly 310.

[0107] Other components may also be present. A third microphone assembly, i.e., microphone assembly 311, may be present in the rear direction. Microphone assembly 311 may work in conjunction with the microphone of top sub-board 308 to isolate verbal commands from background noise. Power board 316 may convert power received from an AC power source into DC to power components of device 300. Power board 316 may be installed in device 300 using power support assembly 317. Rocker switch support 312, rocker switch board 313, and rocker switch button 314 may be used together to receive user input, such as up / down input. For example, this input may be used to adjust the volume of the sound output by speaker 318. As another user input, button assembly 309 may include a toggle button that the user may actuate. This user input may be used to activate and deactivate all microphones, such as when the user wishes privacy and / or does not wish device 300 to make voice commands.

[0108] Figure 4 An embodiment of a state machine 400 for determining when a person is sleeping is illustrated. Based on the data output by the radar processing module 112, the sleep state detection engine 114 can use the state machine 400 to determine whether a person is sleeping. It should be understood that in some embodiments, the sleep state detection engine 114 is incorporated as part of the functionality of the radar processing module 112 and does not exist as a separate module. The state machine 400 may include five possible sleep states: entering bed state 401; not in bed state 402; moving in bed state 403; no movement in bed state 405; and leaving bed state 404.

[0109] If there is no motion indicating waveform data, this may indicate that the user is not in bed. Due to the vital signs of a user in bed, it may be expected that the user is always moving at least a small amount. Therefore, if zero movement is observed, it may be determined that the user is in state 401. After determining state 401, the next possible state that may be determined is state 402. In state 402, the monitored user enters the bed. Significant user movement may be sensed, for example, according to Table 2. This may indicate that the user enters the bed, and the state may be transitioned from state 401 to state 402.

[0110] From state 402, motion in bed may continue to be detected, for example, due to the user turning over, getting into position, moving pillows, sheets and / or blankets, reading a book, etc. State 402 may transition to state 403 while continuing to detect such motion. Alternatively, if motion is detected, then zero motion is detected, which may indicate that state 405 has been entered by the monitored user leaving the bed. If this occurs, state 402 may transition to state 405 and then return to state 401. In general, state 404 may be interpreted as the user sleeping, and state 403 may be interpreted as the user awake. In some embodiments, more than a threshold amount of time (or at least partially based on some other form of determination of the time threshold standard form) is required in state 404 to classify the user as sleeping, and more than a threshold amount of time (or at least partially based on some other form of determination of the time threshold standard form) is required in state 403 to classify the user as awake. For example, if the user was previously determined to be sleeping, less than 5 seconds of movement in bed may be interpreted as the user moving while still asleep. Thus, if a user transitions from state 404 to state 403, experiences a certain number of movement events, and then returns to state 404 in less than a period of time, the user may be identified as experiencing a "sleep arousal" in which the user's sleep was disturbed but the user was not awakened. Such a sleep arousal may be tracked along with a period of time in which the user is determined to be fully awake, or may be kept separate data from a period of time in which the user is determined to be fully awake.

[0111] From state 403, it can be determined that the monitored user leaves the bed in state 405 and can become immobile in state 404. In state 404, "immobile" means that the monitored user does not make large movements, but the user continues to make small movements due to vital signs. In some embodiments, only when the state of the monitored user is determined to be state 404, the vital signs are considered accurate and / or stored, recorded or otherwise used to measure the vital signs of the user. The data collected during state 403 and state 404 can be used to determine the general sleep pattern of the monitored user (e.g., how much time tossing and turning, how much high-quality sleep, when deep sleep occurs, when REM sleep occurs, etc.). After the user enters state 404 for a predefined time period, it can be assumed that the user is in a sleeping state until the user exits state 404. When the user initially transitions to state 404, the user may be required to stay in state 404 for a certain amount of time (e.g., two to five minutes) to be considered asleep. If the user reaches at least a defined time period in state 403, the user can be identified as awake. However, if the user enters state 403 from state 404 for less than a defined period of time and returns to state 404, the user may be identified as moving only while sleeping and has been continuously in the sleeping state.

[0112] Figure 5 Schematic diagram 500 of received reflected radar chirps transmitted in burst mode is illustrated. Schematic diagram 500 represents reflected, received radar chirps when radar subsystem 205 (or more generally, radar subsystem 120) operates in burst mode. In burst mode, radar subsystem 205 receives bursts 510 (which may also be referred to as frames). A single burst includes a certain number of chirps. For each chirp transmitted by radar subsystem 205, a reflected chirp is received. As illustrated, burst 510-1 may include five reflected chirps: chirp 511-1, chirp 511-2, chirp 511-3, chirp 511-4, and chirp 511-5. In other embodiments, a burst includes a greater or lesser number of chirps. For example, in some embodiments, a burst includes 3, 4, 10, 15, 20, or some other number of chirps. A burst may include any number of chirps between 2 and 100 chirps.

[0113] In a burst (e.g., burst 510-1), a chirp may be transmitted approximately every 0.333 ms, and thus reflected and received. Thus, time 512 may be 0.333 ms. In other embodiments, time 512 may be larger or smaller. For example, time 512 may be between 0.01 ms and 5 ms. The burst period (i.e., time 514), which represents the amount of time between the start of burst 510-1 and the start of burst 510-2, may be 33.3 ms. In other embodiments, time 514 may be larger or smaller, such as between 15 ms and 500 ms. Between each burst, such as between burst 510-1 and burst 510-2, a relatively larger amount of time may be passed than between adjacent chirps within the burst. For example, the amount of time between burst 510-1 and burst 510-2 (time 513-1) (that is, the amount of time passed between chirp 511-5 and chirp 511-6) may be 26.7 ms. In other embodiments, time 513-1 is larger or smaller, such as any value between 1 ms and 200 ms. Regardless of the embodiment, time 513-1 is greater than time 512; that is, the time between adjacent chirps within a burst or frame is less than the time between bursts.

[0114] The timing between bursts 513 can be fixed; thus, time 513-1, time 513-2, and time 513-4 can be the same. Similarly, the time between chirps within a burst can be constant. The radar subsystem can operate continuously in this mode without changing to life detection mode.

[0115] Figure 6Schematic diagram 600 illustrates radar chirps received as part of a burst mode radar data stream and converted into a virtual continuous mode radar data stream. When the radar subsystem receives the reflected radar chirp, the reflected radar chirp may be mixed with the frequency of the current transmission to produce a mixed received radar chirp. The conversion of schematic diagram 600 may be performed by virtual continuous chirp generator 240 based on the original chirp waterfall output by the radar subsystem. Figure 5 In detail, the radar sensor may be operated in burst mode and may therefore receive reflected radar chirps in a burst pattern. The waveform data or raw chirp waterfall output by a radar subsystem operating in burst mode may be referred to as a burst mode radar data stream. This burst mode radar data stream is Figure 6 This is indicated by the graphical representation of burst 510 and chirp 511.

[0116] Transformations 610 are performed to transform each reflection burst in the original chirp waterfall into a single representative virtual continuous radar chirp 620. Transformation 610-1 (indicated by an arrow) transforms reflection burst 510-1 into virtual continuous radar chirp 620-1; transformation 610-2 transforms burst 510-2 into virtual continuous chirp 620-2; transformation 610-1 transforms burst 510-3 into virtual continuous chirp 620-3; transformation 610-4 transforms burst 510-4 into virtual continuous chirp 620-4, and so on.

[0117] To perform conversion 610-1, each of chirps 511-1 to 511-5 is sampled multiple times from the burst mode radar data stream. This sampling may occur at a certain number of chirp sampling points, such as chirp sampling point 605-1 to chirp sampling point 605-7. (For Figure 6 For simplicity, the arrows representing sampling points 605-3, 605-4, 605-5, and 605-6 are not labeled. ) As an example, if there are five chirps in burst 510-1 and seven chirp sampling points are used, a total of 35 samples (7 samples per chirp) can be sampled from burst 510-1. The number of chirp sampling points used may vary depending on the embodiment and may be greater or less than Figure 6 The seven chirp sampling points are shown in .

[0118] As part of the conversion 610-1, an averaging or combining process may be performed.In general, each of the chirps (chirps 511-1 through 511-5) of the burst 510-1 may be averaged together. To achieve this averaging: the samples of each chirp of the burst 510-1 obtained at the chirp sampling point 605-1 can be averaged together; the samples of each chirp of the burst 510-1 obtained at the chirp sampling point 605-2 can be averaged together; the samples of each chirp of the burst 510-1 obtained at the chirp sampling point 605-3 can be averaged together; the samples of each chirp of the burst 510-1 obtained at the chirp sampling point 605-4 can be averaged together; the samples of each chirp of the burst 510-1 obtained at the chirp sampling point 605-5 can be averaged together; the samples of each chirp of the burst 510-1 obtained at the chirp sampling point 605-6 can be averaged together; and the samples of each chirp of the burst 510-1 obtained at the chirp sampling point 605-7 can be averaged together. More generally, each sample of a chirp is averaged with the corresponding sample of each other chirp of the burst. For example, the third sample of a chirp is averaged with the third sample of each other chirp of the burst.

[0119] For the illustrated embodiment, the seven average samples can then be used to construct a virtual continuous radar chirp 620-1. For example, the first average value from the chirp sampling point 605-1 is used as the first sample 621-1 of the virtual continuous radar chirp 620-1; the second average value from the chirp sampling point 605-2 is used as the second sample 621-2 of the virtual continuous radar chirp 620-1, etc. More generally, the average samples of the burst 510-1 are concatenated to create the virtual continuous radar chirp 620-1.

[0120] In other words, a sample-by-sample average is performed for each burst 510. Mathematically, this can be represented using the notation c i , j represents each chirp in the burst, where i is the index of the chirp sample and j is the index of the chirp within the burst. Therefore, c 2,3 will be the second sample of the third chirp within a given burst. Equation 4 can be used to find the average value used to create a virtual continuous radar chirp:

[0121] V k =average(c k,1 , c k，2 ,c k，3 , ...c k，N ) Equation 4

[0122] In Equation 4, N represents the number of chirps within a burst and k represents the number of samples. kDenotes the average value corresponding to the kth sample. Equation 4 is repeated k times to obtain all V used to construct the virtual continuous radar chirp k value.

[0123] The virtual continuous radar chirps 620 created may be spaced apart in time at equal intervals. For example, time 630-1 may be 33.3 ms, which may be consistent with the Figure 5 Detailed burst period matching. Time 630-2, 630-3, etc. can match time 630-1 so that the virtual continuous radar chirp is output periodically. More generally, since a single virtual continuous radar chirp is generated for each burst in burst 510, time 630 can be the same as the period of burst 510. By equally spacing each virtual continuous radar chirp in time, the virtual continuous radar chirp can be more effective than a burst mode radar data stream for health monitoring. The virtual continuous radar chirp 620 can be output as a virtual continuous mode radar data stream to other components for processing. If such other components are implemented as software, the same processor or processing system used to perform the functions of the virtual continuous chirp generator 240 can be used to perform the processing.

[0124] By averaging the chirps 511 of the burst 510-1 together, the virtual continuous radar chirp 620-1 can have a higher resolution than any individual chirp 511. The reflected radar received by the radar subsystem 120 (or 205) can be converted into digital data using an analog-to-digital converter (ADC). The ADC can have a limited resolution, such as 8 bits. Therefore, all values ​​output by the ADC can be limited to a range of 0 to 256, which corresponds to eight binary bits. Dithering is a concept in signal processing that can cause an increase in resolution in exchange for signal fidelity. By averaging or otherwise combining multiple reflected noise chirps, dithering can be used to obtain a higher resolution than the output of the ADC used to convert the analog radar signal into the digital waveform data present in the digital burst mode radar data stream. The intuition is that as the ADC noise increases, the samples in a series of noise chirps will be slightly jittered around the true value. By aggregating multiple chirps, the measurement value can be captured in a statistically accurate manner. In the absence of noise, even if there are many chirps in a burst, there can be no statistical advantage because each sample on the chirp will have the same value which may be affected by the ADC quantization. The resulting higher resolution signal with dithering has higher fidelity in detecting weak features such as human vital signs (e.g., breathing, heartbeat).

[0125] You can use About Figures 1 to 6 The detailed systems and arrangements perform various methods. Figure 7An embodiment of a method 700 for monitoring non-contact human interaction and monitoring health using a single radar modulation pattern is illustrated. Figures 1 to 3B The detailed apparatus and system implement method 700 .

[0126] At block 705, the radar subsystem or radar sensor transmits radio waves. The transmitted radio waves may be continuous wave radar, such as FMCW, as described in relation to Figure 2C The radar sensor can be operated in burst mode so that Figure 2C The radar chirp is transmitted (and thus received) in a pattern. The transmitted radio waves may be transmitted by RF transmitter 206 of radar subsystem 205, which may transmit the radio waves using one or more antennas. At block 710, reflections of the radio waves are received, for example, by RF receiver 207 of radar subsystem 205. The reflections received at block 810 may be reflected from moving objects (e.g., a person with a heartbeat and breathing) and stationary objects. The pattern of received, reflected radio waves may closely reflect the pattern of transmitted radio waves; thus, schematic diagram 500 may closely represent the pattern of transmitting chirp bursts and also receiving reflected chirp bursts.

[0127] At block 715, a raw chirp waterfall, which may be referred to as a burst mode radar data stream, is created based on the received reflected radio waves. The radar subsystem or sensor may convert the received reflected radar signal into the digital domain using an on-board ADC. In some embodiments, data received from different antennas of the radar subsystem is maintained separately. The digital burst mode radar data stream may be output by the radar subsystem to a processing system, such as a processing system that performs the functions of the virtual continuous chirp generator 240 and / or the human interaction engine 245.

[0128] At block 720, the burst mode radar data stream may be analyzed to determine if there is any contactless human interaction, such as a gesture or human presence. Regardless of the state of the health monitoring performed in blocks 725 and 730, the burst mode radar data stream may be directly analyzed for such human interaction. If contactless human interaction is detected at block 720, one or more actions may be taken in response to the detection at block 722. For example, a command may be executed or output to another device based on a detected gesture. If human presence is detected, an action may be taken, such as illuminating the electronic display 140 (and presenting information) or outputting audio, such as synthesized speech.

[0129] Independent of blocks 720 and 722, method 700 may involve block 725 being performed. Thus, block 725 may be performed while blocks 720 and / or 722 are being performed. At block 725, the same burst mode radar data stream that was, is, or will be analyzed for contactless human interaction at block 720 may be processed to create a virtual continuous chirp radar data stream. Figure 6 In detail, a conversion that may involve an averaging process may be performed to convert a burst of radar chirps into a virtual continuous radar chirp. Multiple virtual continuous chirps are assembled into a virtual continuous radar data stream, and the virtual continuous radar data stream is output (as digital data) to one or more other components that may be implemented using the same processing system that created the virtual continuous radar data stream or a separate processing system. By combining multiple chirps within a burst into a single virtual continuous pattern chirp, the virtual continuous pattern chirp may benefit from jitter and therefore have a higher resolution than the individual chirps of the burst present in the burst mode radar data stream.

[0130] At block 730, health monitoring, such as sleep tracking, vital sign monitoring, cough monitoring, or sleep disorder attribution, may be performed, as detailed with respect to the functionality of radar processing module 210. Health monitoring may function or may function more efficiently based on a continuous radar data stream rather than a burst mode radar data stream. In other embodiments, some other form of monitoring or tracking may be performed based on a continuous radar data stream.

[0131] The methods, systems and devices discussed above are examples. Various configurations may omit, replace or add various processes or components as appropriate. For example, in alternative configurations, the method may be performed in an order different from that described, and / or various stages may be added, omitted and / or combined. Moreover, features described with respect to certain configurations may be combined in various other configurations. Different aspects and elements of the configurations may be combined in a similar manner. In addition, technology is developing, and therefore many of the elements are examples and do not limit the scope of the present disclosure or claims.

[0132] Specific details are given in the description to provide a comprehensive understanding of example configurations (including implementation schemes). However, configurations can be practiced without these specific details. For example, in order to avoid confusing configurations, well-known circuits, processes, algorithms, structures and techniques have been shown without unnecessary details. This specification only provides example configurations without limiting the scope, applicability or configuration of the claims. On the contrary, the foregoing description of the configuration will provide an enabling description for implementing the described technology to those skilled in the art. Various changes can be made to the functions and arrangements of elements without departing from the spirit or scope of the present disclosure.

[0133] In addition, the configuration can be described as a process depicted as a flow chart or block diagram. Although each operation is described as a sequential process, many operations can be performed in parallel or simultaneously. In addition, the order of operations can be rearranged. The process can have additional steps not included in the diagram. In addition, the example of the method can be implemented by hardware, software, firmware, middleware, microcode, hardware description language or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segment for performing the necessary tasks can be stored in a non-transitory computer readable medium (e.g., a storage medium). The processor can perform the described tasks.

[0134] Several example configurations have been described, and various modifications, alternative configurations, and equivalents may be used without departing from the spirit of the present disclosure. For example, the above elements may be components of a larger system, in which other rules may take precedence over or otherwise modify the application of the present invention. Moreover, multiple steps may be performed before, during, or after consideration of the above elements.

Claims

1. A non-contact health monitoring device, comprising: shell; a radar sensor housed by the housing, the radar sensor configured to operate in a burst mode in which the radar sensor transmits a plurality of bursts of radar chirps, wherein: A first amount of time that elapses between adjacent radar chirps of the plurality of bursts of radar chirps is less than a second amount of time that elapses between adjacent bursts of the plurality of bursts of radar chirps; and the radar sensor outputting a burst mode radar data stream based on received reflections of the radar chirp of the plurality of bursts of radar chirps; a processing system housed by the housing, the processing system comprising one or more processors, the processors in communication with the radar sensor, the processing system being configured to: receiving the burst-mode radar data stream from the radar sensor; analyzing the burst mode radar data stream to identify contactless human interactions; converting the burst mode radar data stream into a virtual continuous mode radar data stream; and The virtual continuous mode radar data stream is used to monitor the health of the user.

2. The non-contact health monitoring device according to claim 1, wherein: The virtual continuous mode radar data stream includes a plurality of virtual reflections of radar chirps equally spaced in time.

3. The non-contact health monitoring device according to claim 2, wherein: The processing system is configured to convert the burst mode radar data stream into the virtual continuous mode radar data stream includes the processing system being configured to: A virtual reflection of a radar chirp is created based on a plurality of radar chirps of a burst in the plurality of bursts, wherein the virtual reflection of the radar chirp is a portion of the plurality of virtual reflections of the radar chirp that are equally spaced in time.

4. The non-contact health monitoring device according to claim 3, wherein: The processing system being configured to create the virtual reflection of the radar chirp based on the plurality of radar chirps of the burst in the plurality of bursts comprises the processing system being configured to perform an averaging process, the process comprising: sampling a plurality of samples of each radar chirp in the plurality of chirps of the burst; averaging each sample of the plurality of samples with corresponding samples from other chirps of the plurality of chirps of the burst to create a plurality of averaged samples; and The average samples are assembled to create the virtual reflection of the radar chirp.

5. The non-contact health monitoring device according to claim 4, wherein: The contactless human interaction is detected while performing health monitoring.

6. The non-contact health monitoring device according to claim 1, wherein: The radar sensor outputs a frequency modulated continuous wave (FMCW) radar with a frequency between 57-64 GHz and a peak EIRP less than 20 dBm.

7. The non-contact health monitoring device according to claim 1, wherein: The contactless human interaction is a hand gesture.

8. The non-contact health monitoring device according to claim 1, wherein: The contactless human interaction is presence detection.

9. The non-contact health monitoring device according to claim 1, wherein: The health monitoring includes sleep monitoring of the user.

10. The non-contact health monitoring device according to claim 1, wherein: Due to jitter, the virtual continuous mode radar data stream has a higher resolution than the burst mode radar data stream.

11. The non-contact health monitoring device according to claim 1, further comprising: a wireless network interface housed by the housing; an electronic display housed by the housing; a microphone housed by the housing; as well as A speaker is housed by the housing, wherein the wireless network interface, the electronic display, the microphone, and the speaker are in communication with the processing system.

12. The non-contact health monitoring device according to claim 11, wherein: The processing system is further configured to: receiving a verbal command via the microphone; transmitting an indication of the spoken command to a cloud-based server system via the wireless network interface; receiving a response from the cloud-based server system via the wireless network interface; as well as Based on the response from the cloud-based server system, information obtained from the performed health monitoring is output via the electronic display, the speaker, or both.

13. The non-contact health monitoring device according to claim 1, wherein: The processing system is further configured to output a report based on the health monitoring.

14. A method for monitoring non-contact human interaction and monitoring health using a single radar modulation pattern, the method comprising A radar chirp is emitted by a radar sensor operating in a burst mode such that the radar sensor emits a plurality of bursts of radar chirps, wherein: a first amount of time that elapses between subsequent radar chirps of a burst in the plurality of bursts of radar chirps is less than a second amount of time that elapses between subsequent bursts of the plurality of bursts of radar chirps; as well as The radar sensor outputs a burst mode radar data stream, the burst mode radar data stream being based on the plurality of bursts of the radar chirps or reflections of the radar chirps; receiving, by a processing system, the burst mode radar data stream from the radar sensor; analyzing, by the processing system, the burst mode radar data stream for contactless human interaction; converting, by the processing system, the burst mode radar data stream into a virtual continuous mode radar data stream; as well as Health monitoring of a user is performed by the processing system using the virtual continuous mode radar data stream.

15. The method for monitoring non-contact human interaction and monitoring health using a single radar modulation pattern according to claim 14, wherein: The virtual continuous mode radar data stream consists of multiple virtual reflections of the radar chirp that are equally spaced in time.

16. The method for monitoring non-contact human interaction and monitoring health using a single radar modulation pattern according to claim 15, wherein: The processing system is configured to convert the burst mode radar data stream into the virtual continuous mode radar data stream includes the processing system being configured to: A virtual reflection of a radar chirp is created based on the plurality of radar chirps of a burst in the plurality of bursts, wherein the virtual reflection of the radar chirp is a portion of the plurality of virtual reflections of the radar chirp that are equally spaced in time.

17. The method for monitoring non-contact human interaction and monitoring health using a single radar modulation pattern according to claim 16, wherein: Creating the virtual reflection of the radar chirp based on the plurality of radar chirps of the burst in the plurality of bursts comprises: sampling a plurality of samples of each radar chirp in the plurality of chirps of the burst; averaging each sample of the plurality of samples with corresponding samples from other chirps of the plurality of chirps of the burst to create a plurality of averaged samples; and The average samples are assembled to create the virtual reflection of the radar chirp.

18. The method for monitoring non-contact human interaction and monitoring health using a single radar modulation pattern according to claim 17, wherein: The contactless human interaction is a hand gesture.

19. The method for monitoring non-contact human interaction and monitoring health using a single radar modulation pattern according to claim 17, wherein: The contactless human interaction is presence detection.

20. The method for monitoring non-contact human interaction and monitoring health using a single radar modulation pattern according to claim 17, wherein: The health monitoring includes sleep monitoring of the user.

Images