Method and system for estimating vital sign of vehicle occupant using ultra-wideband

Abstract

To provide a method and system for estimating vital signs of a vehicle occupant using ultra-wideband communication.SOLUTION: A UWB signal is transmitted from one UWB system node to another UWB system node. Channel impulse responses (CIRs) associated with the UWB signals are determined. On the basis of the CIRs, in some embodiments, velocities associated with the UWB signals are specified, and activities of vehicle occupants are classified on the basis of the CIRs along with vital signs such as breathing rates. In other embodiments, on the basis of the CIRs, phase unwrapping of phase features associated with the CIRs is performed, and principle component analysis (PCA) transforms the CIRs, allowing estimated vital signs such as breathing rates to be identified.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to methods and systems for estimating vital signs of vehicle occupants using ultra-wideband (UWB) communications. [Background technology]

[0002] In automotive applications, keyless entry systems provide remote access that allows a user the ability to remotely lock or unlock vehicle doors. For example, a key fob includes a button to lock / unlock the vehicle doors. More recently, automobile manufacturers have introduced passive keyless systems. Passive keyless systems may not require the pressing of a button to lock / unlock the vehicle doors. Passive keyless systems may also not require a physical key to start the vehicle. Instead, passive keyless systems may allow such actions to be performed when the key fob is located near or inside the vehicle. Such systems may rely on radio frequency (RF) signals, but may be limited outside of these applications. Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment, a method for estimating vital signs of a vehicle occupant using ultra-wideband communications includes transmitting an ultra-wideband (UWB) signal from a first UWB system node in the vehicle, receiving the UWB signal at a second UWB system node in the vehicle, calculating a channel impulse response (CIR) associated with the UWB signal received at the second UWB system node, calculating a velocity associated with the UWB signal based on the CIR, classifying an activity of the vehicle occupant as breathing based on the calculated velocity falling between a lower threshold and an upper threshold, and estimating a respiratory rate of the vehicle occupant based on the calculated velocity.

[0004] According to another embodiment, a method for estimating vital signs of a vehicle occupant using UWB communications includes transmitting a UWB signal from a first UWB system node in the vehicle, receiving the UWB signal at a second UWB system node in the vehicle, calculating a channel impulse response (CIR) associated with the received UWB signal at the second UWB system node, phase unwrapping phase features associated with the CIR, truncating the CIR to select a chunk of CIR taps, performing a principal component analysis (PCA) transform on the CIR taps to project the CIR data onto a hyperplane, applying a bandpass filter to the projected CIR data to remove high frequency components associated with the projected CIR data, where the applying produces filtered projected CIR data, and estimating a respiratory rate of the vehicle occupant based on the filtered projected CIR data.

[0005] According to another embodiment, a system is configured to estimate vital signs of a vehicle occupant using UWB communications, the system including a transmitting node operable to transmit a UWB signal, a receiving node operable to receive the UWB signal, the UWB signal including a channel impulse response (CIR), and a processor in communication with the transmitting node and the receiving node, the processor being programmed to receive the UWB signal at the receiving node, extract the CIR from the UWB signal received at the receiving node, calculate a velocity associated with the UWB signal based on the CIR, and estimate a respiration rate of the vehicle occupant based on the calculated velocity. [Brief description of the drawings]

[0006] [Figure 1A] 1 is an illustrative example of a vehicular system having an ultra-wideband (UWB) infrastructure and sensing system located within the vehicle. [Figure 1B] FIG. 2 is a diagram of an example of a communication system node according to an example embodiment of the present disclosure. [Figure 1C] FIG. 2 is a diagram of an example of a dual mode system node in accordance with an example embodiment of the present disclosure. [Figure 1D] FIG. 2 is a diagram of an example target device in accordance with an example embodiment of the present disclosure. [Diagram 2] 1 is an illustrative example of a channel impulse response (CIR) signal seen by a UWB node in a vehicular system. [Figure 3A] FIG. 2 is a diagram of an example set of system nodes including a dual mode system node associated with a vehicle in accordance with an example embodiment of the present disclosure. [Figure 3B] FIG. 2 is a diagram of another example set of system nodes including a dual mode system node associated with a vehicle in accordance with an example embodiment of the present disclosure. [Figure 4] FIG. 2 is an exemplary operational diagram of a vehicle system according to an example embodiment of the present disclosure. [Diagram 5] FIG. 2 is an exemplary operational diagram of a vehicle system according to an example embodiment of the present disclosure. [Figure 6] FIG. 2 is an exemplary operational diagram of a vehicle system according to an example embodiment of the present disclosure. [Figure 7] FIG. 2 is an exemplary operational diagram of a vehicle system according to an example embodiment of the present disclosure. [Figure 8] FIG. 2 is an exemplary operational diagram of a vehicle system according to an example embodiment of the present disclosure. [Figure 9] FIG. 2 is an exemplary operational diagram of a vehicle system according to an example embodiment of the present disclosure. [Figure 10] FIG. 2 is an exemplary operational diagram of a vehicle system according to an example embodiment of the present disclosure. [Figure 11] FIG. 2 is an exemplary operational diagram of a vehicle system according to an example embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] The embodiments of the present disclosure are described herein. However, it should be understood that the disclosed embodiments are merely examples, and that other embodiments may take various and alternative forms. The drawings are not necessarily to scale, and some features may be exaggerated or minimized to show the details of certain components. Therefore, specific structural and functional details disclosed herein should not be interpreted as limiting, but merely as a representative basis for teaching those skilled in the art to use the embodiments in various ways. As those skilled in the art will understand, various features illustrated and described with reference to any one of the drawings may be combined with features illustrated in one or more other drawings to produce embodiments not explicitly illustrated or described. The combinations of the illustrated features provide representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of the present disclosure may be desirable for specific applications or implementations.

[0008] As used herein, the terms "a," "an," and "the" refer to both singular and plural referents unless the context clearly dictates otherwise. As an example, "a processor" programmed to perform various functions may refer to one processor programmed to perform any and all functions, or to multiple processors collectively programmed to perform various functions. As another example, "a system node" configured to transmit UWB signals and receive UWB signals may refer to (1) a system node that transmits UWB signals and the same node that receives UWB signals, or (2) a system node that transmits UWB signals and a different system node that receives UWB signals.

[0009] With the continued advancement of wireless technology, people are now using a number of connected and personalized services. As the number of wireless systems and services increases, manufacturers are beginning to leverage those existing systems and services in ways other than those originally intended. For example, manufacturers are beginning to leverage radio frequency (RF) transceivers (such as WiFi) to track human movement through walls and behind closed doors. The use of WiFi to track movement is significantly different from its original use case of communicating data between electronic devices. By leveraging an existing system outside of its original intended use, manufacturers are able to reduce the need for additional hardware, which in turn reduces costs, saves space, and / or increases power savings.

[0010] In automotive applications, key fobs have become common to perform various functions. For example, when a user is located in the vicinity of the vehicle, the key fob may be used to automatically unlock the door. Or, when the user is located in the vehicle, the key fob may allow the user to start the vehicle with the push of a button. To perform these functions, the key fob wirelessly communicates with the vehicle to perform an authentication process. Currently, automotive manufacturers may rely on two types of radio frequency (RF) technologies. In passive entry system (PES) and comfort entry-go (CEG) applications, low frequency (LF) technology may be used for key fob proximity and location identification. In remote keyless entry, ultra-high frequency (UHF) technology may be used. However, conventional LF and UHF technologies may not be sufficient for further exploitation outside of this original intended application. For example, conventional LF and UHF technologies may not be able to perform detection of the user in the vehicle. As a result, additional systems may be required to perform such functionality. Also, LF and UHF systems are known to be subject to security breaches, including "relay" attacks.

[0011] As disclosed by U.S. Patent No. 10,573,104, which is incorporated herein by reference, ultra-wideband (UWB) systems have been disclosed and are operable to perform certain automotive functions such as vehicle access (i.e., keyless entry). UWB technology may be preferred over LF and UHF technologies because it may provide more robust functionality and improved security capabilities. It is believed that UWB systems may also be able to provide increased context awareness, safety, and security applications.

[0012] FIG. 1A is a diagram of an example of a vehicle system 100 having a UWB infrastructure that also provides vehicle access systems and on-board sensing, according to an example embodiment. In this embodiment, the system 100 includes multiple system nodes 110 located at various locations on the vehicle 10. It is understood that the particular number of system nodes 110 and the particular locations of the system nodes 110 will depend on the desired accuracy and performance, as well as the particular make and model of the vehicle 10. The system nodes 110 are configured to communicate with target devices 120 (which may be mobile and / or portable) to determine the relative position and / or location of the target devices 120. In an example embodiment, the system 100 may be configured such that certain system nodes are designated as master system nodes and other system nodes are designated as slave system nodes, and the master system node controls communication with the slave system nodes and collects data from the slave system nodes S for purposes of locating the target devices 120. Processing of the data collected from the system nodes 110 to locate the target devices 120 is performed by the master system node or processing system 130. In one example embodiment, processing system 130 is or includes an electronic control unit (ECU). In at least one embodiment, UWB communications are utilized between system nodes 110 and target devices 120 to enable their location determination. Each system node 110 is a communications system node 110A or a dual mode system node 110B. Either system node 110A or 110B may be referred to as a UWB system node, even though the system nodes may be configured for other types of additional communications.

[0013] FIG. 1B illustrates an example of a communication system node 110A according to one embodiment. In the illustrated embodiment, each communication system node 110A includes a processor 112A, a memory 114A, and a transceiver 116A. The memory 114A is configured to store program instructions that, when executed by the processor 112A, enable each communication system node 110A to perform various operations described elsewhere herein, including locating a target device 120 and sensing a predetermined area. The memory 114A may be any type of device capable of storing information accessible by the processor 112A, such as a writeable memory, a read-only memory, or other non-transitory computer-readable medium. Additionally, the processor 112A includes any hardware system, mechanism, or component that processes data, signals, or other information. The processor 112A may include a system having a central processing unit, multiple processing units, dedicated circuitry to accomplish functions, or other systems. In one example, the communication system node 110A includes a microcontroller that includes at least a processor 112A and a memory 114A along with programmable input / output peripherals.

[0014] The transceiver 116A includes at least a UWB transceiver configured to communicate with the target device 120, includes the capability to transmit and receive communication signals, and may include any of a variety of other devices configured to communicate with other electronic devices. In some embodiments, the transceiver 116A includes multiple UWB transceivers and / or multiple UWB antennas arranged in an array. In one example embodiment, the transceiver 116A includes at least one additional transceiver configured to communicate with other system nodes 110 (e.g., the communications system node 110A, the dual mode node 110B, etc.), the target device 120, and / or the processing system 130 via a wired or wireless connection.

[0015] 1C illustrates an example of a dual mode system node 110B according to one embodiment. The dual mode system node 110B is configured to switch between a UWB communication mode and a UWB radar mode. More specifically, in the illustrated embodiment, the dual mode system node 110B includes at least a processor 112B, a memory 114B, and a transceiver 116B. The processor 112B includes any hardware system, mechanism, or component that processes data, signals, or other information. The processor 112B may include a central processing unit, multiple processing units, dedicated circuits to achieve functionality, a digital signal processor (DSP), or a system having other processing technology. The memory 114B is configured to store program instructions that, when executed by the processor 112B, enable each system node 110 to perform various operations described elsewhere herein, including locating a target device 120, sensing a sensing area, switching between a communication mode and a radar mode, performing signal processing, etc. The memory 114B may be any type of device capable of storing information accessible by the processor 112B, such as a writable memory, a read-only memory, or other non-transitory computer-readable medium. In one example, the dual mode system node 110B includes a microcontroller that includes at least the processor 112B and the memory 114B along with programmable input / output peripherals.

[0016] The transceiver 116B includes at least a transceiver configured to switch between transmit / receive UWB communications and transmit / receive UWB radar, respectively. The transceiver 116B is configured to communicate with the target device 120, includes the ability to transmit communication signals and receive communication signals, and may include any of a variety of other devices configured to communicate with other electronic devices. In some embodiments, the transceiver 116B includes multiple UWB transceivers and / or multiple UWB antennas arranged in an array. The multiple UWB transceivers and / or multiple UWB antennas are configured to transmit / receive UWB communications and UWB radar, respectively. In one example embodiment, the transceiver 116B includes at least one additional transceiver configured to communicate with other system nodes 110 (e.g., communication system node 110A, dual mode system node 110B, etc.), the target device 120, and / or the processing system 130 via a wired or wireless connection.

[0017] The dual mode system node 110B is operable to switch between a communications mode and a radar mode, respectively. The dual mode system node 110B is also operable to transmit pulses in the radar mode and the communications mode, respectively. The duration of these pulses and / or the number of these transmitted pulses differ in these two different modes. For example, the one or more pulses generated in the radar mode differ from the one or more pulses generated in the communications mode in terms of pulse shape, repetition frequency, pulse power, number of pulses, duration of pulse transmission, any suitable pulse characteristics, or any number and combination thereof.

[0018] In one example embodiment, for example, the dual mode system node 110B provides a communication mode and a radar mode, respectively, and includes one or more switching mechanisms implemented via hardware, software, or a combination thereof configured to enable the dual mode system node 110B to switch between these two modes. As a non-limiting example, for example, the dual mode system node 110B may include a switch connected to an antenna and a radio integrated circuit (IC), which may be present in FIG. 1C but is not shown in this high-level block diagram. The switch controls whether the antenna is connected to at least one transmit circuit or at least one receive circuit. Additionally, the switch controls whether the antenna is connected to a radar receive circuit or a communication mode receive circuit. If the dual mode system node 110B has multiple antennas, the dual mode system node 110B may include a switch for each antenna to control its operation (e.g., transmit or receive), or a switch to select an antenna and a switch to enable operation (e.g., transmit or receive radar or receive communication).

[0019] As described above, the dual mode system nodes 110B are advantageously configured to selectively switch between a radar mode and a communication mode. More specifically, the dual mode system nodes 110B are configured to operate in a communication mode or a radar mode. For example, when in a communication mode, each dual mode system node 110B is enabled to contribute to on-board sensing throughout the vehicle 10 via UWB communications. Also, when in a radar mode, each dual mode system node 110B is operable to provide target sensing for a particular location (e.g., a seat). Additionally, the use of UWB radar contributes to providing health data (e.g., heart rate, respiration rate) of at least one living organism present within the vehicle 10, as further described herein.

[0020] 1D illustrates a non-limiting example of a target device 120, which may include a key fob, a smartphone, a smartwatch, or any suitable electronic device. In the illustrated embodiment, the target device 120 includes at least a processor 122, a memory 124, a transceiver 126, an I / O interface 128, and a battery 129. The memory 124 is configured to store program instructions that, when executed by the processor 122, enable the target device 120 to perform various operations described elsewhere herein, including communicating with the system node 110 for purposes of locating the target device 120. The memory 124 may be any type of device that may store information accessible by the processor 122, such as a memory card, ROM, RAM, a hard drive, a disk, flash memory, or other non-transitory computer-readable medium. Additionally, the processor 122 includes any hardware system, mechanism, or component that processes data, signals, or other information. The processor 122 may include a system having a central processing unit, multiple processing units, dedicated circuits for accomplishing functions, or other systems.

[0021] The transceiver 126 includes at least a UWB transceiver configured to communicate with the system nodes 110 (e.g., communication system node 110A, dual mode node 110B, etc.), includes the capability to transmit and receive communication signals, and may also include any of a variety of other devices configured to communicate with other electronic devices. In an example embodiment, the transceiver 126 further includes additional transceivers common to smartphones and / or smartwatches, such as a Wi-Fi or Bluetooth transceiver and a transceiver configured to communicate over a wireless telephone network. The I / O interface 128 includes software and hardware configured to facilitate communication with one or more interfaces (not shown) of the target device 120, such as tactile buttons, switches and / or toggles, a touch screen display, a microphone, a speaker, and a connection port. The battery 129 is configured to power the various electronic devices of the target device 120, and may include a replaceable or rechargeable battery.

[0022] In one example embodiment, processing system 130 is configured to control and monitor various electronic functions associated with vehicle 10. Processing system 130 may communicate with node 110 to operate the node and / or to process data received by the node. For example, node 110 may be used to perform automotive functions described herein, such as starting vehicle 10 when processing system 130 determines that target device 120 is located within vehicle 10. The system 130 may also be operable to perform the following functions: (1) detect the state of the vehicle 10 (e.g., whether the vehicle 10 is unoccupied or occupied, or whether the doors, windows, or trunk are open); (2) monitor vital signs of occupants in the vehicle 10 (e.g., heart rate, respiratory rate, or the emotional state of the user); (3) determine the occupancy of the vehicle 10 (i.e., count the number of creatures, humans, and animals in the vehicle 10); (4) detect human movement or activity in or near the vehicle; (5) detect occupancy when a driver / passenger approaches (or leaves) the vehicle 10; and (6) detect intrusion into the vehicle 10 while ensuring complete privacy. Based on these detections, various warnings may be triggered, for example, if there is a respiratory rate of an occupant while the driver is exiting the vehicle, the vehicle may warn the driver that the passenger is still in the back seat of the vehicle, thereby prompting the driver to check for other occupants in the vehicle before leaving it. In other embodiments, if the occupant's respiratory rate is above a threshold, it may indicate a particular mood or health of the occupant, and an appropriate message or warning may be delivered to the driver to alert the driver of the mood or health. The processing system 130 includes at least one electronic control unit (ECU). In one example, the processing system 130 includes a microcontroller and / or a microprocessor. In one example, the processing system includes at least a processor, a memory, and an I / O interface.The memory is configured to store program instructions that, when executed by the processor, enable the processing system 130 to perform various operations described elsewhere herein, including locating the target device 120, sensing one or more sensing regions, and using UWB radar to provide health data (e.g., heart rate, respiration rate, emotional state) of at least one living being in the vehicle 10. The memory may be any type of device capable of storing information accessible by the processor, such as a memory card, ROM, RAM, hard drive, disk, flash memory, or other computer readable medium. Additionally, the processor includes any hardware system, mechanism, or component that processes data, signals, or other information. The processor may include a system having a central processing unit, multiple processing units, dedicated circuits to accomplish functions, or other systems. The I / O interface includes software and hardware configured to facilitate monitoring and control of various electronic devices and their functions.

[0023] Various nodes 110 (including nodes 110A and / or 110B) may be operable as transceivers to transmit and receive UWB messages. One or more of the nodes 110 may periodically transmit (or blink) UWB messages. One or more of the nodes 110 may perform UWB-based sensing of vehicle status or occupancy vitals using channel state information, such as a channel impulse response (CIR) calculated by a given receiver. For example, FIG. 2 illustrates node 110B transmitting a UWB message that may be received by node 110A. Of course, either node 110A or 110B may be a different type of node as described herein, this is merely an example shown. As illustrated, the UWB message may be reflected at various points (indicated by τ1-τ5) around the vehicle 102.

[0024] The CIR of a UWB signal generally refers to the response of the wireless communication channel through which the signal travels. When a UWB signal is transmitted from one node and received by another, during transmission the signal encounters various obstacles and environmental factors that attenuate, scatter and reflect the signal. One of these obstacles can be a human occupant in a vehicle. The CIR describes how the channel behaves in the time domain. It represents the time-varying characteristics of the channel, including the delay, attenuation and phase shift that the signal undergoes while propagating from the transmitter to the receiver.

[0025] Graph 210 illustrates CIRs that may be calculated by node 110A based on reflected UWB messages. As illustrated, the CIR for τ1 may have the largest amplitude and the smallest time delay because it was not reflected by any point within vehicle 10 while being transmitted from node 110B to node 110A. Conversely, the CIR for τ5 may have one of the smallest amplitude and largest delay because it was reflected by a rear point (e.g., the trunk) of vehicle 10 before being received by node 118. The other CIRs τ2, τ3, and τ4 may have different amplitudes and delays based on their propagation paths and their reflections within vehicle 10.

[0026] 3A and 3B illustrate a non-limiting example of a set of system nodes on a vehicle 10, according to an example embodiment. FIGs. 3A and 3B illustrate an example having at least one communication system node 110A and at least one dual mode system node 110B. In this regard, FIGs. 3A and 3B illustrate a non-limiting example of a node arrangement on a vehicle 10. Additionally, FIGs. 3A and 3B include a non-limiting conceptual representation of a sensing region of the dual mode system node 110B in the form of a shaded triangle. The embodiments are not limited to these node arrangements, as several other node arrangements exist. FIGs. 3A and 3B also illustrate an example of a node arrangement in which UWB radar and UWB communications may be combined to provide sensing status data and / or sensing applications.

[0027] Figure 3A illustrates a first arrangement including a UWB communications system node 110A at a first location (labeled 1), a UWB dual mode system node 110B at a second location (2), a UWB dual mode system node 110B at a third location (3), and a UWB communications system node 110A at a fourth location (4). Figure 3B illustrates a second arrangement including a UWB communications system node 110A at a first location (1), a UWB dual mode system node 110B at a second location (2), a UWB dual mode system node 110B at a third location (3), and a UWB dual mode system node 110B at a fourth location (4). In this regard, the second node arrangement of Figure 3B differs with respect to the first node arrangement of Figure 3A in that the fourth location of Figure 3B includes a dual mode system node 110B, whereas the fourth location of Figure 3A includes a communications system node 110A. In this regard, the first node arrangement of Figure 3A is operable to provide rear seat sensing with the dual mode system node 110B in a second and third location, while the second node arrangement of Figure 3B is operable to provide driver seat sensing with the dual mode system node 110B in a fourth location and rear seat sensing with the dual mode system node 110B in a second and third location. In this regard, Figures 3A and 3B illustrate examples of how the dual mode system node 110B and the communication system node 110A may be strategically used together to generate sensor fusion data, thereby enabling a variety of sensing condition data to be generated to benefit a variety of in-vehicle sensing applications.

[0028] The UWB signals transmitted and received by the node 110 may include packets or messages such as those described in U.S. Patent Nos. 10,573,104 and 11,402,485, which are incorporated herein by reference in their entirety. In particular, the processor described herein is configured to process the CIR metadata to determine vital signs, such as, for example, the respiratory rate and breathing pattern of an occupant in a vehicle. The processing system may operate on given radio frequency (RF) signal properties (e.g., channel information during communication), such as the CIR metadata, to determine a given operating condition. In one embodiment, the CIR metadata provides diagnostic information that the processing system may use to determine whether a received RF signal is operating in a line-of-sight (LOS) or non-line-of-sight (NLOS) operating condition.

[0029] The system 100 may identify LOS and NLOS operating conditions by calculating the difference between a first path of the CIR metadata and a peak path location of the CIR metadata. It is contemplated that the difference between the first path and the peak path may be greater for NLOS conditions than for LOS operating conditions. The system 100 may also evaluate a confidence level of the CIR metadata to identify whether an LOS or NLOS operating condition exists. It is also contemplated that the system 100 may use the CIR metadata to evaluate the likelihood of an undetected early path to identify whether an LOS or NLOS condition exists.

[0030] It is contemplated that system 100 may identify an NLOS or LOS operating condition using only one of the CIR metadata (i.e., first and peak path indexes, probability of NLOS estimate, confidence level, or likelihood of undetected early paths). It is also contemplated that system 100 may combine one or more of the CIR metadata to more accurately identify whether an LOS or NLOS operating condition exists. Additional signal processing that may be implemented and performed by the processors described herein is disclosed herein.

[0031] In-vehicle sensing enables new features for in-vehicle entertainment as well as advanced safety and security features for the car. Understanding the health status of the driver and passengers can provide better and more intelligent in-vehicle assistance. Detecting the activity of living creatures inside the car can also prevent children from being left alone in the vehicle, which unfortunately happens frequently and can cause heatstroke and / or death. The basic objective of the present disclosure is for vital signs detection and estimation that can enable in-vehicle sensing applications such as: driver / passenger vital signs monitoring, driver / passenger health and emotional state estimation, driver / passenger stress level estimation and sudden illness detection, occupancy detection, child abandonment detection, object classification based on vital signs estimation.

[0032] The health detection concept described herein may leverage the UWB signal processing described herein. The system disclosed herein has two main features. First, the ability to detect and classify various types of human activity by processing UWB signals. Second, the ability to detect breathing, analyze breathing patterns, and estimate breathing rate. To implement such features, the system provided herein uses UWB radar to perform vital signs detection and estimation, the main system architecture of which is shown in FIG. 4, where system 400 may be implemented by or in conjunction with system 100 described above with reference to FIG. 1, as well as other systems described herein. At 402, the system transmits and receives UWB radar, for example, to or from node 110. UWB is a wireless technology that uses low energy and high bandwidth for short-range signal transmission and reception. In one embodiment, the UWB radar operates with a bandwidth of 500 MHz within the 6-8.5 GHz frequency band. At 404, a radar transmitter of one node transmits an impulse signal, and a radar receiver of the other node receives the impulse response as a complex channel impulse response (CIR). At 406, the raw CIR may be processed. Each CIR may be analyzed in a defined time domain. At 408, vital signs may be estimated based on the CIR signal processing. This may include respiration rate estimation and respiration pattern extraction.

[0033] FIG. 5 shows an operational flow diagram 500 of how the system 100 can be used to perform a velocity-based respiration detection method. In this embodiment, the system 100 utilizes a method based on velocity features calculated using raw radar CIR data, where the velocity features are used to identify the activity type of the object in front of the radar node. Once respiratory activity is identified, the amplitude changes of the velocity features are exploited over time to extract the respiratory pattern. The respiratory rate can be estimated using a Fast Fourier Transform (FFT) calculation on the respiratory pattern over time.

[0034] At 502, CIRs are generated periodically. The time between two consecutive CIRs is referred to herein as "slow time" and the sampling interval of each CIR is referred to herein as "fast time". Each sample within a CIR is referred to as a "tap". First, a 2D array may be constructed to include multiple CIRs in slow time, with each CIR having multiple taps in fast time. The design of the 2D array allows the system to analyze a single CIR across multiple taps and the changes of a particular tap across multiple CIRs.

[0035] In one embodiment, each raw radar CIR consists of 64 taps, and each tap of the CIR is 4 bytes. If the entire CIR is analyzed, it is 256 bytes. If a truncated CIR (e.g., 12-25 taps) is analyzed, it is 48-100 bytes.

[0036] At 504, a range-velocity profile is generated based on the raw radar CIR. When the vehicle occupant's chest moves up and down due to breathing, it moves at a certain speed. So here, the system is observing the speed of the movement indicated by the CIR. The range-velocity profile is also called range-Doppler profile and FFT profile because it is generated by applying a discrete Fourier transform to each tap of the CIR over slow time. This process generates a Doppler spectrum of each CIR tap over a given time domain. From the range-velocity profile, the system can obtain both distance (CIR taps in fast time) and velocity (Doppler spectrum in slow time) information. The system further truncates the range-velocity profile in both fast time and Doppler spectrum domains. For truncation in the fast time domain, the system selects taps within a certain distance range to obtain the return signal from the intended target. For truncation in the Doppler spectrum domain, the system uses positive frequencies. Each range-velocity profile is generated from consecutive CIRs buffered at the receiver / PC side. The resulting area-speed profile is stored in the receiving node's memory, or in the vehicle's storage, or in a remote computer's storage, and is updated periodically.

[0037] In one embodiment, each region-rate profile requires buffering 32-128 CIRs. In the worst case, the number of FFTs to process is 64, thus requiring 8.2-32.8 KB of memory. With truncated CIR, the number of FFTs to process is 12-25 FFTs, thus requiring 1.6-12.8 KB of reduced memory.

[0038] In one embodiment, the system may use a calibration method to remove static noise from the environment. During the calibration process, an average FFT profile from the static environment is generated. The average FFT amplitude and total energy of the Doppler spectrum for each tap of the CIR are generated from the calibration FFT profile. The system then uses this calibration FFT profile to perform clutter reduction of the incoming CIR. The calibration FFT profile is subtracted from subsequent FFT profiles generated from subsequent CIRs.

[0039] At 506, velocity features are determined based on each location point along the velocity profile. Individual velocity features for each point may be extracted from the CIR or region-velocity profile and stored in memory. After calibration (if provided) and region-velocity profile generation, the system calculates a vector of velocity features from each region-velocity profile. These velocity features may include the following: velocity average (e.g., average of all velocities for each CIR tap in the region-velocity profile), calibration velocity average (e.g., average of all velocities for each CIR tap in the region-velocity profile minus the average of all velocities in the calibration region-velocity profile), velocity energy (e.g., total energy (sum of squares) of all velocities for each CIR tap in the region-velocity profile), calibration velocity energy (e.g., total energy (sum of squares) of all velocities for each CIR tap in the region-velocity profile minus the total energy of all velocities in the calibration region-velocity profile), and calibration normalized velocity energy (e.g., calibration velocity energy minus the average over all CIR taps).

[0040] At 508, an activity classification model is executed. The purpose of the activity classification model is to identify the state of the person. In an example where it is desired to analyze the breathing of a person, the movement of the person may interfere with the breathing detection, and the goal of the system is to remove this other movement so that the system can focus on processing only data related to, for example, chest movement. Thus, first, the activity classification model may classify the person based on the overall movement of the person.

[0041] The activity classification model is primarily based on the analysis of the speed features from 506. Within each region-speed profile, the system selects the top N CIR taps with the maximum speed energy after calibration. In one embodiment, the number N is equal to 3 by default. Then, an average speed energy is calculated for the top N CIR taps. This average speed energy may be referred to as the activity level. The system further defines different thresholds for different activity classes based on different target environments. The activity classification is generated based on comparing the activity level with various thresholds.

[0042] In one embodiment, the various activity classes may include active, breathing, and still. For the active class, according to one embodiment, if the activity level is above an activity threshold (e.g., a lower threshold), the subject is classified as active. The active class means that the subject is moving around (above a motion threshold) or the environment contains excessive dynamic signals.

[0043] For the respiratory (or breath) class, according to one embodiment, if both of the following conditions are met: (1) the activity level is below the active threshold but above the respiratory threshold (e.g., the upper threshold), and (2) the difference between the maximum energy of all CIR taps and the average energy among all CIR taps is greater than a predetermined threshold.

[0044] For the static class, according to one embodiment, all other conditions not met by either the active or breathing class are considered to be static.

[0045] A classification result may be generated for each region-velocity profile and stored as a numerical value along with the activity level. This numerical value for the activity classification result is referred to as the activity tag. The activity tag values ​​for all region-velocity profiles in the region-velocity profile buffer may be stored in a buffer, such as a first-in-first-out (FIFO) buffer. On the other hand, if the activity tag is identified as respiration, a list of dominant taps is added to which the index of the tap with the maximum energy among all CIR taps in the region-velocity profile is added. To make a prediction of the activity type over a longer period of time, the activity tag values ​​of the individual region-velocity profiles are averaged over various time intervals. For example, a sample decision flow chart shows that in one embodiment, if the average of the activity tag value over the last 10 seconds is within two predefined thresholds, the respiration tag is set to true, if the average of the activity tag value over the last 5 seconds is outside another predefined set of thresholds, the respiration tag is reset to false, and if the average of the activity tag value over the last 3 seconds is outside yet another predefined set of thresholds, the respiration tag is also reset to false.

[0046] In one embodiment, the system may then proceed to select a dominant tap. The concept here is that each tap is one particular value, and each tap indicates one distance away from the sensor. If there is noise or other motion in the distance, it can be removed since it is not in the dominant tap. For example, if the respiration tag is true for the current region-velocity profile buffer, and the minimum number of region-velocity profiles in the buffer are classified with the respiration activity tag, then the most frequently occurring CIR tap index in the list of dominant tap indexes is selected. This tap, along with the two adjacent taps, is finally considered as the dominant tap and is used for further respiration pattern extraction and respiration rate estimation.

[0047] At 510, breathing patterns associated with the target occupant are extracted from the wireless signal. Typically, the system relies on velocity over time to obtain an average breathing pattern or rate. In one embodiment, the velocity energy for the three dominant taps is extracted from each region-velocity profile in the buffer. Over slow time, an average over these three taps is calculated and recorded. This gives the person's breathing pattern. The dominant taps are also used to calculate the distance from the target to the UWB radar sensor.

[0048] At 512, the system estimates the target occupant's respiration rate. In one embodiment, the system applies a bandpass filter to the extracted breathing pattern to remove DC and high frequency components. The filtered breathing pattern is then transformed into the frequency domain using an FFT function. The frequencies at which the three frequency spectra reach their respective maxima are found, and the average of these three frequencies represents an estimate of the target's respiration rate.

[0049] The above techniques allow processing of UWB radar signals to identify the rate of movement (e.g., chest movement) of a target vehicle occupant in order to classify the activity of the target vehicle occupant and estimate the occupant's respiratory rate. The use of UWB radar signals allows these calculations to be performed with less complexity and at lower cost compared to other methods, such as processing of frequency modulated continuous wave (FMCW) radar. These teachings may also be utilized to identify the presence of a child in the rear seat of a vehicle, the respiratory rate of a baby, etc.

[0050] 6 illustrates an operational flow diagram 600 of how system 100 may be used to perform a phase-based respiration detection method, according to one embodiment. Unlike the embodiment of FIG. 5, in this embodiment herein, system 100 utilizes a method based on phase features of the raw CIR data of the UWB radar signal, where the phase features of the signal are used to identify the activity type of a subject in front of the radar node, the subject's respiration rate, etc., as described above.

[0051] At 602, the system receives raw CIRs as input, similar to 502 in Figure 5. A set of input CIRs arranged in fast and slow time is received and formatted as a 2D array, which the system uses as input to a phase-based method.

[0052] At 604, the system performs phase unwrapping. According to one embodiment, to obtain a continuous phase variation pattern, the system performs phase unwrapping by adding or subtracting appropriate multiples of 2π to remove discontinuities in the phase signal.

[0053] At 606, the system performs CIR truncation. CIR truncation refers to the process of limiting or clipping the length of the CIR to the number of CIR taps of a discrete sample size. To remove noise from distant reflections, the system truncates the CIR in the fast-time domain. In one embodiment, truncation is performed by selecting a chunk of CIR taps that may encompass all multipath components of the target environment. Again, respiratory activity detection and subsequent activity classification and activity prediction output (e.g., step 508) described above with reference to FIG. 5 may be similarly performed in FIG. 6.

[0054] At 608, the system performs noise reduction where the system removes static components in the target environment.

[0055] At 610, the system performs Principal Component Analysis (PCA). The breathing pattern of the target person may be obtained from multiple CIR taps due to multipath effects in the target environment. To build a robust system, the phase analysis performed is not based on a single tap of the CIR at a suitable distance from the target. Instead, the system uses all the CIR taps after truncation and performs PCA transformation to obtain a clean signal.

[0056] In an embodiment, the PCA transformation is described as follows: First, the system identifies the hyperplane that is closest to the data points. Then, the system selects the axis with the largest variance as the principal components. In one embodiment, the system uses a standard matrix factorization technique called singular value decomposition (SVD) to obtain the principal components.

number

[0057] At 612, the system applies a bandpass filter to remove DC and high frequency components in the slow time.

[0058] The result is a respiratory wave at 614. This can be visualized as a stream of data points rising and falling as breathing causes the target's chest to move up and down. The data points associated with this movement result in body movement. At 616, the system can separate the data points into breaths. In one embodiment, the inflection points of the data points represent the moment when a person stops inhaling and starts exhaling, or stops exhaling and starts inhaling. These can be the peaks and valleys of the respiratory wave. This also allows the system to perform a respiratory rate estimation at 618, which is based on the distance between the peaks or valleys over time. The respiratory rate estimation 618 can be similar to 512 of FIG. 5.

[0059] 7 shows an operational flow diagram 700 of how system 100 may be used to perform a hybrid rate-based and phase-based breath detection method, according to one embodiment, where region-velocity profile 504 and velocity features 506 are used for activity detection, classification and prediction output 508 as described in FIG. 5 and are provided to the system, for example, prior to performing phase unwrapping at 604. In this embodiment, the system utilizes a rate-based method to detect activity and identify the presence of a living being, and then the system continues with a phase-based approach to identify the breathing pattern and estimate the breathing rate of the detected living being.

[0060] It should be understood that the methods and systems described herein may rely on machine learning (e.g., classification learning and deep learning). For example, FIG. 8 illustrates an operational flow diagram 800 of how the system 100 may be used to perform machine learning based activity classification, where machine learning (ML) and / or deep learning (DL) models are utilized for activity classification, according to one embodiment. Machine learning may be performed on non-static data (e.g., active and respiration) as described above, and the final predicted activity may rely on the ML / DL output along with static data from activity detection. It is contemplated that the machine learning algorithm may use known machine learning classification algorithms such as linear classifiers, support vector machines, decision trees, boosted trees, random forests, neural networks, or nearest neighbors. In FIG. 8, the ML based activity classification uses velocity information (e.g., signal processing) while ML (e.g., DL) is used for activity classification. FIG. 9 illustrates another embodiment in which ML is used on phase-based data, and activity detection uses phase information (e.g., signal processing) while ML (e.g., DL) is used for activity classification. FIG. 10 shows another embodiment in which an ML-based architecture is used with both velocity and phase information, and activity detection is performed (e.g., signal processing) using both velocity and phase information while ML is used for activity classification.

[0061] FIG. 11 shows an overview of the system architecture of the system 100 for real-time implementation of the system, according to one embodiment. Data acquisition and signal processing can be implemented as daemon modules running, for example, on a PC or embedded system. A main application process creates multiple sub-daemon threads for data buffering and signal processing. At 1102, an input CIR is received from a UWB radar device. At 1104, a raw data update buffer thread stores the CIR in a (e.g., fixed-length) FIFO buffer. New input data may overwrite old data. At 1106, an FFT profile update buffer thread retrieves multiple CIRs from the raw data update buffer, performs FFT calculations for each tap of the CIR over slow time, and stores the FFT profile in a (e.g., fixed-length) FIFO buffer. At 1108, a velocity-based processing module performs velocity-based signal processing as described herein, and at 1110, a phase-based processing module performs phase-based signal processing as described herein. At 1112, vital signs detection is performed on the raw data update buffer and the FFT profile update buffer, and the vital signs results are stored in a FIFO buffer.

[0062] The various steps and logic performed herein may be performed using non-volatile storage devices, memories, and processors. Non-volatile storage devices may include one or more persistent data storage devices, such as hard drives, optical drives, tape drives, non-volatile solid state devices, cloud storage, or any other device that may persistently store information. Processors may include one or more devices selected from high performance computing (HPC) systems including high performance cores, microprocessors, microcontrollers, digital signal processors, microcomputers, central processing units, field programmable gate arrays, programmable logic devices, state machines, logic circuits, analog circuits, digital circuits, or any other device that manipulates signals (analog or digital) based on computer executable instructions present in memory. Memory may include a single memory device or several memory devices, including but not limited to random access memory (RAM), volatile memory, non-volatile memory, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, cache memory, or any other device that may store information.

[0063] Although exemplary embodiments have been described above, these embodiments are not intended to describe all possible forms encompassed by the claims. The words used herein are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the present disclosure. As previously mentioned, features of various embodiments may be combined to form further embodiments of the present invention that may not be expressly described or illustrated. Although various embodiments may be described as offering advantages or being preferred over other embodiments or prior art implementations with respect to one or more desired characteristics, those skilled in the art will recognize that one or more features or characteristics may be compromised to achieve desired overall system attributes that depend on the particular application and implementation. These attributes may include, but are not limited to, cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, maintainability, weight, manufacturability, ease of assembly, and the like. Thus, to the extent that any embodiment is described as being less desirable than other embodiments or prior art implementations with respect to one or more characteristics, these embodiments are not outside the scope of the present disclosure and may be desirable for the particular application.

Claims

1. 1. A method for estimating vital signs of a vehicle occupant using ultra-wideband communications, comprising: The method comprises: Transmitting an ultra-wideband (UWB) signal from a first UWB system node in the vehicle; receiving the UWB signal at a second UWB system node within the vehicle; Calculating a channel impulse response (CIR) associated with the UWB signal received at the second UWB system node; calculating a rate associated with the UWB signal based on the CIR; classifying the vehicle occupant activity as respiration based on the calculated rate falling between a lower threshold and an upper threshold; estimating a respiratory rate of the vehicle occupant based on the calculated rate; and A method comprising:

2. generating a region-speed profile associated with the CIR, the speed characteristics being calculated based on the region-speed profile; The method of claim 1 further comprising:

3. calculating a vector of rate characteristics associated with the UWB signal, the vector of rates including (1) the calculated rate associated with the UWB signal and (2) an average of rates for each CIR tap within the region-rate profile; The method of claim 2 , further comprising:

4. The method of claim 1 , wherein the lower threshold is a respiration threshold and the upper threshold is an activity threshold.

5. The CIR includes a number of CIR taps, The method further includes selecting a dominant tap of the CIR taps, the dominant tap representing one of the CIR taps having a higher rate energy than the remaining CIR taps; The method of claim 1 , wherein the respiration rate is estimated based on the rate energy associated with the dominant tap.

6. Extracting a breathing pattern from the rate energy associated with the dominant tap. The method of claim 5 further comprising:

7. Triggering a respiratory rate alert for the vehicle occupant to an operator of the vehicle. The method of claim 1 further comprising:

8. The method of claim 1 , wherein the first UWB system node and the second UWB system node are similar UWB system nodes.

9. performing phase unwrapping of phase features associated with said CIR; truncating the CIR to select a chunk of CIR taps; performing a principal component analysis (PCA) transform on said chunk of CIR taps to project the CIR data onto a hyperplane; applying a band pass filter to the projected CIR data to remove high frequency components associated with the projected CIR data, said applying producing filtered projected CIR data; and estimating the respiration rate of the vehicle occupant based on the filtered projected CIR data; and The method of claim 1 further comprising:

10. 1. A method for estimating vital signs of a vehicle occupant using ultra-wideband communications, comprising: The method comprises: Transmitting an ultra-wideband (UWB) signal from a first UWB system node in the vehicle; receiving the UWB signal at a second UWB system node within the vehicle; Calculating a channel impulse response (CIR) associated with the UWB signal received at the second UWB system node; performing phase unwrapping of phase features associated with the CIR; truncating the CIR to select a chunk of CIR taps; performing a principal component analysis (PCA) transform on the CIR taps to project the CIR data onto a hyperplane; applying a band pass filter to the projected CIR data to remove high frequency components associated with the projected CIR data, said applying generating filtered projected CIR data; estimating a respiratory rate of the vehicle occupant based on the filtered projected CIR data; and A method comprising:

11. and subsequent to said truncating, denoising said chunk of CIR taps to remove static components associated with UWB signals reflecting within said vehicle. The method of claim 10 further comprising:

12. The performing of the PCA transformation comprises: identifying the hyperplane as the one that is closest to the CIR data; selecting the axis having the greatest variance in the CIR data; projecting the CIR data associated with the axis onto the identified hyperplane; The method of claim 10, comprising:

13. The method of claim 12 , wherein the selecting the axis comprises using a singular value decomposition (SVD) technique.

14. calculating a rate associated with the UWB signal based on the CIR; classifying the vehicle occupant activity as respiration based on the calculated rate falling between a lower threshold and an upper threshold; estimating the respiratory rate of the vehicle occupant based on the calculated rate; and The method of claim 10 further comprising:

15. generating a region-speed profile associated with the CIR, the speed characteristics being calculated based on the region-speed profile; The method of claim 14 further comprising:

16. calculating a vector of rate characteristics associated with the UWB signal, the vector of rates including (1) the calculated rate associated with the UWB signal and (2) an average of rates for each CIR tap within the region-rate profile; The method of claim 15 further comprising:

17. 1. A system configured to estimate vital signs of a vehicle occupant using ultra-wideband communications, comprising: a transmitting node operable to transmit an ultra-wideband (UWB) signal; a receiving node operable to receive the UWB signal, the UWB signal including a channel impulse response (CIR); a processor in communication with the sending node and the receiving node, receiving the UWB signal at the receiving node; extracting the CIR from the UWB signal received at the receiving node; Calculating a rate associated with the UWB signal based on the CIR; Estimating the breathing rate of the vehicle occupant based on the calculated rate. a processor programmed to A system including:

18. The processor, 20. The system of claim 17, further programmed to utilize a machine learning model to classify the vehicle occupant activity as breathing based on the calculated rate.

19. The processor, performing phase unwrapping of phase features associated with said CIR; truncating the CIR to select a chunk of CIR taps; performing a principal component analysis (PCA) transform on said chunk of CIR taps to project the CIR data onto a hyperplane; applying a band pass filter to the projected CIR data to remove high frequency components associated with the projected CIR data to generate filtered projected CIR data; Estimating the respiration rate of the vehicle occupant based on the filtered projected CIR data. The system of claim 17 further programmed to:

20. The performing of the PCA transformation comprises: identifying the hyperplane as the one that is closest to the CIR data; selecting the axis having the greatest variance in the CIR data; projecting the CIR data associated with the axis onto the identified hyperplane; 20. The system of claim 19, comprising:

Images